Messages

If you are a Canon or Adobe Executive, Developer, Programmer, or Representative, please come in and address Magic Lantern and the community of users.  We know you come here and see what is going on and we would all appreciate your input, suggestions, comments or anything else you have to say.  As Magic Lantern is now a very well known advanced tool for Canon owners from enthusiasts to top professionals, we would like to know what you think. 

We welcome all of you to join us.

What do you think about this?
https://bitbucket.org/escho/escho-magiclantern/branch/cold-pix-fix
Edgar

It's a dead link

Good news!!! 

Okay, I ran a huge battery of tests today using this recent build magiclantern-v2.3.NEXT.2014Feb14.60D111 (The Valentine's Day build). My card is a Sandisk Extreme Pro 95 MB/s (SD card).  Pink frames are listed in their frame number(s).

I started in sRAW...

Canon menu: sRaw, 1920x1080 24

(raw_rec.mo) Canon menu: mRAW
24 fps override
Shot 1  PinkFrame 36            (141 frames)
Shot 2  PinkFrame 11,71            (169 frames)
Shot 3  clean                        (169 frames)
Shot 4  clean                       (170 frames)
Shot 5  clean                       (173 frames)
Shot 6  PinkFrame 47,137,167         (170 frames)
Shot 7  clean                       (169 frames)
Shot 8  clean                       (168 frames)
Shot 9  PinkFrame 1,5         (170 frames)
Shot 10 PinkFrame 1         (173 frames)

This was giving me a headache seeing all these pink frames still    so I switched into mRAW in the Canon menu and things got a little better.

(raw_rec.mo) Canon menu: mRAW
FPS override - 23.976 FPS
Shot 1  clean                    (197 frames)
Shot 2  clean               (211 frames)
Shot 3  PinkFrame 117      (214 frames)
Shot 4  clean               (212 frames)
Shot 5  clean               (211 frames)
Shot 6  PinkFrame 45,105   (213 frames)
Shot 7  PinkFrame 71      (214 frames)
Shot 8  clean               (212 frames)
Shot 9  PinkFrame 193      (210 frames)
Shot 10 clean             (211 frames)

(raw_rec.mo) Canon menu: mRAW
FPS override - 24 FPS
Shot 1  clean            (197 frames)
Shot 2  PinkFrame 2,139         (211 frames)
Shot 3  clean               (212 frames)
Shot 4  clean               (211 frames)
Shot 5  clean               (214 frames)
Shot 6  clean               (213 frames)
Shot 7  clean               (213 frames)
Shot 8  clean               (214 frames)
Shot 9  clean               (213 frames)
Shot 10 clean               (179 frames)

With mRAW and MLV mode the results are dramatically improved!  Only one pink frame out of 2114!
MLV/ (mlv_rec.mo) Canon menu: mRAW
FPS override - 23.976 FPS
Shot 1  clean            (207 frames)
Shot 2  clean               (212 frames)
Shot 3  clean            (212 frames)
Shot 4  clean               (211 frames)
Shot 5  clean               (212 frames)
Shot 6  clean            (213 frames)
Shot 7  PinkFrame 1         (212 frames)
Shot 8  clean               (212 frames)
Shot 9  clean            (212 frames)
Shot 10 clean            (211 frames)

MLV/ (mlv_rec.mo) Canon menu: mRAW...  The sweet spot!  A perfect score! 2113 frames and no pink frames!  Is it the 60D Holy Grail?
FPS override - 24 FPS
Shot 1  clean                    (213 frames)
Shot 2  clean               (212 frames)
Shot 3  clean            (212 frames)
Shot 4  clean               (211 frames)
Shot 5  clean               (211 frames)
Shot 6  clean            (212 frames)
Shot 7  clean            (212 frames)
Shot 8  clean               (212 frames)
Shot 9  clean            (206 frames)
Shot 10 clean            (212 frames)

In full RAW 5184x3456 with mlv_rec.mo (MLV mode) it got even better and I got full takes without any issues.
24 FPS override NO PINK FRAMES!
23.976 FPS override NO PINK FRAMES!
I can get sustained 210-217 frames per shot. (8-9 seconds)

I think three key settings I changed from previous attempts were...
Auto power off - off (was on)
Auto preview - off (was set at 2 seconds)
sRaw 2592x1728 (changed to mRaw 3888x2592 and RAW 5184x3456))

MLV MODE in mRAW and RAW is what works on my 60D right now, 10 out of 10 shots and no pink frames or any other kind of glitches.  I can shoot in RAW!!! 

Here are all my pertinent settings both in ML and the Canon camera menus in a text document...

www.dueymoore.com/raw_settings.txt

Okay next test, I used raw2dng_escho.exe and got clean dng files (not cdng).  As those won't open in PPro, they do open in After Effects so I loaded them up in AE.  No funky pixels.  My camera is fine I now understand.  Keep on going, testing, winning!

I did indeed try that one just now and I can open the dng files in After Effects (with ACR popup first) but they don't open in PPro, which of course is what is needed.  In AE the files are clean, no funky pixels!  Sadly, they don't work in PPro.

Okay, thank you very much a1ex for clarifying, and I think now that I only have them at the first frame does indicate a pattern which may be helpful for you.  Before I was getting these pink frames in the middle and end of clips so you are on the right track, now it is only at the first frame and all other frames are clean. 

I'll shoot a set of 10 each MLV/RAW tomorrow and see what percentage of pinkies I get.  If you need my help with more 60D specific testing let me know, I'm glad to help.  Right now I can get sustained 7 second clips on my 60D and only first frame corruption in 1280x720 which scales up just fine.  And of course with frame skipping on I get continuous recording (I shot for 90 seconds before turning record off, but of course I had lots of skipped frames.

So if I convert with raw2dng I won't see these cold pixels?  Hmm... I thought I tried that.  Let me check.  But those files don't work in PPro either (dng) only the Cdng files work if I try to import them.

Okay, I just ran another set of shots using this build (magiclantern-v2.3.NEXT.2014Feb14.60D111) ...



RAW takes 1 & 5 I got pink frames, but only on frame 1.
MLV takes 2, 3, & 5 I got pink frames, again, only on the first frame. 

I also noticed the glitchy frames are ONLY in the first frame, and all look very similar, like a pattern!    a1ex, you may also want to reference the pixel bug issues being discussed on this thread as well.  http://www.magiclantern.fm/forum/index.php?topic=5618.new#new

I don't know if that will be useful for you but I have been doing extensive testing with raw2Cdng and have these weird buggy pixel issues, but only in RAW/MLV converted files.

Thanks Malex.  I get them with both, MLV and RAW.

[EDIT: Sorry a1ex, I added it to my profile.] 

I tested today also with the Feb. 12th build [EDIT - used the wrong version] and now I get pink corrupted frames on all takes (usually within the first 5 frames).  MLV

A couple of more questions...  I cleaned my sensor, reshot RAW/MLV then took out my ML card and loaded a normal SD card (Sandisk 32gig 45mb/s') and then shot some plain old H.264 clips.  I loaded the new RAW/MLV clips into the timeline in PPro and marked the offending pixels locations with yellow circles (zoomed in 400X).  In all RAW/MLV files they appear in the same location.






If I load the same SOURCE MLV/RAW files into PPro with Drastic Media Reactor's plugin I get the same pattern but the pixels look entirely different (pluses and colored squares).



The same pattern appears in any RAW/MLV conversion or source MLV/RAW file dropped into PPro via CDNG conversion or DMR's plugin. 

Now the part I don't understand.  Next, I loaded the regular old Canon video H.264 clip and pixel peeped it at 400X also and I see no funky pixels at all, anywhere!  Neither in the marked positions that the MLV/RAW files show them or anywhere else in the H.264 clips.  My question is, if my camera has bad/dead/hot/other messed up pixels then why don't they show up in H.264 footage? 

Hi Duey101
This is something we have talked about.  There is a discussion happening on this thread:
http://www.magiclantern.fm/forum/index.php?topic=10401.0

I know, I started that thread.     I have a question for you, I am testing DMR in PPro and also DP.  Whenever I open Drastic Preview this is how it looks on my screen...



I'm running
Toshiba Satellite
Win 7 Ultimate x64 (SP1)
8 gigs ram
7200 rpm HD
GeForce GT 540 M GPU

Hmm... I was just experimenting with how to make a poll and clicked the save by mistake and then you can't edit it after that (time wise).  I was going to make it for 30 days.  Even in one day with the answers it is 75% yes. 

I just discovered my sensor was filthy!  So I gave it a good cleaning with Pec pads and Eclipse so now it is clean, I'll try to shoot some more RAW and see if that had anything to do with the results I got.

chmee: you get my RAW files?

In the case with my 60D I think these corrupted pink frames as I posted earlier in the thread come from too slow of read/write speed for the camera itself so the frames get corrupted.  This happens on 3-10 takes so far for me on 1280x720 and my max frames are 400-415 so far in MLV mode and up to 15 seconds in RAW mode, and that is probably the best the 60D can do.  I'm going to have to save and get a 5D III!

Truthfully, I thank every individual on this forum. 

When this gets sorted out it will be amazing!  Working with what I have right now from raw2cdng is very nice in PPro!  Glitch free playback and scrubbing, very smooth on my Toshiba with 7200 rpm HD, 8 gigs ram, GEForce 540M GPU (which I tricked PPro into thinking it's a CUDA card).  Very good work flow and fast.

Thanks to chmee and tonybeccar!   

when they right click on a cdng sequence and they go to Source Settings, it opens in ACR and they can tweak things in there, close it and it updates on Premiere. Dreamy! Now.. I cannot seem to do it with your cdngs, the option is greyed out

The same for me, this option is greyed out.  You can see in this R3D tutorial http://vimeo.com/52037147 that he uses this option of source settings in PPro. 

I don't get why the DNG's came out without bad pixels but the TIFF's did?  I just ran this program http://www.starzen.com/imaging/deadpixeltest.htm on RAW files I took with the lens cap on, VF covered, ISO 100, and 1,2,5,15 second shots.  Converted those to uncompressed TIFF's in LR5.  Then tested those TIFF's with that Dead Pixel Test program at the default threshold setting of 60 and then halved that to 30.  No issues,  0 dead pixels / 0 hot pixels.

Anyone know a better way to test for dead pixels?  I don't see any in any images I shoot in RAW/JPG.

CR2's sent. 

[ONLY]

One more test, with the new version of MLVConverter 1.8.2....
http://www.magiclantern.fm/forum/index.php?topic=10198.0

In this test the output dng file is clean, but the tiff proxy file from the exact same conversion shows the same dead/cold/pixels/Sensels but ONLY in the proxy tiff files, not the dng file output.  They are also red and not black.  Any ideas on what is happening?  Very odd indeed.

DNG output...



TIFF output...

I have exactly the same problem with my 60D...

Sandisk Extreme 95mb
MLV video
1280x720
1920 24p (in Canon menu)
Cropped mode,
16:9 aspect ratio,
fps override 24,
Exact fps,
Frame skipping OFF
Global Draw OFF
Image Quality = Raw/sRaw
Auto Lighting Optimizer = OFF
Long exp. noise reduction = OFF
High ISO speed NR = OFF
Highlight tone priority = OFF
Multiple Exposure = Disable

[THEN]

Okay, after another battery of tests here is the result.

raw2dng - no issues
raw2CDNG convert raw file, no issues, convert mlv file no issues.
mlv2dng - no issues.

All the files are clean including the 10X crop.  Now...

If I take the raw2CDNG file and open it in PPro, drag it to the project window, open it in the source monitor, blow it up to 400%... THEN I see the dead/black/sensel issues.  Look here...



But outside of PPro these pixels from the same DNG/CDNG files are not present.  I sent you the converted files in email.  I don't know with the mlv2dng or raw2dng files cause they don't open in PPro (because they are not Cdng I assume).  EDIT: And if I output the sequence from PPro the black/dead/sensels show up in the output video, but oddly enough not in the files outside of PPro.

I just ran Rawdigger and used a picture of a white wall (off white) and then blew it up in Rawdigger.  Looks like this...



Other than these wht/grey colors there are no black/hot/weird looking pixels, they all look like this so I think my camera is fine, no dead pixels or other colored pixels at all.

Let me test the other options and get back to you.

I loaded a raw CR2 file and didn't see any deal pixels so what does that mean?  Why do they show up in the RAW/MLV converted CDNG's?  I don't get it. 

Okay that's good to know and thanks for the education.  When that dead pixel shows up in the rawdigger does that mean my sensor actually has a dead pixel in it?!!!    Or is is that during the recording in raw a dead pixel happens?  I hope my sensor isn't being damaged.  Still new to all this technical stuff.

Those shots with the color banding were shot in 5X crop mode so I get how that would effect the image.  Overall, when I shoot raw on my 60D I can get 400-420 frames in MLV mode and about 7-10 seconds in RAW mode @1280x720 which scales up quite well to 1920x1080.

As far as the pink corrupt frames not sure what the cause of that is.

Thank you very much David, your tutorials are great, simple and easy to understand.  Great contribution!

Yes, that good old grease, gotta love it.  Makes things move really well.  Well, everything is advancing so fast in the modern world I know a great solution is on its way right now.  The best thing of course is Adobe adds native MLV/RAW support, could happen.  I wonder why they never drop in here, you know they are all looking!