PinkDotRemover tool 650D

[kashkam]

Hi guys, i've been desperately trying to get this RAW footage sorted for my 650d, spent all night installing it on to my 650d, filmed some RAW footage, which I really liked, I'm going out of my mind, trying to fix the pink dots issue, I'm sorry if I'm being extremely stupid, but it is possible for someone to give me a short run through on how I use RawTherepee and/or install/use the PinkDotRemover, I am having an absolute nightmare, as I just can't seem to work it out, after numerous attempts at it,

Any help would be greatly appreciated, a simple step by step instruction, may be beneficial to many fellow videographers who wish to take advantage of these excellent tools that have been developed.

Thank you in advance,

Kash

[Rewind]

spent all night installing it

Holy cr2! ))

Well, I assume you're running windows:

1. Go to java.com and install the runtimes if you don't have them already;
2. Download PinkDotRemover (this is my modified bild which has better interpolation for now, less artifacts), extract the contents wherever you want;
3. Go inside extracted folder and just double-click the PinkDotRemover.jar — if java installed correctly, double-click from windows explorer will execute the program;
4. Jast drag and drop your *.RAW files onto the PDR's window and hit “Convert Files”

That's it! Wait for a few seconds, then process your raw files the way you used to (extract the dngs directly by raw2dng, or use Rawanizer, Batchelor etc.)

[maxotics]

Rewind, Mixer2

Neither of your versions of PDR seem to be removing pink dots form my EOS-M in crop mode anymore.  I think Gary might also be stuck.  Any thoughts?  I have a new build from 1%, which is about a week old.    Mixer2, your version will process the files (though pink dots stay).  Rewind, your version doesn't process; says they are corrupted.  I'm using a Sigma 10-20mm EF on the EOS-M.  Was working a couple of weeks ago just fine. 

[Rewind]

Rewind, Mixer2

Neither of your versions of PDR seem to be removing pink dots form my EOS-M in crop mode anymore.  I think Gary might also be stuck.  Any thoughts?  I have a new build from 1%, which is about a week old.    Mixer2, your version will process the files (though pink dots stay).  Rewind, your version doesn't process; says they are corrupted.  I'm using a Sigma 10-20mm EF on the EOS-M.  Was working a couple of weeks ago just fine.

Have you tried this dot data file?
http://www.magiclantern.fm/forum/index.php?topic=3648.msg73461#msg73461

[maxotics]

I copied that to you build, but didn't work.  Copies it to Rewind2's, ran, but pink dots still there.  BTW, 1% has gotten higher resolutions in crop mode so imagine that's going to create more need for PDR.

http://www.magiclantern.fm/forum/index.php?topic=3648.msg73840#msg73840

In the meantime, what is the best alternative method to remove pink dots?  Is there one?  I'm on a PC.  Thanks!

[Rewind]

Well, I have no EOS-M, and I'm not interested in crop mode recordings, so I can't help you atm.
But I would suggest you to take two DNGs from old build and from the new one, compare the positions of af dots and make a corresponding changes to dot data file.
Or just wait for foorgol or mixer2, when they hopefuly figure the problem out (seems like something changed with centering of the crop box)

[maxotics]

Thanks for your help Rewind!  Much appreciated! 

[maxotics]

I've figured out how to open the DNGs and get pixel locations of the pattern, but have no idea how they match the config files.  Are those x/y coordinates?  Are they the center lines?  I do notice that a DNG form a video that works does not match the one that doesn't currently work.   I'm shooting EOS-M, 1280x720 crop mode.  Used to work.  Thanks!

[maxotics]

I'm not sure if any of the devs are following PDR anymore, so I wrote this in EOS-M, which is the camera I use, and in which I have  come up against these limitations.

http://www.magiclantern.fm/forum/index.php?topic=3648.msg73969#msg73969

[Rewind]

Temporary solution for EOS-M crop mode:
http://www.magiclantern.fm/forum/index.php?topic=3648.msg74097#msg74097

[maxotics]

Devs?  Rewind?

If I want to open DNGs created from video RAW and find the pixels that the camera uses for auto-focus, the pixels that create "pink dots", which attributes of each pixel best identify them?  Or, is there are definitive attribute that can be used?  My plan at moment is to use some low-grade statistical analysis to distinguish them from neighboring pixels that also seem to be hot.  That plan doesn't excite me   Thanks!

When PDR was down I began looking into a hot-pixel mapping solution that would output a bad pixel file for dcraw which could be used to remove pink dots.   Don't know if you know, but PDR wasn't working after 1% latest (greatest!) release.   Thanks to REWIND again for saving the day! 

Rewind, if we can get such a process it would at least give you a clean bad-pixel file from which to work if we need you to put on your superman cape again

[coco_net]

Sorry, I can't help you with LV or similar. I have absolutely no clue what happens inside ML and how data is converted back and forth inside ML before being dumped to the card. I'm not an ML developer.

But regarding your idea to integrate the dot removal into ML:
From an architectural point of view, I would not do that. Writing the RAW data in higher resolutions is already time critical enough today (okay, mostly limited by writing speed, AFAIK). So I would push as much data processing as possible out of ML and up to your normal workstations with practically unlimited processing power, time and disk space. You have to touch all the image files anyway for post-processing, so adding the dot removal is not so much of a pain here.

Dot removal inside ML would only make sense if the built-in camera processors are extremely optimized for these kind of tasks (what's not unlikely) and/or if you have additional information inside the camera at runtime which would support/optimize the removal process but which can't be exported to the outside world.

From all I know today, I would rather strive for an integration of the removal process into raw2dng....

A mi me dá permanentemente el error mostrado en el siguiente link http://www.flickr.com/photos/98427703@N08/9715433322/
Alguien puede ayudarme como solucionarlo?

[Rewind]

A mi me dá permanentemente el error mostrado en el siguiente link http://www.flickr.com/photos/98427703@N08/9715433322/
Alguien puede ayudarme como solucionarlo?

Use PDR with your RAW files, not dng.
By the way, this is an english board.

[Rewind]

...If I want to open DNGs created from video RAW and find the pixels that the camera uses for auto-focus, the pixels that create "pink dots", which attributes of each pixel best identify them?  Or, is there are definitive attribute that can be used?...

These pixels changes their values dynamically, so there are no any distinctive attributes, except may be they LOOKS wrong ))

[coco_net]

Use PDR with your RAW files, not dng.
By the way, this is an english board.

El mismo error me lo daba con los archivos RAW. Pero encontré otra version de pinkdotremover que si lo hace pero quedaban los puntos rosas. Insistí hasta que le encontré la vuelta haciendole una primera pasada con la opcion "EOS-M" y despues otra pasada con la opcion "650D" y logré hacer desaparecer los puntos rosas. Esto anda bien con la configuración de captura de 960 x 540. Con la opción 864 x 486 quedan algunos puntos indeseables.
Ya se Rewind que es un foro en inglés, pero me imagino que por ello no me irán a discriminar, no es así?

[maxotics]

PDR alternative Windows BATCH Script using ufRAW

When PDR broke a couple of weeks ago I became paranoid about using my EOS-M.  Been looking at alternatives.  Here is one. 

First, I mapped out all the focus pixels and sent a "bad pixel" map to dcraw, but it didn't interpolate around them.  Apparently, dcraw only does that to original RAW files. 

I then experimented with ufRAW http://ufraw.sourceforge.net/ and noticed that it has a function to find hot pixels and interpolate around them.  It seems to do a good job on the pink dots. 

Then, reading through Hudson's source code I noticed a quick BASH script that makes an avi file out of jpgs.  That gave me the idea to modify it to process pink-dot files under Windows.

Here is the code I have so far.  You could easily modify this to have ufRAW create TIFFs. 

Code: [Select]

REM Inspired by https://bitbucket.org/hudson/magic-lantern/src/
REM 26f79287de2bf06a6ecaa320962b0fd25c0d68fe/modules/

REM Assumes you've extracted the DNGs from RAW using raw2dng
REM This is the simplest process, you can change ufRAW arguments
REM to create TIFFs instead, and with any other processing you want.
REM You could also change ffmpeg to save
REM ffmpeg higher quality use -vcodec ffv1 ?


REM ******** SET PATHS ***************
set "ufraw_exe=C:\Files2013_VidPhotoSoft\ufRAW\ufraw-batch"
set "ffmpeg_exe=C:\Files2013_VidPhotoSoft\RAWCinema\ffmpeg_latest\ffmpeg"
set "framerate=24"
REM **********************************

REM Delete target files first
del *.jpg
del video.avi
%ufraw_exe% *.dng --out-type=jpg --compression=100 --hotpixel-sensitivity=80

Goto ufRAW
Goto FFMPEG

:ufRAW
@echo off & setlocal
set "ext=jpg" & set "Count=100000"
for /f "tokens=*" %%a in (
'dir/b/on/a-d *.%ext%^|findstr/ie "[0-9]\.%ext%"2^>nul') do (
set/a "Count +=1"&&call :ReNum %%Count:~-6%% %%~xa "%%~na")

:FFMPEG
%ffmpeg_exe% -i 1%%5d.jpg -vcodec mjpeg -q:v 0 -r %framerate% video.avi
REM del *.jpg
Goto:End

:ReNum
if not exist %1%2 ren "%~3%2" %1%2
Goto:End


[demetrisag]

Anyone ever get the feeling that this project is abandoned?

[Rewind]

Anyone ever get the feeling that this project is abandoned?

Neither fire nor wind, birth nor death can erase our good deeds
Buddah

[demetrisag]

Neither fire nor wind, birth nor death can erase our good deeds
Buddah

True that! but how about them deeds keeps evolving! and getting greater and greater!

[a1ex]

Anybody stopping you from doing that?

Welcome to open source world

[demetrisag]

Anybody stopping you from doing that?

Welcome to open source world

Man you have no idea how greatful I am from that!

But on the other hand I'm not one of those guys who understand nich about programming. So am on the side that waits and use whatever its given as much as I can!

[gary2013]

PDR is not working correctly now with the latest builds on the M.

Gary

[gary2013]

Rewind, Mixer2

Neither of your versions of PDR seem to be removing pink dots form my EOS-M in crop mode anymore.  I think Gary might also be stuck.  Any thoughts?  I have a new build from 1%, which is about a week old.    Mixer2, your version will process the files (though pink dots stay).  Rewind, your version doesn't process; says they are corrupted.  I'm using a Sigma 10-20mm EF on the EOS-M.  Was working a couple of weeks ago just fine.

same here. i get messages from this version saying files are corrupted. I then try the older version and none of the pink dots get removed. I get different results from crop mode and non crop mode.

Gary

[benlen]

Hey folks,

I'm new to shooting RAW video, but I downloaded Raw Therapee on my Mac and have been running into a myriad of issues:

The program doesn't seem to eliminate the pink dots in my DNGs.

The program is extremely laggy/crashes often.

My Macbook Pro Retina is brand new/top of the line, so I can't account for the slowness of the program. What's the solution?

[Rewind]

Hey folks,

I'm new to shooting RAW video, but I downloaded Raw Therapee on my Mac and have been running into a myriad of issues:

The program doesn't seem to eliminate the pink dots in my DNGs.

The program is extremely laggy/crashes often.

My Macbook Pro Retina is brand new/top of the line, so I can't account for the slowness of the program. What's the solution?

That's exactly why you should use PDR tool ))
Jokes aside, Raw Therapee is the great program. Not the fastest, and has nothing to do with removing focusing pixels, but gives you almost full control over your dngs (advanced debayering algorithms, true color management etc). I'm using it a lot, and kind of satisfied. May be you should read some manuals first.