Please be sure to follow the instructions of a1ex to help solve this issue.
See the quote below and read this post (http://www.magiclantern.fm/forum/index.php?topic=5614.msg42144#msg42144)
Sorry Sir, this is my first post in here...
Using 5D mark III
Magic Lantern May 22nd(807981e)
in 1920X1080 25P 14-bit raw
It will show in different place in different frame
https://www.dropbox.com/s/ze1khq9t9h2fnyd/23-5-2013%2014-04-01.jpg?v=0mcn (https://www.dropbox.com/s/ze1khq9t9h2fnyd/23-5-2013%2014-04-01.jpg?v=0mcn)
https://www.dropbox.com/s/72ohlah4n7cuubh/23-5-2013%2014-06-00.jpg?v=0mcn (https://www.dropbox.com/s/72ohlah4n7cuubh/23-5-2013%2014-06-00.jpg?v=0mcn)
Quote from: a1ex on May 24, 2013, 08:37:02 AM
Okay, I'll fix it if you do the pixel peeping part.
5D3 bin: autoexec.bin (http://a1ex.magiclantern.fm/bleeding-edge/5D3/experiment/autoexec.bin)
for (int type = 0; type < 100; type++)
{
/* from lv_af_raw -> lv_set_raw_type(arg0 ? 4 : 7) -> MEM(0x2D168) = a bunch of values, default 34 */
MEM(0x2D168) = type;
/* this enables a LiveView debug flag that gives us 14-bit RAW data. Cool! */
raw_lv_request();
msleep(50);
/* after filling one frame, disable the flag so we can dump the data without tearing */
raw_lv_release();
msleep(50);
/* update raw geometry, autodetect black/white levels etc */
raw_update_params();
/* save it to card */
char* fn = silent_pic_get_name();
bmp_printf(FONT_MED, 0, 60, "Saving raw type %d, %d x %d...", type, raw_info.jpeg.width, raw_info.jpeg.height);
save_dng(fn);
}
Take a picture to reset the file counter.
Then, hit "don't click me" to try these raw flags (they are related to bad pixel handling).
Then, write down for each picture type whether it has bad pixels, banding, vertical stripes and whatever else you can find.
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html
Quote from: crazypig on May 23, 2013, 08:13:14 AM
It will show in different place in different frame
https://www.dropbox.com/s/ze1khq9t9h2fnyd/23-5-2013%2014-04-01.jpg?v=0mcn (https://www.dropbox.com/s/ze1khq9t9h2fnyd/23-5-2013%2014-04-01.jpg?v=0mcn)
https://www.dropbox.com/s/72ohlah4n7cuubh/23-5-2013%2014-06-00.jpg?v=0mcn (https://www.dropbox.com/s/72ohlah4n7cuubh/23-5-2013%2014-06-00.jpg?v=0mcn)
please indicate which build you are using when you report bugs.
200% zoom in
https://www.dropbox.com/s/bkfs5nfendqfpsz/23-5-2013%2014-27-43.jpg?v=0mcn (https://www.dropbox.com/s/bkfs5nfendqfpsz/23-5-2013%2014-27-43.jpg?v=0mcn)
https://www.dropbox.com/s/1idnp26ypzpnquc/23-5-2013%2014-28-37.jpg?v=0mcn (https://www.dropbox.com/s/1idnp26ypzpnquc/23-5-2013%2014-28-37.jpg?v=0mcn)
https://www.dropbox.com/s/8f5y6c0u9xxe2hh/23-5-2013%2014-28-59.jpg?v=0mcn (https://www.dropbox.com/s/8f5y6c0u9xxe2hh/23-5-2013%2014-28-59.jpg?v=0mcn)
it will appear in big contrast area when camera moving
https://www.dropbox.com/s/tixx7a9lophsyti/23-5-2013%2014-44-50.jpg?v=0mcn (https://www.dropbox.com/s/tixx7a9lophsyti/23-5-2013%2014-44-50.jpg?v=0mcn)
https://www.dropbox.com/s/zvr5qdbk3je0f4e/23-5-2013%2014-45-42.jpg?v=0mcn (https://www.dropbox.com/s/zvr5qdbk3je0f4e/23-5-2013%2014-45-42.jpg?v=0mcn)
https://www.dropbox.com/s/r0ilz88532utxg3/23-5-2013%2014-46-05.jpg?v=0mcn (https://www.dropbox.com/s/r0ilz88532utxg3/23-5-2013%2014-46-05.jpg?v=0mcn)
Try a sensor cleaning; if that won't fix it, you are out of luck.
Technically is possible to mark the bad pixels, but it's not a priority for me. Canon's bad pixel engine is already turned on.
but the light pixel only appear in some moving shot...
full dark and full bright will not happen..
I find the early build don't have this problem
14th MAY build OK
17th MAY build OK
21st MAY Build Fail
22nd MAY Build Fail
May 22nd(240050c) Fail
I install back 14May build ML
Don't have the hot pixel problem...
https://www.dropbox.com/s/hr1jqp1dzl0v76p/23-5-2013%2017-35-45.jpg?v=0mcn (https://www.dropbox.com/s/hr1jqp1dzl0v76p/23-5-2013%2017-35-45.jpg?v=0mcn)
17MAY Build no problem
https://www.dropbox.com/s/20vuxzexivcf6tg/23-5-2013%2017-53-55.jpg?v=0mcn (https://www.dropbox.com/s/20vuxzexivcf6tg/23-5-2013%2017-53-55.jpg?v=0mcn)
21st MAY build Fail
https://www.dropbox.com/s/3mhiy02pt7l7uqn/23-5-2013%2018-12-15.jpg?v=0mcn (https://www.dropbox.com/s/3mhiy02pt7l7uqn/23-5-2013%2018-12-15.jpg?v=0mcn)
Magic Lantern May 22nd(240050c) Fail
https://www.dropbox.com/s/5ynrb92ejccm60r/23-5-2013%2018-28-13.jpg?v=0mcn (https://www.dropbox.com/s/5ynrb92ejccm60r/23-5-2013%2018-28-13.jpg?v=0mcn)
I think FPS override affects the defect manager engine. Get lots of hot pixels at low fps + the bands.
Quote from: 1% on May 23, 2013, 04:49:43 PM
I think FPS override affects the defect manager engine. Get lots of hot pixels at low fps + the bands.
But it doesn't happen in early build (like 14 or 17 build )
Those screen capture are same setting with different ML only (14, 17, 21 and 22 build)
It happened before raw even existed. On 5d3 you just never had a build that pushed the camera.
Quote from: 1% on May 23, 2013, 05:56:40 PM
It happened before raw even existed. On 5d3 you just never had a build that pushed the camera.
what is the different with 17th MAY build and 21st MAY build?
the color pixel just appear from 21st MAY build...
Dunno, I have a 6D and the same noise patterns you're seeing in all LV images. Newest builds which turn on defect detection make it better.
It appears to be affected by ISO settings if you look in that thread. With FPS override I found turning down ML gain helped it go away.
Quote from: 1% on May 23, 2013, 06:30:33 PM
Dunno, I have a 6D and the same noise patterns you're seeing in all LV images. Newest builds which turn on defect detection make it better.
It appears to be affected by ISO settings if you look in that thread. With FPS override I found turning down ML gain helped it go away.
I tried iso 100 or 1600...also have same color pixel on DNG files....
NOW I am using 17th MAY Build. color pixel go away...
Quote from: Picture and Color on May 23, 2013, 06:59:33 PM
My latest! Comparing H264 Technicolor Cinestyle to ML Raw. Highlight recovery, curves tweaked slightly in Adobe Camera Raw. No other correction applied, and no post sharpening applied. Didn't touch cinestyle H264 footage at all because frankly, it's no longer pleasurable to tweak compressed footage. Camera settings were identical for each set of shots.
Shot on May 22nd build, on 128GB Komputerbay 1000x.
My write speeds have gotten slower with the May 22nd. I can no longer hit 70MB peak at all and struggle to maintain 64MB/sec needed for 1920 x 840. Most clips have at least one dropped frame. I use CF workaround which seems to help. Global draw fully off.
Vimeo is set to 1080p, so please give it a second to load and then hit fullscreen. The parrot shots and the clocktower shots are my favorites in terms of showing off the power of the extended DR and color corrective capabilities.
As other user (stevefal) reported, this video shows the pixel issue (1:03, right side of the zipper, a big area including hair). I'm sorry but those don't look like dead pixels to me. We would be talking about a lot of dead pixels and very poor testing/manufacturing which I think it¡s not the case. It happens along all the video, if you pay close attention to almost any area, hot blinking points appear: in 0,54 the letters have tons of them. I think some of them even move with the letters.
I insist on the issue because I believe they are not hardware defects but data corruption/noise but I might be wrong.
MAy 23rd build, still hot pixels. confirmed 17th working fine.
Interesting side effect to which change I wonder... ?
problem persists in 24th build.
Is there anyone else who isn't getting the random pixels with any of the builds since the 17th? I haven't noticed it in my footage so I'm wondering if maybe there's some camera function that might be triggering it.
Just spent a bit of time testing . Not conclusive yet but it's looking like the build on 6D from 15th May doesn't have issue compared to latest build I'm using. Again though - need to test more to be sure.
Just to add - I checked and it's not chroma noise. I'll try and share some crops when I've slept :P
Yep - definitely a difference in builds. Doesn't exist in older version. Don't worry though - 1% is on the case as soon as he finds time between the bazillion other things him and the other Dev's are sorting :)
The only difference seems like aewb was auto turned off in those builds? Thats all I can think of.
Okay, I'll fix it if you do the pixel peeping part.
5D3 bin: autoexec.bin (http://a1ex.magiclantern.fm/bleeding-edge/5D3/experiment/autoexec.bin)
for (int type = 0; type < 100; type++)
{
/* from lv_af_raw -> lv_set_raw_type(arg0 ? 4 : 7) -> MEM(0x2D168) = a bunch of values, default 34 */
MEM(0x2D168) = type;
/* this enables a LiveView debug flag that gives us 14-bit RAW data. Cool! */
raw_lv_request();
msleep(50);
/* after filling one frame, disable the flag so we can dump the data without tearing */
raw_lv_release();
msleep(50);
/* update raw geometry, autodetect black/white levels etc */
raw_update_params();
/* save it to card */
char* fn = silent_pic_get_name();
bmp_printf(FONT_MED, 0, 60, "Saving raw type %d, %d x %d...", type, raw_info.jpeg.width, raw_info.jpeg.height);
save_dng(fn);
}
Take a picture to reset the file counter.
Then, hit "don't click me" to try these raw flags (they are related to bad pixel handling).
Then, write down for each picture type whether it has bad pixels, banding, vertical stripes and whatever else you can find.
Thank you.
one more detail, only getting noise (at least noticing noise) at 1250 ISO+
RAM:0001DABC ADR R0, aRawTypeRshd0_0 ; "RAW TYPE ::: RSHD ==> 0\n"
RAM:0001DAC0 BL sub_FF0C1F28
RAM:0001DAC4 ADR R0, aShade1_0 ; " SHADE ==> 1\n"
RAM:0001DAC8 BL sub_FF0C1F28
RAM:0001DACC ADR R0, aHivshd2_0 ; " HIVSHD ==> 2\n"
RAM:0001DAD0 BL sub_FF0C1F28
RAM:0001DAD4 ADR R0, aOrbit3_0 ; " ORBIT ==> 3\n"
RAM:0001DAD8 BL sub_FF0C1F28
RAM:0001DADC ADR R0, aDefcorre4_0 ; " DEFCORRE ==> 4\n"
RAM:0001DAE0 BL sub_FF0C1F28
RAM:0001DAE4 ADR R0, aCcd5_0 ; " CCD ==> 5\n"
RAM:0001DAE8 BL sub_FF0C1F28
RAM:0001DAEC ADR R0, aDefmark6_0 ; " DEFMARK ==> 6\n"
I think this is what your bits mean?
6D address is 0x7E020
Almost. There's a lookup table between those numbers and the value written to digic.
Rawtype is the first bit tho?
Not sure. After 64, the images repeat, so there should be 6 bits.
Quote from: a1ex on May 24, 2013, 08:37:02 AM
Okay, I'll fix it if you do the pixel peeping part.
I don't know if you need more tests or if 1% have taken care of that. I'll be happy to help with the pixel peeping :)
As I side note, to keep all the test data in this post, it doesn't happen either in 19th May Build.
Still need the tests. I don't see any significant difference on my 5d3, there's one bright pixel that's been there since the beginning, and I don't see any signs of vertical stripes or banding...
I put a DNG comparison in another post http://www.magiclantern.fm/forum/index.php?topic=5413.msg39600#msg39600 . If you zoom in the big "N" or in other letters the pixels are massive.
I'll run the tests and post them ASAP
So I ran the tests... I get hard lock at type 23 and when I counted backwards type 52.... I see no difference in these, they are all 4.2 MB :(
BTW, 1320x562 or whatever that was is fixed now.
If anyone is able to notice a difference between older and newer builds, run a hg bisect to find the offending changeset.
Also some of the noise bits like noisyboy showed me only ended up in the tiffs when I used rawanizer.. didn't see them in actual dngs and when I exported video.
* One of those raw functions is to set white point?
There are some more fun regs:
0xC0F1A014
0xC0F083C0
0xC0F37014 - what we're messing with I think
This is what was complained about:
http://i41.tinypic.com/149n40.jpg
Test done!
Summary:
Several images show image with color pixels but they are not the issue described.
I've found two RAW types which match exactly the problem (they are in caps): 19 y 83. They are also 1/3 more exposed that the ones marked as "NORMAL". They are equal images for what I can see (the two have same exposure and same hot pixels)
I've found 6 types which shows a normal image with no problems, they seem to be equal among them: 17, 35, 37, 39, 81, 99
*The ones marked as "Color points" are color points covering the whole image in which appears to be the sensor pattern.
I've saved the recorded frames in case you need them for reference. I've double-checked the NORMAL and ISSUE type numbers.
1 Horizontal color lines
2 Color points
3 Image tearing mostly Green/magenta
4 Color points but with some form from the objects photographed
5 Normal image with a few color pixels
6 Color points but with all the form from the objects photographed – Green/magenta
7 Normal image with a few color pixels
8 Normal image with a few color pixels – less exposure
9 Normal image with a few color pixels
10 Color points but with all the form from the objects photographed – Green/magenta
11 Normal image with a few color pixels
12 Color points but with all the form from the objects photographed – Green/magenta
13 Normal image with a few color pixels
14 Color points but with all the form from the objects photographed – Green/magenta
15 Normal image with a few color pixels
16 Color points but with all the form from the objects photographed – Green/magenta
17 NORMAL IMAGE, nothing I can see
18 Color points but with all the form from the objects photographed – Green/magenta
19 IMAGE SHOWING THE ISSUE DESCRIBED – Hot pixels along contrasted lines
20 Color points but with all the form from the objects photographed – Green/magenta
21 Horizontal color lines
22 Color points
23 Horizontal color lines
24 Normal image with a few color pixels
25 Color points but with all the form from the objects photographed – Green/magenta
26 Normal image with a few color pixels
27 Color points but with all the form from the objects photographed – Green/magenta
28 Normal image with a few color pixels
29 Normal image with a few color pixels – more exposure
30 Color points but with all the form from the objects photographed – Green/magenta
31 Normal image with a few color pixels
32 Color points but with all the form from the objects photographed – Green/magenta
33 Rubbish + horizontal lines
34 Color points
35 NORMAL IMAGE, nothing I can see
36 Color points but with all the form from the objects photographed – Green/magenta
37 NORMAL IMAGE, nothing I can see
38 Color points but with all the form from the objects photographed – Green/magenta
39 NORMAL IMAGE, nothing I can see
40 Image tearing mostly Green/magenta
41 Weird horizontal lines-points
42 Color points
43 Image tearing mostly Green/magenta
44 Color points but with some form from the objects photographed
45 Image tearing mostly Green/magenta
46 Color points but with some form from the objects photographed
47 Image tearing mostly Green/magenta
48 Color points but with some form from the objects photographed
49 Rubbish + Color points
50 Color points
51 Image tearing mostly Green/magenta
52 Color points but with some form from the objects photographed
53 Horizontal color lines
54 Color points
55 Horizontal lines
56 Image tearing mostly Green/magenta
57 Image tearing mostly, some magental and vertical stripes one info+one black pattern
58 Color points but with some form from the objects photographed
59 Image tearing mostly Green/magenta
60 Color points
61 Image tearing mostly Green/magenta
62 Color points but with some form from the objects photographed
63 Image tearing mostly Green/magenta
64 Color points but with some form from the objects photographed
65 Horizontal color lines
66 Color points
67 Image tearing mostly Green/magenta
68 Color points but with some form from the objects photographed
69 Normal image with a few color pixels
70 Color points but with all the form from the objects photographed – Green/magenta
71 Normal image with a few color pixels
72 Normal image with a few color pixels – less exposure
73 Normal image with a few color pixels
74 Color points but with all the form from the objects photographed – Green/magenta
75 Normal image with a few color pixels
76 Color points but with all the form from the objects photographed – Green/magenta
77 Normal image with a few color pixels
78 Color points but with all the form from the objects photographed – Green/magenta
79 Normal image with a few color pixels
80 Color points but with all the form from the objects photographed – Green/magenta
81 NORMAL IMAGE, nothing I can see
82 Color points but with all the form from the objects photographed – Green/magenta
83 IMAGE SHOWING THE ISSUE DESCRIBED – Hot pixels along contrasted lines
84 Color points but with all the form from the objects photographed – Green/magenta
85 Horizontal color lines
86 Color points
87 Horizontal color lines
88 Normal image with a few color pixels
89 Color points but with all the form from the objects photographed – Green/magenta
90 Normal image with a few color pixels
91 Color points but with all the form from the objects photographed – Green/magenta
92 Normal image with a few color pixels
93 Normal image with a few color pixels – more exposure
94 Color points but with all the form from the objects photographed – Green/magenta
95 Normal image with a few color pixels
96 Color points but with all the form from the objects photographed – Green/magenta
97 Rubbish + horizontal lines
98 Color points
99 NORMAL IMAGE, nothing I can see
100 Color points but with all the form from the objects photographed – Green/magenta
EDIT: I used your autoexec.bin with the other files from 22nd build and the RAW module loaded.
So I'm not crazy and the color points show up in the image with certain types? I saw sprinkles in raw studio when moving among the different types then they disappear when it processes.
A couple crops (without any other adjustments)
This is type 17 (NORMAL):
https://dl.dropboxusercontent.com/u/21789470/ML%20PIXELS/Type%2017%20CROP.jpg
This is type 19 (ISSUE):
https://dl.dropboxusercontent.com/u/21789470/ML%20PIXELS/Type%2019%20CROP.jpg
I'm amazed with the exposure change, it remembers me to the 1/3 reported difference between the H.264 and the HDMI output in the new 1.2.1 firmware.
Totally see the difference... no more sprinkles.
Guess this has to go in raw.c? So that we can account for the address on different cameras.
Shit, mine all have them. Have to try 26-51
26, 27 freezes... 28-46 doesn't freeze and its all noise free. Haven't tried 48-52 yet but I guess I don't have to.
It turns out that lv_af_raw enables something that looks very similar to a vertical edge detection algorithm instead of defect correction (at least on the 650D) :P
https://dl.dropboxusercontent.com/u/3904468/DNG/AF-SERVO.DNG
You can even enable something that looks like a full-blown contrast-detection algorithm
https://dl.dropboxusercontent.com/u/15964163/MAGIC_4.DNG
Completely defect free at 28 tho.
Quote from: nanomad on May 25, 2013, 07:11:45 PM
It turns out that lv_af_raw enables something that looks very similar to a vertical edge detection algorithm instead of defect correction (at least on the 650D) :P
https://dl.dropboxusercontent.com/u/3904468/DNG/AF-SERVO.DNG
I think that's exactly the issue we were describing!
Thanks for the awesome RAW hack! I really hope this hot pixels isue can be solved, because there are so many of those hot pixels in my 5d MIII footage that they make it unusable.
As far as I can assess, the hot pixels problems started from 19 May, exactly since the banding problem was solved.
I really wish you guys luck with solving the pixel problem so that the 5d Mark III can film perfect professional footage without picture isues.
Thanks for your work.
I've attached a 400x magnified picture crop of a developed RAW DNG (5D Mark III, developed with Adobe Camera RAW).
Here one can see that it is actually not a "hot pixel" problem: Because those faulty pixels are only in those areas where there is a transition from dark to bright (see the radiator in the lower half of the photo).
There are no pixel problems on the wall, because on the wall there are no transitions from darker to brighter areas.
You can download the original DNG here: http://www.webpage-design.org/000003.dng (http://www.webpage-design.org/000003.dng)
(http://www.webpage-design.org/3.jpg)
Reporting 5D3
May 15 good
May 16 good
May 17 good
May 18 good
May 19 good
May 20 BAAAAAD! mercurial 3ab36d8a50de+
May 21 Bad
May 22 Ban
May 23 Bad
May 26 (now) Bad
I am running the autoexec.bin now
Here is my description of what I saw.
Best images in: 17, 35, 37, 39, 81, 83, 99
1 Horizontal lines , colors off
2 All Black..Phtoshop cannot open this file
3 Some Vertical lines, image tearing, colors off
4 Vertical, horizonta, tearting, colors off
5 Good but sprinkles of many colors
6 Vertical lines , colors off, image is vissible
7 Good but sprinkles of many colors
8 Good but sprinkles of many colors
9 Good but sprinkles of many colors
10 Vertical lines , colors off, image is vissible
11 Good but sprinkles of many colors
12 Vertical lines , colors off, image is vissible
13 Good but sprinkles of many colors
14 Vertical lines , colors off, image is vissible
15 Good but sprinkles of many colors
16 Vertical lines , colors off, image is vissible
17 PERFECT IMAGE
18 Vertical lines , colors off, image is vissible
19 Good buT SOME sprinkles of many colors
20 Vertical lines , colors off, image is vissible
21 hORIZONTAL Lines
22 Sort of banding near top, criss croos no image
23 Horizontal lines , colors off
24 Good but sprinkles of many colors
25 Vertical lines , colors off, image is vissible
26 Good but sprinkles of many colors
27 Vertical lines , colors off, image is vissible
28 Good but sprinkles of many colors
29 Good but sprinkles of many colors
30 Vertical lines , colors off, image is vissible
31 Good but sprinkles of many colors
32 Vertical lines , colors off, image is vissible
33 Horizontal lines, off colors, image barely apearing on top
34 Balck ??
35 Excellent
36 Vertical lines , colors off, image is vissible
37 Excellent
38 Vertical lines , colors off, image is vissible
39 Excellent
40 Horizontal lines, off colors,tearing on bottom, image near top, small band on bottom
41 Horizontal lines, off colors, image barely apearing on top
42 Sort of banding near top, then criss croos no image , small band on top
43 Horizontal lines, off colors,tearing on bottom, image near top, small band on bottom
44 Sort of banding near top, then criss croos no image , small band on top
45 Same as above less less the criss cross pattern
46 Sort of banding near top, then criss croos no image , small band on top
47 Same as above less less the criss cross pattern
48 Black?
49 Horizontal lines, off colors, image barely apearing on top
50 Sort of banding near top, then criss croos no image , small band on top
51 Horizontal lines, off colors, image barely apearing on top, tering on center, bootom image
52 Same as above with noise (criss cross=)
53 Horizontal Lines, colors off, no image
54 Criss cross, no image, some banding above
55 Horizontal Lines, colors off, no image
56 Horizontal lines, off colors, image barely apearing on top, tering on center, bootom image
57 Same as above but less contrast?
58 Same as above with criss-cross
59 Horizontal lines, off colors, image barely apearing on top, tering on center, bootom image
60 Same as above with criss-cross
61 Horizontal lines, off colors, image barely apearing on top, tering on center, bootom image
62 Same as above with criss-cross
63 Horizontal lines, off colors, image barely apearing on top, tering on center, bootom image, more noise
64 Black?
65 Horizontal Lines, colors off, no image
66 Black?
67 Horizontal lines, off colors, image barely apearing on top, tering on center, bootom image
68 Same as above with criss-cross
69 Good but sprinkles of many colors
70 Horizontal lines on image, color off
71 Good but sprinkles of many colors
72 Good but sprinkles of many colors
73 Good but sprinkles of many colors
74 Horizontal lines on image, color off
75 Good but sprinkles of many colors
76 Horizontal lines on image, color off
77 Good but sprinkles of many colors
78 Horizontal lines on image, color off
79 Good but sprinkles of many colors
80 Horizontal lines on image, color off
81 Excellent
82 Horizontal lines on image, color off
83 Excellent
84 Horizontal lines on image, color off
85 Horizontal Lines, colors off, no image
86 Sort of banding near top, criss croos no image
87 Horizontal lines on image, color off
88 Good but sprinkles of many colors
89 Horizontal lines on image, color off
90 Good but sprinkles of many colors
91 Horizontal lines on image, color off
92 Good but sprinkles of many colors
93 Good but sprinkles of many colors
94 Horizontal lines on image, color off
95 Good but sprinkles of many colors
96 Horizontal lines on image, color off
97 Horizontal Lines, colors off, image above only
98 Black?
99 Excellent
100 Horizontal lines on image, color off
One more thing I think is important to note:
I spent about an hour shooting this and that, and I could not get the dam specs, or whatever they are. Finally I shot some video of a very busy area of books and magazines just as shown in one of these post and then I could see them all over! Worst problems with may 20th but getting better with the latest build but still noticeable.
I thought I had some perfect ones and then they had sprinkles later on.
I think lv_af_raw is for something else. I dunno if its hurting or helping, the noise pattern just looks slightly different with it off.
What about vertical stripes? Can you reproduce this? http://www.magiclantern.fm/forum/index.php?topic=5490.0
FYI: before may 20, the raw type was 35, after 20 it was (and still is) 19.
As far as I know the stripes disappeared before the 17th build, somewhere around the 13th? I shot a lot of stuff with the 17th build on the weekend and it was clean. I'm onto the 19th build now.
Don't remember the exact date, but this was the only change regarding raw image type. Can you run the test too?
I've just despatched a Pinot hand crushed by Hobbits so I'll get onto it tomorrow. On any particular build or a bunch of them?
All non freezing types seem to work on 600D.
Reporting 5D3 with new compile bfd891f
Standard RAW video, CROPPED VIDEO, AND 60 FPS VIDEO tested at 24, 23.97, 29.9 fps from 1920x1080 to 1280x720 at 16:6 SHOW NO BAD PIXELS.
What about vertical stripes? (squig?)
Quote from: a1ex on May 27, 2013, 06:19:16 PM
What about vertical stripes? (squig?)
Reporting 5D3 with new compile bfd891f VERTICAL BANDING in Blue Sky
I can get vertical noise (I dont think is banding) in the sky by applying excessive sharpening. ACR stadardr sharpening setting is 25. If I apply more than 50 the noise begins to concentrate in vertical lines.
Sharpened at 100 with ACR and 25 Unshar pmask in Phototshop. Original a 600% cropp stretched from 120 pix to 640 pix
(http://s20.postimg.org/kna9egcz1/vertical_noise_640.jpg)
Quote from: a1ex on May 27, 2013, 06:19:16 PM
What about vertical stripes? (squig?)
The 17/5 build looks good, no stripes, no bad pixels. I'll try to shoot something with the 19/5 build tomorrow.
I see no extra noise just leaving the type at default.
Quote from: RenatoPhoto on May 27, 2013, 06:31:27 PM
Reporting 5D3 with new compile bfd891f VERTICAL BANDING in Blue Sky
I can get vertical noise (I dont think is banding) in the sky by applying excessive sharpening. ACR stadardr sharpening setting is 25. If I apply more than 50 the noise begins to concentrate in vertical lines.
That's just garden variety FPN not the stripes, but hey at 600% it could be aliens for all I know. I think the stripes are long gone but I've been stuck in post workflow hell for the last couple of days so I haven't kept up with the last couple of builds.
Last build (016ea7d ) seem to have fixed the issue for me!!!
I have only made a couple test but there's nothing I can see. Thanks a lot, devs!
Quote from: squig on May 27, 2013, 07:12:57 PM
That's just garden variety FPN not the stripes, but hey at 600% it could be aliens for all I know. I think the stripes are long gone but I've been stuck in post workflow hell for the last couple of days so I haven't kept up with the last couple of builds.
;D ;D ;D
With the Magic Lantern May 27th(016ea7d), i haven't hot pixels, but now i have vertical banding.
In the previous Magic Lantern May 25th(1e0738f), no vertical banding and hot pixels ...
Hi Guys. Anyone know which was the last build without hot pixels or this vertical banding which was just mentioned. I forgot which build I last installed which didn't have these issues. I just took the day to go film some stuff and found out the build I have installed which I think is May 21st, has the hot pixels.
"With the Magic Lantern May 27th(016ea7d), i haven't hot pixels, but now i have vertical banding.
In the previous Magic Lantern May 25th(1e0738f), no vertical banding and hot pixels ..."
See my footage :
(http://www.actionstudio.re/client/May25th.jpg)
(http://www.actionstudio.re/client/May27th.jpg)
Doesn't help.
Go here and run the pixel peeping test.
http://www.magiclantern.fm/forum/index.php?topic=5614.25
With the newest build (7fdfc08), color pixels are gone, but same old banding is back.
http://www.webpage-design.org/000000.dng (http://www.webpage-design.org/000000.dng)
DNG 200% magnified:
(http://www.webpage-design.org/0.jpg)
Great new feature that the files are named by date and number! :-)
Unless you run the pixel peeping test (see above), you'll be stuck with this. I don't see any banding in these samples.
The "same old banding" in that shot is FPN, it's the Canon sensor doing that not ML. This thread shows the difference between FPN and a banding/stripe bug that was affecting some earlier builds. http://www.magiclantern.fm/forum/index.php?topic=5490.0 What ISO did you shoot it at?
A1ex 7fdfc08 is a sweet build ;D
I would love to help with the pixel beeping test but I don't know how it works. I downloaded the autoexec file from this post and combined it with 28th build, but my camera hangs up when I try to load the modules. Maybe someone can please explain more in detail about how to make the pixel beeping test.
I'm really sorry, but I still get banding with the 28th (7fdfc08) build on the 5D Mark III. I don't understand why this banding isn't a problem for others or why the others don't see it.
But for me, this banding would make it impossible to use the RAW footage for pro use.
Don't misunderstand me, the RAW footage is the best and sharpest stuff I've seen for a long time (color pixels are also gone), but the banding is still there and really distracting.
Photo at ISO400 with 7fdfc08 built at 200% magnification:
http://webpage-design.org/000379.dng (http://webpage-design.org/000379.dng)
(http://webpage-design.org/000379.jpg)
Read the post again, I did not mention anything about modules...
Issues on 28th (7fdfc08):
As I shoot in raw mode (1880/800-2:35) enabling separate audio wave, recording starts ok, but as I stop it the Build crashes on (red light is permanent red and the information in superimpose still)
As I look for files inside the cards, I detect 0 files in SD card and a corrupted file with a strange file "backup.Raw" inside the CF
That just as enabling separate audio wave, selecting Beep or Off it's all ok
I'm having the same issue since the May 27th build.
For the 1000th time: if you don't run the pixel peeping tests, there's nothing I can do.
http://www.magiclantern.fm/forum/index.php?topic=5614.25
Also try shooting with a manual lens (or unscrew the Canon lens). Who knows...
I can`t figure out how to do the pixel peeping, but I can confirm the banding issue on the latest build (7fdfc08) on both electronic and manual canon lenses.
If you can't follow something as simple as downloading some autoexec and hitting "don't click me", I wonder how did you install ML...
sometimes i wonder about that myself.. but anyway, when i download the autoexec and boot with it my camera just locks up with a frozen frame and it says "34" in the uper left corner.. so i can`t get into the ML menu.
ok i got it working, will start peeping at em now.
1 scrambled
2 scrambled
3 scrambled
4 scrambled
5 Normal image with a few color pixels - NO BANDING
6 scrambled
7 Normal image with a few color pixels - NO BANDING
8 Normal image with a few color pixels – VERTICAL BANDING
9 Normal image with a few color pixels - NO BANDING
10 scrambled
11 Normal image with a few color pixels - NO BANDING
12 scrambled
13 Normal image with a few color pixels - NO BANDING
14 scrambled
15 Normal image with a few color pixels - NO BANDING
16 scrambled
17 NORMAL IMAGE - VERTICAL BANDING
18 scrambled
19 Hot pixels along contrasted lines - NO BANDING
20 scrambled
21 scrambled
22 scrambled
23 scrambled
24 Normal image with a few color pixels - VERTICAL BANDING
25 scrambled
26 Normal image with a few color pixels - VERTICAL BANDING
27 scrambled
28 Normal image with a few color pixels - VERTICAL BANDING
29 Normal image with a few color pixels – NO BANDING
30 scrambled
31 Normal image with a few color pixels - VERTICAL BANDING
32 scrambled
33 scrambled
34 scrambled
35 NORMAL IMAGE - VERTICAL BANDING
36 scrambled
37 NORMAL IMAGE - VERTICAL BANDING
38 scrambled
39 NORMAL IMAGE - VERTICAL BANDING
40 scrambled
41 scrambled
42 scrambled
43 scrambled
44 scrambled
45 scrambled
46 scrambled
47 scrambled
48 scrambled
49 scrambled
50 scrambled
51 scrambled
52 scrambled
53 scrambled
54 scrambled
55 scrambled
56 scrambled
57 scrambled
58 scrambled
59 scrambled
60 scrambled
61 scrambled
62 scrambled
63 scrambled
64 scrambled
65 scrambled
66 scrambled
67 scrambled
68 scrambled
69 Normal image with a few color pixels - NO BANDING
70 scrambled
71 Normal image with a few color pixels - NO BANDING
72 Normal image with a few color pixels – VERTICAL BANDING
73 Normal image with a few color pixels - NO BANDING
74 scrambled
75 Normal image with a few color pixels - NO BANDING
76 scrambled
77 Normal image with a few color pixels - NO BANDING
78 scrambled
79 Normal image with a few color pixels - NO BANDING
80 scrambled
81 NORMAL IMAGE - VERTICAL BANDING
82 scrambled
83 Hot pixels along contrasted lines - NO BANDING
84 scrambled
85 scrambled
86 scrambled
87 scrambled
88 Normal image with a few color pixels - VERTICAL BANDING
89 scrambled
90 Normal image with a few color pixels - VERTICAL BANDING
91 scrambled
92 Normal image with a few color pixels - VERTICAL BANDING
93 Normal image with a few color pixels – NO BANDING
94 scrambled
95 Normal image with a few color pixels - VERTICAL BANDING
96 scrambled
97 scrambled
98 scrambled
99 NORMAL IMAGE - VERTICAL BANDING
100 scrambled
so no perfect number in there as i can see it, it`s either pixels or banding, or both.
a1ex has made it quite clear on a number of occasions that he does not experience the issue, and cannot, and will not, fix the issue without details from the pixel peeping build.
General discussion/banter about the issue is fine, but any users contributing nothing to the thread, and specifically, mentioning issues regarding specific builds without having run the build from a1ex to help fix the issue, can expect a 3 day ban.
See this post (http://www.magiclantern.fm/forum/index.php?topic=5614.msg39696#msg39696) for information about how to help solve this problem.
There was no fix for me... all types on 6D enable the stupid faux peaking.
leaving the type at default seemed like the best idea...
Easy to reproduce the sprinkles (focus an image with decent bit of contrast, ie wires) but hard to reproduce the different FPN that is produced. The FPN was in .422 and H264 as well.
The FPN is fairly easy to remove with de-noising plugins... the sprinkles are really not.
At least yours doesn't freeze running the test.
Quote from: a1ex on May 29, 2013, 12:00:20 AM
If you can't follow something as simple as downloading some autoexec and hitting "don't click me", I wonder how did you install ML...
LOL
There's definitely no stripes in my footage so I suspect some setting in the camera is triggering it. In addition to performing the peeping test can anyone who's seeing the stripes/banding restore their MK3 to the default settings in the Canon menu and turn global draw and every other ML feature except raw recording off and shoot another test and upload a DNG. Make a note of the ISO, WB setting, shutter speed, lens, and nightly build number/date.
EDIT: Anyone who's experiencing the stripes/banding should do a test with FPS override set to 24/25fps exact and report whether that resolves the problem.
The banding for me comes from regular places... ISOs like 160, fps override, etc. Mostly on under-exposure or OK exposure at high ISO.
Quote from: squig on May 29, 2013, 01:45:33 AM
LOL
There's definitely no stripes in my footage so I suspect some setting in the camera is triggering it. In addition to performing the peeping test can anyone who's seeing the stripes/banding restore their MK3 to the default settings in the Canon menu and turn global draw and every other ML feature except raw recording off and shoot another test and upload a DNG. Make a note of the ISO, WB setting, shutter speed, lens, and nightly build number/date.
could you share a dng?
I zoomed in 400% on my footage from yesterday and there's barely even any FPN. I shot it at 200 ISO. I have everything but raw recording off in both menus. I've never seen this banding/stripes on anything I've shot since the 1st raw build.
Quote from: jonzero on May 29, 2013, 01:50:48 AM
could you share a dng?
There's no point, there's nothing to see. The ACR denoiser was set to 0 luma and 25 chroma. To reduce the stripes on other peoples DNGs I set it to 50 luma 50 chroma.
Do some fps override... you'll see it I bet.
I always have FPS override set to 24 exact.
Here's a DNG https://docs.google.com/file/d/0BzJ3L6nv6Fn0dW00MU5aTGNoa0k/edit?usp=sharing
Set some below 6fps. 24 is OK I think.
I haven't seen stripes or the hot pixel problem as described yet in my Mark II footage. I've seen just a hint of a flicker in one clip. https://vimeo.com/66414746 Using may 24 build from a.d. and the 'alternative' raw module. In fact, I'm pretty stoked so far. It has gotta be donation time.
Quote from: 1% on May 29, 2013, 02:03:51 AM
Set some below 6fps. 24 is OK I think.
Hehe, why would I want to do that?
Then try some modes besides exact fps. Low fps is just a quick way to reproduce.
Amazing to see that the "sprinkles" (love that that's what we're referring to it has ha!) have completely disappeared when testing on the latest module for 6D. Finally have some time to spare so I can run the don't click me test if it's still required or is it not needed now?
Quote from: 1% on May 29, 2013, 02:57:18 AM
Then try some modes besides exact fps. Low fps is just a quick way to reproduce.
So what you're implying is people are experiencing this issue because they haven't configured their camera correctly for 24p recording.
Kinda. FPS should be 23.976 in theory.. none of the modes are really wrong... just exact works really well via trial+error.
All FPS overrides *should* work.
The other modes are more likely to produce defects so if you're not seeing ANY.... I'm not seeing them most of the time either... I didn't even find any sprinkles in normal shooting (with lv_af_raw) but tested some more and they were there... then we found the cause was this.
I don't want to be the guy who gets sprinkles (or excessive FPN) and has to edit them out :) That said, people are really peeping hard core.
Nothing wrong with a bit of hardcore peeping 8) :o
Anyone who's experiencing the stripes/banding should do a test with FPS override set to 24/25fps exact @ 100 multiple ISOs and report whether that resolves the problem.
Jonzero made a good job with his color pixels / banding analysis. I've just also done the complete test twice (though I only come to RAW type 74 because my camera always hang up at raw type 56 or 74). I made the test with 5D MarkIII and 8ba641e build. I also reseted all canon settings and turned of all other ML settings and played around with FPS.
However, results were always exactly as in Jonzeros list: Either color pixels, or vertical banding (repeating exactly every 8 pixels) or both.
For me, that is so tragic. I really hope Alex finds a way to fix the banding OR color pixels.
From all RAW types, mode 19 had fewest color pixels. But still too many color pixels.
Alex, do you think you might find another way to get out the color pixels OR banding, or do you assume that we will be stuck with it for a long time?
It's a mistery why some users are getting banding and others don't. I've also cleared all Canon settings and I get perfect image without any signs of banding or bad pixels, even with extreme curves.
The artifacts look somewhat like what I've got with vignetting correction. My lens (50/1.8) has no correction data on 5D3. Maybe it's related?
I've done a lot of experiments changing parameters of the camera, in canon menus, trying different lenses. All over 6D and hot pixels are there with low fps (6 fps for example) with fps override and low light optimization for example. Fast TV also has the problem but less noticeable.
Well, finally we observe a subtle transition on lv image when I go from 1x to 5x or 5x to 1x. For a lapse of 1/2 second or so, the image looks OK after change zoom mode, and suddenly the display view adquires some kind of digital gain and the hot pixels appear (and perhaps the banding issue and other image issues). If it where possible to deactivate the second behaviour ...
On 550D there are no trace of these issues.
If people don't start providing us with more info we won't be able to get to the bottom of this. I need to know all your shot settings, lens, CF card type, battery type, ML build number, and camera serial number. Also upload a DNG of the banding because some people confuse the stripes/banding with FPN and I need to be sure it's stripes.
A1ex and I haven't been able to produce the stripes/banding on our cameras which suggests something is interacting with ML and triggering the stripes. Once a number of people provide the info requested hopefully a pattern will emerge.
A1ex somebody mentioned they tried an auto and a manual lens. I don't have any auto glass. The stripes do look like some kind of interference and AF lens was my first suspect. Are you guys anywhere near a phone tower? Just humour me and go shoot in a forest. ???
the image is clear when switching zoom modes because override is reset. same thing as hitting play to reset bad fps override.
FPS override has nothing to do with this problem; these stripes are not noise, but a fixed pattern with 8-pixel periodicity.
A closer look reveals that banding is present only in blue and one of the green channels, at least with this dng: http://webpage-design.org/000379.dng
dcraw -4 -d 000379-1.dng
Bayer pattern:
RG
GB
Left column is OK, right is bad. The difference seems to be an EV factor (gaps are higher in bright areas). Probably easy to fix if we know exactly when it happens.
Correction factors for top-right green channel: 1, 1.0087, 1.0015, 0.9889, repeat.
Correction factors for blue channel: 1, 1.0080, 1.0017, 0.9882, repeat.
Multiply the bad channels with these numbers and you get perfect image.
we got similar patterns from fps override. same lines at regular intervals.
try lv_md_visible,0 i guess and then maybe will have defc + sprinkles off?
FPS override patterns contain noise and increases over time; this one is only a luma multiplier.
I'd like a few RAW files to take a closer look. They should be from different users, with different settings, but they all should show banding. Don't upload large stuff, 1-2 seconds of video is enough.
5D3 + 24-70- 2.8 II + 64GB 1000X Lexar
F/4 1/50th ISO 1600 Files were both set to WB 2900, +22 Magenta and +1.0 Exposure
Banding is present on the May 28th build. Also, not sure why I can only record 50 frames or so until it frame skips with the May28th build, which wasn't even usable.
May 22nd Build link - https://drive.google.com/folderview?id=0B3TpGf3-Q35ETl9PczVFTFl3cU0&usp=sharing
May 28th Build link - https://drive.google.com/folderview?id=0B3TpGf3-Q35EeldRY1JFYTdtS0k&usp=sharing
I did not ask for DNGs or for comparison between builds; I've asked for raw files that show banding.
Quote from: a1ex on May 30, 2013, 08:18:07 AM
I did not ask for DNGs or for comparison between builds; I've asked for raw files that show banding.
I asked for DNGs.
Jason that's just stock standard FPN, the 28th shots are a bit noisier because they're a bit more underexposed. I don't see any of the stripes/banding in those shots. Set the luma denoising slider to 50 in ACR and like magic it disappears.
a1ex this is all your fault, you upped the res so much now we can see all the crap on the sensor! ::)
Quote from: squig on May 30, 2013, 08:31:39 AM
I asked for DNGs.
Jason that's just stock standard FPN, the 28th shots are a bit noisier because they're a bit more underexposed. I don't see any of the stripes/banding in those shots. Set the luma denoising slider to 50 in ACR and like magic it disappears.
Well I'm not really understanding why the exposures should be any different, identical settings, identical controlled lighting. The 28th build clearly is worse and I cannot record more than 70 frames with it for some reason.
Quote from: a1ex on May 30, 2013, 08:18:07 AM
I did not ask for DNGs or for comparison between builds; I've asked for raw files that show banding.
https://docs.google.com/file/d/0B3TpGf3-Q35EODd2ZFVMVllpaUE/edit?usp=sharing
Quote from: a1ex on May 30, 2013, 12:22:28 AM
I'd like a few RAW files to take a closer look. They should be from different users, with different settings, but they all should show banding. Don't upload large stuff, 1-2 seconds of video is enough.
Canon 5D Mark III
Canon EF 16-35 2.8L USM II
(The tint is due to Genus ND filter)
ML May 29th(5419e04)
ISO 100
Aparture 5.6
fps override Off
fps 23.976
DNG https://www.dropbox.com/s/d2yb2ycg20vtpbo/M30-0910000000.dng
RAW https://www.dropbox.com/s/36e4284k5jarzx5/M30-0910.RAW
Did a test with fps override 24 for 24 Exact but no change.
ML Magic Lantern May 22nd(807981e)
ISO 100
Aparture 5.6
fps override Off
fps 23.976
DNG https://www.dropbox.com/s/5bnirpd2heexuug/M0000000000000.dng
RAW https://www.dropbox.com/s/yf0hggsrqg6x2v0/M0000000.RAW
Vertical banding gone in 22may!
@a1ex,
Are you using the same ML files that lourenco kindly compiles for us every day? same source file and same camera, but not the same result. I am wondering if there is a possibility of this issue entering during compilation. I appreciate what lourenco does, I really do. I am just thinking out loud.
Luca your sample has the stripes/banding but the interesting thing is it responds identically to the same level of luma denoising as Jasons sample. Do you have a manual focus lens you can try the same test with?
I'm using the lourenco builds and I've never had the sitripes/banding.
Quote from: squig on May 30, 2013, 09:39:56 AM
I'm using the lourenco builds and I've never had the sitripes/banding.
That could very well be a poor speculation out of my ignorance.
Quote from: squig on May 30, 2013, 09:39:56 AM
Do you have a manual focus lens you can try the same test with?
Sorry I wish I had a nice Zeiss prime, but no. my autofocus lens set to manual focus tho :D
Just tested with my tamron 24-70 and with my cheap nikon 35mm manual lens. There isn't much difference.
@squig, are you looking at your dng files without noise reduction?
I think I have a fix. Download latest raw2dng from the sticky topic, try it and post some conversion logs (copy/paste the output from the terminal).
Quote from: bumkicho on May 30, 2013, 10:05:03 AM
Just tested with my tamron 24-70 and with my cheap nikon 35mm manual lens. There isn't much difference.
@squig, are you looking at your dng files without noise reduction?
Affirmative.
Quote from: LucaBrasi on May 30, 2013, 09:50:50 AM
Sorry I wish I had a nice Zeiss prime, but no. my autofocus lens set to manual focus tho :D
(If a1ex's fix doesn't work) shoot a few frames @ 100 ISO with your lens cap on and then remove the lens and shoot a few more with the body cap on. Identical settings both times. Upload the first dng from both.
Quote from: squig on May 30, 2013, 08:31:39 AM
Jason that's just stock standard FPN, the 28th shots are a bit noisier because they're a bit more underexposed.
Try the new converter and you'll have a little surprise ;)
May 28, 2013, i test the Magic Lantern May 28th (7fdfc08) : Perfect for me !!!
Vertical banding : FIXED
Hot pixels : FIXED
And finally, i test the Magic Lantern May 28th (8ba641e) : I have Vertical Banding ... grrrr !
Today i reset my 5D MKIII "clear all camera settings" and i run Magic Lantern May 28th(8ba641e) without global draw (Global Draw OFF), without FPS (set to OFF). And vertical banding is fixed ...
Now, i should test with Global Draw ...
Lens : Tamron 24-70 2.8 VC
EDIT : OK, i active GLOBAL DRAW, and the image is perfect ... I think that the "RESET" fixe my Vertical Banding ...
I reshot today with newest build (5419e04). I used manual lens (Nikon 50 mm). ISO 100, 1/50th shutter. 8-Pixel Vertical Banding is still there:
http://www.webpage-design.org/M30-0948.RAW (http://www.webpage-design.org/M30-0948.RAW)
Thanks Alex so much for updating the RAW2Dng, but I just tried and 8-Pix-Vertical-Banding is still there. Please also try with the RAW file above.
8-Pixel-Vertical-Banding is not FPN and is not coming from the sensor, because at RAW Type 19 it completely disappears (but color pixels appear).
Those who don't see the 8-Pixel-Vertical-Banding in their footage maybe don't shoot the right subjects in order to see it. One has to slightly underexpose an even background (like sky or even wall) to see it.
Luma denoising slider to 50 indeed makes the banding less visible, but it also takes away a lot of grain and shadow details, making the footage much more look like compressed.
Uh... for some reason, the windows version doesn't do the math right, but the linux one does... looking into it.
Edit: solved, download again, it was a windows-specific compiling flag.
https://bitbucket.org/hudson/magic-lantern/commits/b5e91e24995e
I can't believe it, but banding is really completely gone with newest RAW2DNG tool! What a historic moment, now we can shoot RAW FullHD even under the most demanding situations without color pixels or banding or whatsoever.
Alex, I think you (and the rest of the team) will really make filmmaking history with this. FullHD raw from a superb fullframe sensor, without any negative sideeffects whatsoever. You should get the Technical Oscar for this. And the resolution is the maximum one can get out of FullHD. What an amazing time to live in. Thank you so much!
https://www.dropbox.com/sh/bua4hwbm8w1i47f/3ArdOXBS5p
i have installed the newest build on my 5d mk3, 29_May,
made a complete reset of canon settings,
recorded some raw,
converted on the mac with raw2dng 0.10
got the same vertical noise.. :(
check the dropbox link.
the blue one is a screenshot and unsqueezed cause of anamorphic lens.
the dng is untouched.
thx.
The mac version isn't updated yet, try again tomorrow with version 0.11.
great, thank you.
Thanks a1ex and dev team for continuing the BMCC ass kicking, respect!!!
New raw2dng worked a treat ;D
A1lex. Put up donation link so you can get 5D3. I cannot keep up with all the testing! ;D ;D ;D ;D
Congratulations!
I only have a 60D so I can't test this debug build. I made a dng with a lot of hot pixels by overexposing and using high iso. This way I found out that Adobe's raw engine corrects dead or hot pixels, but only the once that are not next to each other. Maybe this is a reason why results are so inconsistent...
I would suggest to use a program like RawTherapee for pixel peeping, it's free and open source.
(http://s21.postimg.org/ijoj71vhj/hot_pixel.png)
600D benefited from af_raw being set, try it on 60D. Did not get any banding.
I've got also a problem with some bad/hot pixels on 60D (I've just build ML from latest source).
https://docs.google.com/file/d/0B-HdscXfsKpgdXZGUUc0VTRVMEE/edit?usp=sharing (https://docs.google.com/file/d/0B-HdscXfsKpgdXZGUUc0VTRVMEE/edit?usp=sharing)
Quote from: vroem on May 30, 2013, 02:32:17 PM
I only have a 60D so I can't test this debug build. I made a dng with a lot of hot pixels by overexposing and using high iso. This way I found out that Adobe's raw engine corrects dead or hot pixels, but only the once that are not next to each other. Maybe this is a reason why results are so inconsistent...
[/img]
thx, I will try it
Quote from: 1% on May 30, 2013, 02:43:42 PM
600D benefited from af_raw being set, try it on 60D. Did not get any banding.
ok, no dead/hot pixels after compiling with lv_af_raw
Quote from: 1% on May 30, 2013, 02:43:42 PM
600D benefited from af_raw being set, try it on 60D. Did not get any banding.
lv_af_raw enables edge detection. Don't use it.
ok
I've just checked RawTherapee Hot/Dead pixel filter and it works..
Quote from: a1ex on May 30, 2013, 03:07:22 PM
lv_af_raw enables edge detection. Don't use it.
hmm ... does edge detection work when AF is in manual mode ?
Quote from: a1ex on May 30, 2013, 03:07:22 PM
lv_af_raw enables edge detection. Don't use it.
There is a bug in the new RAW2DNG. When I convert RAWs with it, there are black vertical lines in those areas of the photo with are strongly overexposed.
This doesn't happen with the old RAW2DNG, however old RAW2DNG has vertical banding. Vertical banding is completely gone with new RAW2DNG. The footage looks soooo clean now without old vertical banding problem.
Please download and use Windows RAW2DNG to see the problem (vertical lines):
http://www.webpage-design.org/M30-1805.RAW (http://www.webpage-design.org/M30-1805.RAW)
Problem occures both with Global Draw Off and On.
Those are zebras :D
Just tried new raw2dng window version. I see black vertical lines too.
Quote from: a1ex on May 29, 2013, 07:21:27 PM
My lens (50/1.8) has no correction data on 5D3.
So do you have a 50/1 or a 50/1.8?
Quote from: a1ex on May 30, 2013, 06:33:32 PM
Those are zebras :D
Now I'm seeing them too!
(http://i.imgur.com/WrNP4zN.jpg)
I have a feature request, I want wombats!
Zebras at overexposed areas are gone with the newest RAW2DNG, thanks A1ex! :)
The footage is so amazingly clean now that the 8-Pixel-Vertical-Banding is gone. I can't believe it, it's so breathtaking. So sharp, such a unbelievable dynamic range, exceeding the best photochemical film in dynamic range.
Where is this newest raw2dng?
@Bumkicho:
It's here: https://bitbucket.org/hudson/magic-lantern/downloads (https://bitbucket.org/hudson/magic-lantern/downloads)
yes, great. thank you. but it is pc only.
mac version 0.10 has the vertical noise problem. so we have to wait for .011 on the mac.
greets.
There are some subjects where the RAW2DNG algorythm does not yet recognize the 8-pixel-vertical-banding and therefore does not substracts it out.
I was about to upload some samples, but have seen that there is a new RAW2DNG update (https://bitbucket.org/hudson/magic-lantern/commits/97d9ed828854ad0d9376dd8e0917db51f3ee6df9) which is not yet available to download for Windows.
So I'll wait to check my RAW footage with the new RAW2DNG (once it will become available for download) before I upload the RAW samples.
Can you upload some samples? (the first DNG from the sequence is enough, along with the conversion log).
OK, shot with 5D Mark III, converted with latest RAW2DNG (30 May).
First, for clarification: The full frame RAW of my 5D Mark III usually shows 8-pixel-banding (unless corrected by new RAW2DNG), but the 5x magnification RAW of my 5D Mark III never shoes 8-pixel-banding.
Sample 1: When converting this full frame RAW shot, the RAW2DNG conversion log does not mention any stripe correction, therefore the stripes remain to be seen:
http://www.webpage-design.org/sample000001.dng (http://www.webpage-design.org/sample000001.dng)
Maybe the algorythm became confused by the dark area in the lower part of the shot.
Sample 2: When converting this 5x magnification RAW shot, the RAW2DNG conversion log does perform stripe correction (1 1 1 1 1.006 1.008 1.0010 1.0012), although 5x magnification RAW never has banding. Thus it creates banding although this banding would not be there with old RAW2DNG conversion:
http://www.webpage-design.org/sample000002.dng (http://www.webpage-design.org/sample000002.dng)
Unfortunately the RAW files belonging to this DNGs are way too big to upload.
Good catch, gradient in source image was the problem here.
Got a few solutions that seem a bit better at first sight:
1) raw2dng_noweight.exe (http://a1ex.magiclantern.fm/bleeding-edge/raw/raw2dng_noweight.exe) - it addresses the gradient issue and uses the old weighting method (constant weights for all pixels that are -9 EV under white level or brighter, the others are discarded). It is able to undo most of the banding from sample 2 (not quite exact, 1.000 1.000 0.999 1.000 0.991 0.992 0.987 0.987). According to your log, they should be 1 1 1 1 0.994 0.992 0.990 0.988.
2) raw2dng_weight.exe (http://a1ex.magiclantern.fm/bleeding-edge/raw/raw2dng_weight.exe) This addresses the gradient issue and uses variable weight for pixels brighter than -12 EV under white level; the weight is equal to EV value relative to black level, so a bright pixel can be ~10x more important than a dark one. Undo coefficients: 1.000 1.000 1.000 1.001 0.994 0.994 0.991 0.990.
3) raw2dng_weight_hilight.exe (http://a1ex.magiclantern.fm/bleeding-edge/raw/raw2dng_weight_hilight.exe) Same as 2, but ignores very bright areas when estimating the coefficients. It only makes a difference when the image contains overexposed areas.
Can you run some tests and see which one seems closest to reality?
Note: these are debug versions, and they will always do some correction, no matter what.
Great Job, A1ex, banding in my two sample RAW footage is gone with the new RAW2DNGs. :-)
I also tested them with some other footage and they seem to work without any banding problems (only time will tell if there are some subjects left which the algorythm does not recognize).
Visually, all 3 solutions are the same as perfect. My eyes cannot see any visible differences in those 3 solutions, they are all good. :)
( By the way, when I drag and drop several RAW files to RAW2DNG (in Windows), it only convertes one of them. )
5Dm3 banding issues. An attempt for clarity for dumb guys like me.
Banding was caused by some of the earlier versions of raw2dng. (Not the camera itself, yeah!)
Raw2DNG v11 fixes the problem.
Where can we find v11 for the mac?
..........
You guys are incredible, you are re-defining the entire film landscape keep up the good work, and many thnaks from all of us.
It is caused by the camera itself; raw2dng tries to correct it.
Hi Guys! It´s ready for mac??? thanks!
Recorded a very short video, I notice that it's underexposed a lot(on purpose) but I didn't bring the shadwos up any.
The lines ALSO show in the bright areas of the video as well.
I used the latest raw2dng(for mac)
I also used the ML from June 1st(I believe)
Any idea as to why these black bars are across the entire image???
https://www.dropbox.com/s/o9zasksokmkgubj/M01-2200.mov
Seems as though dropbox compressed the video
here's a still taken from the video
If anyone knows of a website where I can upload the video(for free) with the best quality, let me know
(http://i40.tinypic.com/k4yphz.png)
"Raw2DNG v11 fixes the problem.
Where can we find v11 for the mac?"
That's right... any clue ?
There is no 0.11 for mac yet.
Sorry to be a bother, but was wondering if there will be a 0.11 for the Mac. In the meantime, is there a no-banding version available for the PC?
Quote from: mannfilm on June 06, 2013, 04:26:52 PM
Sorry to be a bother, but was wondering if there will be a 0.11 for the Mac. In the meantime, is there a no-banding version available for the PC?
It was updated for Mac yesterday with the banding fix. Check the raw2dng thread
Hello A1ex,
at the end what of the three possible solutions (weight, noweight, noweight_hilight ) had you choosen?
It's already implemented in the last commit?
Right now it's the last one, but Thomas Worth found a test case where there's room for improvement.
Edit: this seems to be a lot better: https://bitbucket.org/hudson/magic-lantern/commits/d4b1e587cfab
(at least it covers all the DNGs that I've tried it on, with extremely strong curves)
Based on original chdk's dng.c file I figured out how to fix bad/hot pixels in DNG file.
Pls look at original chdk dng.c
http://subversion.assembla.com/svn/chdk/trunk/modules/dng.c (http://subversion.assembla.com/svn/chdk/trunk/modules/dng.c)
They implemented bad pixel removal using a special map of bad pixels created on cameras manually (they shot two times and choose a file with higher sum of bad pixels and create file called badpixel.bin). They set DNG version to 1.1 because 1.3 or 1.4 isn't working with this method. However there is an option to use special badpixel opcode in idf1 structure, automatically using camera_sensor.cfa_pattern. If DNG version is set to 1.3 ACR automatically fixes bad and hot pixels. I tested it and it works. Pls try it:
Modified chdk-dng.c
https://www.dropbox.com/s/pyby320bap6rhwg/chdk-dng.c (https://www.dropbox.com/s/pyby320bap6rhwg/chdk-dng.c)
DNG with bad/hot pixels:
https://www.dropbox.com/s/9dh1tp4d3vpxo43/000000_bad_pixels.dng (https://www.dropbox.com/s/9dh1tp4d3vpxo43/000000_bad_pixels.dng)
DNG processed with modified chdk-dng.c (without raw2dng fix_bad_pixels)
https://www.dropbox.com/s/000pzcw9l05pw4o/000000.dng (https://www.dropbox.com/s/000pzcw9l05pw4o/000000.dng)
DNG 1.3 specification (http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec.pdf)
"There are two ways to deal with defective pixels in DNG. The first is to map out (interpolate over) the defective pixels before storing the raw data in DNG. The second is to include a bad pixel fixing opcode in the OpcodeList1 tag."
I'm trying to find a better solution for this issue:
https://bitbucket.org/hudson/magic-lantern/pull-requests/732/5d3-attempt-to-fix-vertical-stripes-so-it/diff
Does it mean we'll be able to select PREFERRED_RAW_TYPE in the menu? Cool.
The menu option is only meant for debugging; I don't want to expose it in the public builds.