Magic Lantern Forum

Developing Magic Lantern => General Development => Topic started by: a1ex on February 11, 2014, 03:29:48 PM

Title: Fix for pink frames
Post by: a1ex on February 11, 2014, 03:29:48 PM
Having some major problems with pink frames with 5D3 123 (also reported by pompeiisneaks (http://www.magiclantern.fm/forum/index.php?topic=9827.msg99755#msg99755)), I took a closer look and I believe I've fixed them for good (at least on this camera).

So, besides the well-known pink frame problem (let's call it issue #1), 5D3 123 had a new kind of corruption: all frames were geometrically correct, but colors were scrambled (examples in the link from pompeiisneaks; let's call this one issue #2). I believe I've fixed both of them (but to be sure, it needs lots of testing).

(http://imagizer.imageshack.us/v2/800x600q90/855/pmo3.jpg) (http://i.imgur.com/P4gSYI8l.jpg)

How I've reproduced the issues:

- raw_rec grayscale preview: there was a flicker very similar to the pink frames (half of frame was good, the other half was shifted to the right). This was clearly issue #1. In crop mode, there was roughly a corrupted frame every 5 seconds.

- silent pictures in burst mode corrupted. This was mostly issue #2, but I've also got instances of #1. I've got corruption in roughly one out of 3 burst sequences in "best shots" mode. Downgraded to 1.1.3 to test the same code, all clean.

- silent pictures in slit-scan mode corrupted (clearly issue #2). Harder to reproduce (say one corrupted picture out of 20), but this gave a very clear hint about the source of the problem. Since in this mode I don't use any kind of EDMAC trickery (just plain memcpy), it was clear that issue #2 was from Canon's lv_save_raw code (which, if you remember, is just a debugging flag for LiveView, and it's not used in normal operation of the stock firmware - thus, it probably didn't get much testing coverage from Canon).

The behavior from raw_rec preview confirmed the hint from silent pictures (while raw recorder was idle, there was no EDMAC trickery either - the preview was displayed from Canon's buffer). So... the corruption was there in Canon's raw buffer, and was not caused by our EDMAC calls. I've smelled this a while ago here: http://www.magiclantern.fm/forum/index.php?topic=6567.0

How I've fixed it:

So, I took the edmac_memcpy routine, remembered a hint from g3gg0's first lv_rec implementation (https://bitbucket.org/hudson/magic-lantern/src/a9dfd7b719c0/modules/lv_rec/lv_rec.c) where he said the raw buffer can be extracted by attaching a write EDMAC channel to connection #0, and implemented just that.

Had a small surprise to notice that using EDMAC channel #18 (the one used by Canon's lv_save_raw) with my code resulted in issue #2 present (exactly the same corruption). I now use EDMAC channel #4 and it seems fine at first sight. Luckily, issue #1 seems completely gone (with both channels).

Result: a new raw backend, codenamed CONFIG_EDMAC_RAW_SLURP, that no longer relies on Canon's lv_save_raw. So far, I've got no corrupted silent pics (out of a few hundred) and raw_rec's preview had zero flicker for several minutes (tried all 3 modes - 1080p, 720p and crop). I didn't record much, but so far it seems clean even with Magic Zoom enabled at 60fps. Of course, MZ didn't flicker at all and there were no corrupted frames.

Relevant changesets:

The fix itself:
https://bitbucket.org/chris_miller/magic-lantern/commits/0c5c1c5683dfa7e15dba3024b8fdf7c7fde89f6c

Preparing for the fix (apply in reverse order):
https://bitbucket.org/chris_miller/magic-lantern/commits/5ba68a12e0c6969286c3d21d269d7574b1aa294c
https://bitbucket.org/chris_miller/magic-lantern/commits/61424cefadc03affe647eeb51d786ac22a981469
https://bitbucket.org/chris_miller/magic-lantern/commits/00647042c11a8f764b0c91ce7e825ee27e3e0f0b

Notes:
- I had to hardcode the raw resolutions in all video modes. Fortunately, on 5D3 there are only 3 of them.
- The raw buffer is now allocated by ML. I actually see this as an advantage, because the memory backend does some sanity checks. Sure, right now you lose a few MB of RAM, but you can always recover the old buffer (hopefully allocated by Canon code, but unused) and have it managed by the memory backend. Nanomad knows what I'm talking about.
- I've only tried it on 5D3 123 raw_rec. Trying on 113 is straightforward. To port on other camera, you need to find another free write channel (find_free_edmac_channels is there to help you with that) and hardcode the resolution (must be exact, copy it from Debug->EDMAC and don't forget to add 1 to height).
- There were no changes in the raw recorder code, so I expect mlv_rec to work out of the box.
- TODO: ensure thread safety in raw.c, maybe try some other EDMAC channels, and if this approach proves rock solid, I hope to replace the old code completely.

May the pink frames be gone :D

P.S. If you remember this animation, now it should be a lot less likely to drop useful data (even on full CPU load).
(http://www.lensrentals.com/blog/media/2013/08/lucylantern1.gif)
Title: Re: Fix for pink frames
Post by: dmilligan on February 11, 2014, 05:33:04 PM
might this also provide a workaround for lv_save_raw not working on the 1100D?
Title: Re: Fix for pink frames
Post by: a1ex on February 11, 2014, 06:02:31 PM
Worth trying.
Title: Re: Fix for pink frames
Post by: 1% on February 11, 2014, 06:13:13 PM
How did it affect speed... I remember trying different channels on 50D/6D/7D and picking the fastest ones.. I guess easiest way to see will be if MZ flickers or not.
Title: Re: Fix for pink frames
Post by: a1ex on February 11, 2014, 06:25:36 PM
Benchmarks welcome (didn't do any). MZ doesn't flicker (but I don't remember it flickering on 5D3, not even at 60fps).

edit: did a quick test in crop mode, 2240x954, 30fps, hacked previews, both are running around 99-102 MB/s.
Title: Re: Fix for pink frames
Post by: nanomad on February 11, 2014, 07:29:44 PM
It should help the 1100D to get raw LV hopefully as this won't fire up any of the Canon debug stuff. Raw recording not so much I suppose.
Title: Re: Fix for pink frames
Post by: pompeiisneaks on February 11, 2014, 08:15:36 PM
I tested this out, and not getting pink frames so far, but I do seem to get every other recording to say "Invalid RAW" quickly so I can't play them back.  This is with regular RAW, for some reason the mlv_rec module is failing to build with:


[ GCC      ]   mlv_dump
mlv_dump.c: In function 'lua_call_va':
mlv_dump.c:64: warning: unused variable 'verbose'
mlv_dump.c:63: warning: unused variable 'nres'
mlv_dump.c:63: warning: unused variable 'narg'
In file included from ../lv_rec/lv_rec.h:24,
                 from mlv_dump.c:305:
../../src/raw.h: At top level:
../../src/raw.h:185: warning: declaration does not declare anything
../../src/raw.h:193: warning: declaration does not declare anything
mlv_dump.c: In function 'main':
mlv_dump.c:2027: error: 'struct raw_info' has no member named 'active_area'
mlv_dump.c:2028: error: 'struct raw_info' has no member named 'active_area'
mlv_dump.c:2029: error: 'struct raw_info' has no member named 'jpeg'
mlv_dump.c:2030: error: 'struct raw_info' has no member named 'jpeg'
mlv_dump.c:2036: error: 'struct raw_info' has no member named 'active_area'
mlv_dump.c:2037: error: 'struct raw_info' has no member named 'active_area'
mlv_dump.c:2038: error: 'struct raw_info' has no member named 'jpeg'
mlv_dump.c:2039: error: 'struct raw_info' has no member named 'jpeg'
mlv_dump.c:2557: error: 'struct raw_info' has no member named 'active_area'
mlv_dump.c:2558: error: 'struct raw_info' has no member named 'active_area'
mlv_dump.c:2559: error: 'struct raw_info' has no member named 'active_area'
mlv_dump.c:2560: error: 'struct raw_info' has no member named 'active_area'
make: *** [mlv_dump] Error 1


Title: Re: Fix for pink frames
Post by: a1ex on February 11, 2014, 08:19:46 PM
Never got "Invalid RAW" (tried around 50 clips). How does this message look like?
Title: Re: Fix for pink frames
Post by: 1% on February 11, 2014, 08:32:29 PM
How do the edmacs go now? 18 is 0x11 so is it hex->dec +1?

Also which edmac to copy the sizes from? 0x18? 0x21?
Title: Re: Fix for pink frames
Post by: pompeiisneaks on February 11, 2014, 08:38:39 PM
It happens fast, but it seems like when I press the play button to view the file it just is a black screen with white text, I just got it again:

Invalid Raw:
A:/pathtocard/file

but then I just tested and if I exit and reload then I can view the sequence just fine.  Odd.

Oh I just got the screen to stay:

Invalid RAW:
A:/DCIM/100EOS5D/M11-1226.RAW

I have 'no image' showing up too, so maybe there's some other corrupt file on the card I need to remove and it is unrelated.  I'm guessing it is safe to ignore for now.

~Phil
Title: Re: Fix for pink frames
Post by: a1ex on February 11, 2014, 08:49:47 PM
Commited it to main tree: https://bitbucket.org/hudson/magic-lantern/pull-request/382/fix-for-pink-frames

Also ported on 60D. No more pink frames in grayscale preview at 2.5K. This commit should help you see what exactly you need to port it on your camera: https://bitbucket.org/hudson/magic-lantern/commits/a32489b6c009

Nanomad confirmed it working on 1100D.
Title: Re: Fix for pink frames
Post by: 1% on February 11, 2014, 09:57:54 PM
I did all that but I'm getting garbage pics... edmac 0 is only 1440x200 or something like that... the others keep repeating resolutions that change and seem to correspond to something close to what the old pics were. Some are odd, one is even around edm 10 (according to edmac printout in debug).

I'm using free edmac channel 6... should I write them the same as the old channels were... ie 0x18, 0x6, etc? Any better way to verify the res?
Title: Re: Fix for pink frames
Post by: a1ex on February 11, 2014, 10:03:43 PM
It's not edmac #0, it's connection #0. To copy the resolution, enable raw with the old method, then look at RAW_LV_EDMAC. Width: convert it from bytes to pixels. Height: add 1 to the value from edmac.

Channel: on 5D3 I've got 4, 8 and 15 free from the test routine, but only 4 worked well.
Title: Re: Fix for pink frames
Post by: pilgrim on February 11, 2014, 10:40:37 PM
@ Alex, I admire your instinct... you sniff the problem on the right place  ;) as usual.
@ 1% , nanomad, dmiligan, pompeiisneaks... I am happy that you understand well also and directly implement on the camera which correspond with your development.

I don't know the programming of ML, so I always wonder with all of your works. Today I took some pictures with dual-iso. I got some pink frames but much less than before, and also because with ambiance lots of over-expose. I use the night build Feb. 10,  2014.
Once again, thank you very much for the good news and your nice collaboration.
Title: Re: Fix for pink frames
Post by: a1ex on February 11, 2014, 11:40:36 PM
Dual ISO has nothing to do with raw video pink frames.

Solved memory conflicts by reusing Canon's raw buffer address. This means one more thing to hardcode though, but the code is much simpler now.
Title: Re: Fix for pink frames
Post by: nanomad on February 12, 2014, 12:13:18 AM
1100D is recording raw video. 640p @ 30 fps, but it's raw video nonetheless, right?  8)

And it even plays back the video too. This is a lesson for me: never throw the towel :P
Title: Re: Fix for pink frames
Post by: 1% on February 12, 2014, 02:12:40 AM
Hehe, I'm still getting garbage... even though my silent pics are the same dimensions now.. I guess I'll shamem the Raw_LV_Edmac the old way (exact dimensions) and try to xref the new memory addresses.

I did this: uint32_t raw_write_chan = 0x11;

free indexes on 6D: /* Write: 6 8 15 */
Title: Re: Fix for pink frames
Post by: a1ex on February 12, 2014, 07:24:30 AM
You won't be able to read the edmac the old way because you have to call lv_save_raw for that. Simply hardcode the numbers returned by old EDMAC and should be fine.

The behavior of write channel is still a mystery. Why some of them work and others result in corrupted image? (or sometimes no image at all?) even if they seem unused in the EDMAC dialog?
Title: Re: Fix for pink frames
Post by: SteveScout on February 12, 2014, 12:28:55 PM
Can we help testing? Will this be in the next nightly build or do we have to download the fixed files individually from your bitbucket link and copy them onto our ML card?

thanks for looking into this!!
Title: Re: Fix for pink frames
Post by: a1ex on February 12, 2014, 12:31:05 PM
5D2 is odd: before it didn't have pink frames, but with the new method it has.

It does have some other kind of corrupted frames, like these: http://www.magiclantern.fm/forum/index.php?topic=5763 .

What's interesting is that poking 0xc0f08114 (raw type register aka KindOfCraw) was essential to get a continuous raw feed.
Title: Re: Fix for pink frames
Post by: a1ex on February 12, 2014, 12:50:16 PM
To test it, you should first be able to reproduce the issue with current nightly builds. Let me know if you are (and what camera you are using).

Right now, the fix is available only as source code, for 5D3 (both firmware versions) and 60D.
Title: Re: Fix for pink frames
Post by: 1% on February 12, 2014, 03:37:44 PM
QuoteThe behavior of write channel is still a mystery. Why some of them work and others result in corrupted image? (or sometimes no image at all?) even if they seem unused in the EDMAC dialog?

Some of the edmacs I tried resulted in the LV slowing down to a crawl. I tried 6, 8, 0x11(w-edmac8) and 0x21 (edmac 15), 15. None give me a good image, height + w is copied. When I move the camera around I can kinda see images through what looks like the matrix in the B/W preview.

I'll have to try 50D and 7D and see if its better/faster/worse/nothing
Title: Re: Fix for pink frames
Post by: nanomad on February 12, 2014, 08:07:20 PM
Are you sure the size is correct?
Title: Re: Fix for pink frames
Post by: 1% on February 12, 2014, 08:50:14 PM
Should be...

I did this before changing over:


/* Edmac Sizer */
uint32_t lv_raw_height = shamem_read(RAW_LV_EDMAC+4);
    uint32_t lv_raw_size = shamem_read(RAW_LV_EDMAC+8);
    int pitch = lv_raw_size & 0xFFFF;
    int width = pitch * 8 / 14;
    int height = MAX((lv_raw_height & 0xFFFF) + 1, ((lv_raw_size >> 16) & 0xFFFF) + 1);
    NotifyBox(5000, "H: %d, S: %d, W: %d", height, lv_raw_size, width);



Some mild success on 7D. I can record in 5x... but performance is much lower and there are MORE pink frames. I'm still looking for a memory address that shows a raw buffer there too.... also same raw address in 1x is black while in 5x is a good raw image.


Also interesting: 6D addy 0x76D48 shows the same address as a shamem from the raw LV edmac + 0x40000000... subtracting it didn't help, same matrix style image.
Title: Re: Fix for pink frames
Post by: andy kh on February 13, 2014, 02:52:04 AM
i test today's nightly on 550D. there are pink frames.
Title: Re: Fix for pink frames
Post by: duey101 on February 14, 2014, 09:19:38 AM
[EDIT: Sorry a1ex, I added it to my profile.] 

I tested today also with the Feb. 12th build [EDIT - used the wrong version] and now I get pink corrupted frames on all takes (usually within the first 5 frames).  MLV
Title: Re: Fix for pink frames
Post by: a1ex on February 14, 2014, 09:23:24 AM
Always specify your camera.

Do you get pink frames with raw_rec?

Also, my calendar might be broken, but it shows Feb 14. I've only announced the fix yesterday, and only for 5D3 and partially for 60D. (http://www.magiclantern.fm/forum/index.php?topic=3072.msg101469#msg101469)

Quote from: a1ex on February 13, 2014, 09:01:49 PM
- pink frame fix for 5D3 (http://www.magiclantern.fm/forum/index.php?topic=10443) (I've only tried on 123, let me know if it works on 113)
- partial pink frame fix for 60D (seems OK in crop mode at 2.5K, but still got pink frames at 60fps)
Title: Re: Fix for pink frames
Post by: duey101 on February 14, 2014, 11:52:49 AM
Okay, I just ran another set of shots using this build (magiclantern-v2.3.NEXT.2014Feb14.60D111) ...

(http://www.dueymoore.com/pinklatest.jpg)

RAW takes 1 & 5 I got pink frames, but only on frame 1.
MLV takes 2, 3, & 5 I got pink frames, again, only on the first frame. 

I also noticed the glitchy frames are ONLY in the first frame, and all look very similar, like a pattern!  ;)  a1ex, you may also want to reference the pixel bug issues being discussed on this thread as well.  http://www.magiclantern.fm/forum/index.php?topic=5618.new#new

I don't know if that will be useful for you but I have been doing extensive testing with raw2Cdng and have these weird buggy pixel issues, but only in RAW/MLV converted files.
Title: Re: Fix for pink frames
Post by: a1ex on February 14, 2014, 12:10:06 PM
If it's only in the first frame, it may be from some initialization and may give a hint about what's still missing; will check. The EDMAC is not yet fully understood (for example I couldn't get this working well on 5D2).

The cold pixels are fixed in raw2dng, and this one is the reference converter maintained by me; raw2cdng does not even appear to be open source, so there's nothing I can do. I'm not even sure if these closed-source converters are respecting our copyright on raw2dng/mlv2dng (we have released them as GPL, not as public domain code).
Title: Re: Fix for pink frames
Post by: duey101 on February 14, 2014, 02:37:39 PM
Okay, thank you very much a1ex for clarifying, and I think now that I only have them at the first frame does indicate a pattern which may be helpful for you.  Before I was getting these pink frames in the middle and end of clips so you are on the right track, now it is only at the first frame and all other frames are clean. 

I'll shoot a set of 10 each MLV/RAW tomorrow and see what percentage of pinkies I get.  If you need my help with more 60D specific testing let me know, I'm glad to help.  Right now I can get sustained 7 second clips on my 60D and only first frame corruption in 1280x720 which scales up just fine.  And of course with frame skipping on I get continuous recording (I shot for 90 seconds before turning record off, but of course I had lots of skipped frames.

So if I convert with raw2dng I won't see these cold pixels?  Hmm... I thought I tried that.  Let me check.  But those files don't work in PPro either (dng) only the Cdng files work if I try to import them.
Title: Re: Fix for pink frames
Post by: a1ex on February 14, 2014, 02:43:36 PM
Okay, make sure you try this one: http://www.magiclantern.fm/forum/index.php?topic=9700.msg98302#msg98302

I've almost got the same fix working on 550D, and indeed, the first frame is corrupted. But if all the others are OK, it's still an improvement.

g3gg0 is also doing some related research here: https://bitbucket.org/hudson/magic-lantern/branch/mlv_rec_edmac_tests
Title: Re: Fix for pink frames
Post by: duey101 on February 14, 2014, 05:57:48 PM
I did indeed try that one just now and I can open the dng files in After Effects (with ACR popup first) but they don't open in PPro, which of course is what is needed.  In AE the files are clean, no funky pixels!  Sadly, they don't work in PPro.
Title: Re: Fix for pink frames
Post by: 1% on February 14, 2014, 06:15:59 PM
Quote(for example I couldn't get this working well on 5D2).

So I guess 7D/50D are going to be out... I haven't looked for a place on 50D where I can get the buffer addy.. did you find one on 5DII? Was it also only recording in crop mode? EOSM/650D seems like a good candidate to try this as well.
Title: Re: Fix for pink frames
Post by: a1ex on February 14, 2014, 06:20:01 PM
Nope, I've malloced one.

550D is weird: when lucky it gets perfect sync, when unlucky it has continuous pink frames.

Also, it looks like you must set the preferred raw type for every frame (without it you may not get any raw data at all). So this time it's no longer preferred raw type, it's mandatory.
Title: Re: Fix for pink frames
Post by: 1% on February 14, 2014, 06:40:57 PM
Maybe that is why 6D didn't work and got garbage data. When the canon test routine is on the buffer never changes so in theory it could be hardcoded like the LV buffers were.
Title: Re: Fix for pink frames
Post by: Thejungle on February 15, 2014, 01:11:57 AM
So... what about 600d? :)
Title: Re: Fix for pink frames
Post by: 1% on February 15, 2014, 02:55:45 AM
Its like 60D so it might be worth a try... but remember when we were using raw type and got all of those tiny lines everywhere? Its probably going to be like that.

On the plus side nanomad found another sync function that might fix the MZ better.

BTW setting the raw type on 6D didn't help.


Ok, did this for 600D... also used the 1100D sync for MZ, it now works in 60p. Commits are at the top: https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/commits/all

Much fewer pink frames, seems to write as fast as it did (try more edmacs if you want)... but all of the raw type squiggles are everywhere. No DEFC at ALL. 60P + gd off only pinked on the first frame, 24P is pretty decent.
Title: Re: Fix for pink frames
Post by: duey101 on February 15, 2014, 06:38:47 PM
Good news!!!  :D

Okay, I ran a huge battery of tests today using this recent build magiclantern-v2.3.NEXT.2014Feb14.60D111 (The Valentine's Day build). My card is a Sandisk Extreme Pro 95 MB/s (SD card).  Pink frames are listed in their frame number(s).

I started in sRAW...

Canon menu: sRaw, 1920x1080 24

(raw_rec.mo) Canon menu: mRAW
24 fps override
Shot 1  PinkFrame 36            (141 frames)
Shot 2  PinkFrame 11,71            (169 frames)
Shot 3  clean                        (169 frames)
Shot 4  clean                       (170 frames)
Shot 5  clean                       (173 frames)
Shot 6  PinkFrame 47,137,167         (170 frames)
Shot 7  clean                       (169 frames)
Shot 8  clean                       (168 frames)
Shot 9  PinkFrame 1,5         (170 frames)
Shot 10 PinkFrame 1         (173 frames)

This was giving me a headache seeing all these pink frames still  >:(  so I switched into mRAW in the Canon menu and things got a little better.

(raw_rec.mo) Canon menu: mRAW
FPS override - 23.976 FPS
Shot 1  clean                    (197 frames)
Shot 2  clean               (211 frames)
Shot 3  PinkFrame 117      (214 frames)
Shot 4  clean               (212 frames)
Shot 5  clean               (211 frames)
Shot 6  PinkFrame 45,105   (213 frames)
Shot 7  PinkFrame 71      (214 frames)
Shot 8  clean               (212 frames)
Shot 9  PinkFrame 193      (210 frames)
Shot 10 clean             (211 frames)

(raw_rec.mo) Canon menu: mRAW
FPS override - 24 FPS
Shot 1  clean            (197 frames)
Shot 2  PinkFrame 2,139         (211 frames)
Shot 3  clean               (212 frames)
Shot 4  clean               (211 frames)
Shot 5  clean               (214 frames)
Shot 6  clean               (213 frames)
Shot 7  clean               (213 frames)
Shot 8  clean               (214 frames)
Shot 9  clean               (213 frames)
Shot 10 clean               (179 frames)

With mRAW and MLV mode the results are dramatically improved!  Only one pink frame out of 2114!
MLV/ (mlv_rec.mo) Canon menu: mRAW
FPS override - 23.976 FPS
Shot 1  clean            (207 frames)
Shot 2  clean               (212 frames)
Shot 3  clean            (212 frames)
Shot 4  clean               (211 frames)
Shot 5  clean               (212 frames)
Shot 6  clean            (213 frames)
Shot 7  PinkFrame 1         (212 frames)
Shot 8  clean               (212 frames)
Shot 9  clean            (212 frames)
Shot 10 clean            (211 frames)

MLV/ (mlv_rec.mo) Canon menu: mRAW...  The sweet spot!  A perfect score! 2113 frames and no pink frames!  Is it the 60D Holy Grail?
FPS override - 24 FPS
Shot 1  clean                    (213 frames)
Shot 2  clean               (212 frames)
Shot 3  clean            (212 frames)
Shot 4  clean               (211 frames)
Shot 5  clean               (211 frames)
Shot 6  clean            (212 frames)
Shot 7  clean            (212 frames)
Shot 8  clean               (212 frames)
Shot 9  clean            (206 frames)
Shot 10 clean            (212 frames)

In full RAW 5184x3456 with mlv_rec.mo (MLV mode) it got even better and I got full takes without any issues.
24 FPS override NO PINK FRAMES!
23.976 FPS override NO PINK FRAMES!
I can get sustained 210-217 frames per shot. (8-9 seconds)

I think three key settings I changed from previous attempts were...
Auto power off - off (was on)
Auto preview - off (was set at 2 seconds)
sRaw 2592x1728 (changed to mRaw 3888x2592 and RAW 5184x3456))

MLV MODE in mRAW and RAW is what works on my 60D right now, 10 out of 10 shots and no pink frames or any other kind of glitches.  I can shoot in RAW!!!  :D

Here are all my pertinent settings both in ML and the Canon camera menus in a text document...

www.dueymoore.com/raw_settings.txt (http://www.dueymoore.com/raw_settings.txt)

Title: Re: Fix for pink frames
Post by: andy kh on February 16, 2014, 05:04:24 AM
tried today's nightly on 550d. i get lots of pink frames
Title: Re: Fix for pink frames
Post by: duey101 on February 16, 2014, 05:54:13 AM
@ andy kh 

Oh... unfortunate.  Did you check all my Canon/ML settings?  I think these three things had something to do with it.

Auto power off - off (was on)
Auto preview - off (was set at 2 seconds)
sRaw 2592x1728 (changed to mRaw 3888x2592 and RAW 5184x3456)
Title: Re: Fix for pink frames
Post by: duey101 on February 16, 2014, 06:27:02 AM
One more note, on the newest build "magiclantern-v2.3.NEXT.2014Feb16.60D111" I now get an additional 40+ frames in 1280x720 mlv_rec.mo in RAW (Canon menu: Quality - RAW 5184x3456)  That is a bonus as now I get 10.20 -10.45 second clips and that is very usable.  Thanks a1ex!  Whatever you did seems to be working on the 60D!  I'm still running further tests now on crop 5X-10X/60FPS.
Title: Re: Fix for pink frames
Post by: andy kh on February 16, 2014, 06:48:10 AM
Quote from: duey101 on February 16, 2014, 05:54:13 AM
@ andy kh 

Oh... unfortunate.  Did you check all my Canon/ML settings?  I think these three things had something to do with it.

Auto power off - off (was on)
Auto preview - off (was set at 2 seconds)
sRaw 2592x1728 (changed to mRaw 3888x2592 and RAW 5184x3456)
[/

it doesnt work with 550D
Title: Re: Fix for pink frames
Post by: a1ex on February 16, 2014, 07:50:59 AM
Nobody announced pink frame fix in nightly builds for 550D. Let me repeat:

Quote from: a1ex on February 13, 2014, 09:01:49 PM
- pink frame fix for 5D3 (http://www.magiclantern.fm/forum/index.php?topic=10443) (I've only tried on 123, let me know if it works on 113)
- partial pink frame fix for 60D (seems OK in crop mode at 2.5K, but still got pink frames at 60fps)
Title: Re: Fix for pink frames
Post by: duey101 on February 17, 2014, 06:04:37 AM
@ a1ex  After further extensive testing I can report that I can shoot in mlv_rec.mo/raw_rec.mo with mRaw/RAW (Canon menu) with no pink frames and also the same in Magic Zoom 5X and 10X, no pink frames zoomed in.  All output is clean and perfect with raw2dng_escho with no sensels/cold/hot/dead pixels.  Perfect raw and output on my 60D.  Looks good.  Any changes you want me to test for 60D?

Same format as before 10 shots per setting, no pink frames in 10/10.
Title: Re: Fix for pink frames
Post by: a1ex on February 17, 2014, 06:18:56 AM
Nice!

Can you do some tests in 50/60fps, just like the ones you did before, so I could estimate a rough probability for pink frames in this mode?
Title: Re: Fix for pink frames
Post by: duey101 on February 17, 2014, 11:36:59 AM
Okay, will do.  If only the raw2dng_escho output would go straight into Premiere Pro CC... it's a dream come true.   :D
Title: Re: Fix for pink frames
Post by: a1ex on February 17, 2014, 11:59:38 AM
Sure, just convince chmee and Thomas Worth to share their source code.
Title: Re: Fix for pink frames
Post by: duey101 on February 17, 2014, 05:12:11 PM
After lots of more testing in 60i here are the results...

mlv_rec.mo
Canon menu: Quality: RAW > 1280x720 60i / ML menu:
FPS Override > 60fps
Clean > average 220 frames 10/10 shots no pink frames.

FPS Override > 60fps 5X Magic Zoom
Clean > average 205 frames 10/10 shots no pink frames.

FPS Override > 60fps  10X Magic Zoom
Clean > average 215 frames 10/10 shots no pink frames.


raw_rec.mo
Canon menu: Quality: RAW > 1280x720 60i / ML menu:
FPS Override > 60fps
Clean > average 227 frames 10/10 shots no pink frames.

FPS Override > 60fps 5X Magic Zoom
Clean > average 203 frames 10/10 shots no pink frames.

FPS Override > 60fps  10X Magic Zoom
Clean > average 205 frames 10/10 shots no pink frames.

Total Pink Frame Count = 0

#include <iostream>

using namespace std;

int main ()

cout << "Those are really great results for the 60D"  ;

cout << "Now all I need is a way to go straight to PPro!  ;

return 0;
}


I just took my first C++ lessons today!
Title: Re: Fix for pink frames
Post by: a1ex on February 17, 2014, 05:14:47 PM
Hm... then why I've got a bunch of pink frames in all my tests at 60fps?!

=> will check again.
Title: Re: Fix for pink frames
Post by: duey101 on February 17, 2014, 06:24:21 PM
I don't know but right now this is the FW I'm staying on for now because it works!  Any more ideas?  Let me know if you need more tests.
Title: Re: Fix for pink frames
Post by: andy kh on February 18, 2014, 05:35:53 PM
there is no point of doing test by recording 100- 200 frames. i dont have any pink frames recording only 200 frames on my 550D. problems starts when i record 1000 plus frames. i test by recording 1000 - 3000 frames
Title: Re: Fix for pink frames
Post by: g3gg0 on February 18, 2014, 05:44:27 PM
what impresses me even more is the fact that you get more frames with MLV in some tests...
mlv_rec used to be slower than raw_rec...
Title: Re: Fix for pink frames
Post by: duey101 on February 20, 2014, 06:10:05 PM
Well, that's all I get before it stops automatically.  >:( I WISH I could get 1000 frames!  MLV_REC works well.  I guess my 60D is limited by its hardware.
Title: Re: Fix for pink frames
Post by: andy kh on February 20, 2014, 07:34:51 PM
lower the resolution and u should be able to shoot continous... u may not like low resolution for your project but u need to record for few mins to test. its difficult to say no more pink frames by recording 5/6 secs.
Title: Re: Fix for pink frames
Post by: duey101 on February 21, 2014, 05:39:54 AM
Although I may be able to shoot continuous in low res, it doesn't serve my purpose as I would not shoot that way in real case scenarios.  I want to shoot at the resolutions I need to work with, so if I shot at 640x480 and could shoot for 30 minutes it still isn't going to be anything I would ever use.  I am interested in what I can shoot and use in my projects and right now what I get works without pink frames.  However I'll give it a go in low res and see what happens.
Title: Re: Fix for pink frames
Post by: ted ramasola on February 21, 2014, 05:45:42 AM
Another thing to note is that certain resolutions might trigger pink frames more than others.
Title: Re: Fix for pink frames
Post by: andy kh on February 21, 2014, 01:59:02 PM
i just came to know why i get lot of pink  frames sometimes. it happens if i press magic zoom while recording
Title: Re: Fix for pink frames
Post by: andy kh on February 22, 2014, 04:27:40 AM
wow!! no more pink frames for 550D with today's nightly. i recorded 4 clips. 2 of them 900 frames@1152X460 and other two clips 2000 plus frames @1152X432 and i dont get any pink frames in all my clips. this is wonderful
Title: Re: Fix for pink frames
Post by: andy kh on February 22, 2014, 06:11:33 AM
did another test. recorded 4 more clips. this time recorded 3000 plus frames and not a single pink frame in all the clips. amazing
Title: Re: Fix for pink frames
Post by: mk11174 on February 22, 2014, 08:54:02 AM
Quote from: andy kh on February 22, 2014, 04:27:40 AM
wow!! no more pink frames for 550D with today's nightly. i recorded 4 clips. 2 of them 900 frames@1152X460 and other two clips 2000 plus frames @1152X432 and i dont get any pink frames in all my clips. this is wonderful
I am wondering if it is just coincidence that you are not getting pink frames? I don't remember any pink frame fix added for 550D? Or did I miss a commit? Just curious because I have a 550D as well, just have not used it much since I got the 700D?
Title: Re: Fix for pink frames
Post by: a1ex on February 22, 2014, 09:06:53 AM
g3gg0 did some changes on the EDMAC side for mlv_rec. If the improvement is only on mlv_rec and not on raw_rec, it's from his changes.

Back then I ran my tests without recording, but with grayscale preview on (this way I was getting a lot of pink frames).

May be worth checking both methods combined (5D3/60D users, please report, because you have both fixes applied).
Title: Re: Fix for pink frames
Post by: arrinkiiii on February 22, 2014, 02:44:00 PM

Until now, with the last build, i have not find any pink/cirrupted frames. Thanks you =)) 

GD= On
Buffer= 2

Title: Re: Fix for pink frames
Post by: g3gg0 on February 22, 2014, 05:06:54 PM
are you talking about mlv_rec or raw_rec?
Title: Re: Fix for pink frames
Post by: arrinkiiii on February 22, 2014, 05:44:37 PM

MLV_rec i use.  I stop making tests with raw_rec.  But need to test more, only recorded a few clips but with no pink/corrupted frames. 

What i found is wend you preview the files in camera (hit the play button) and then return to Lv the black box for 2.25:1 is out of the place, more down of the screen, need to turn off and on to correct this bug.

Title: Re: Fix for pink frames
Post by: andy kh on February 22, 2014, 06:07:49 PM
Quote from: mk11174 on February 22, 2014, 08:54:02 AM
I am wondering if it is just coincidence that you are not getting pink frames? I don't remember any pink frame fix added for 550D? Or did I miss a commit? Just curious because I have a 550D as well, just have not used it much since I got the 700D?
please try 2day's nighly. i am eager to hear from others. i recorded 4500 frames now and i found no pink issue
Title: Re: Fix for pink frames
Post by: andy kh on February 22, 2014, 06:42:57 PM
i have reported some good news on pink frame fix on 550D. let me also report that if i use magic zoom while recording i stil get pink frames
Title: Re: Fix for pink frames
Post by: mk11174 on February 22, 2014, 07:51:31 PM
Quote from: andy kh on February 22, 2014, 06:42:57 PM
i have reported some good news on pink frame fix on 550D. let me also report that if i use magic zoom while recording i stil get pink frames
Are you using mlv_rec or raw_rec?
Title: Re: Fix for pink frames
Post by: andy kh on February 22, 2014, 08:13:11 PM
i use mlv_rec.
Title: Re: Fix for pink frames
Post by: Thejungle on February 26, 2014, 11:30:26 PM
Yeah, pink frames were gone few nigltlies back in time, but newest versions are pink-infected again.. :(
I own 600D, you guys know any solutions?
Title: Re: Fix for pink frames
Post by: Audionut on February 27, 2014, 05:33:38 AM
Start by finding the nightly build that caused issues again.
Title: Re: Fix for pink frames
Post by: duey101 on February 28, 2014, 06:11:51 AM
I'm still good no pink frames but I have not updated past the Feb 16th build which works on my 60D.
Title: Re: Fix for pink frames
Post by: andy kh on March 07, 2014, 02:33:51 PM
is there any way to get rid of pink dots on 600D/700D? i know there is a pink dot removing tool but i want to know if there is any other solution?
Title: Re: Fix for pink frames
Post by: a1ex on March 07, 2014, 02:39:47 PM
raw2dng_cs2x2 from here: http://www.magiclantern.fm/forum/index.php?topic=5404.0

but pink dots are not the same as the pink frames discussed in this thread ;)
Title: Re: Fix for pink frames
Post by: andy kh on March 07, 2014, 02:54:48 PM
Quote from: a1ex on March 07, 2014, 02:39:47 PM
raw2dng_cs2x2 from here: http://www.magiclantern.fm/forum/index.php?topic=5404.0

but pink dots are not the same as the pink frames discussed in this thread ;)

oh i am sorry but i havent found any discussions on this. i like cdng and not dng. i think my last option is to sell my camera. i just sold my 550D last week after the pink frame fix and got this 650D thinking it wil be the same and it can record 720P but now i am little bit frustrated with pink dots on this camera. now its time to sell this camera. thank q anyway