Fixing random colour strips across DNGs?

Started by Tai.Fighter, November 24, 2013, 10:12:07 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Tai.Fighter

Hi everyone,

I tried shooting RAW with my recently acquired 5D Mark 2, the problem is that some frames from my DNGs have strange pink bands going across them.

Shooting 14-bit RAW 2.35:1 at 1872x796, Komputerbay 64GB 1050x Card, November 1st Nightly Build I believe. Unpacked through a few methods, including raw2dng.exe

Here is a sample of a DNG and what it's supposed to look like:




If anyone knows the solution to this, I'd greatly appreciate it. Thank you!


.

Kharak

Have a 5d2 and never seen that.

I think you should try reinstalling ml and see if that takes care of it.
once you go raw you never go back

hkarlsen

I saw the same stripes on some of my footage the other day. Have a 5dmrk3 on a late october build. Have used ML raw on four different small docus lately, and this one shooting day is the only one that has the stripes. Anyone know how to avoid it?

How do you "reinstall" ML? Just put a new version on the SD card, or do a new bootflag or something?

EDIT: This was on a new batch of KB64 cards. They otherwise perform great (no indication that anything is off). Could this be CF cards related?

Stedda

How about trying a more recent Nightly build.... you guys are running builds that are almost or over a month old. Things move fast around here.
5D Mark III -- 7D   SOLD -- EOS M 22mm 18-55mm STM -- Fuji X-T1 18-55 F2.8-F4 & 35 F1.4
Canon Glass   100L F2.8 IS -- 70-200L F4 -- 135L F2 -- 85 F1.8 -- 17-40L --  40 F2.8 -- 35 F2 IS  Sigma Glass  120-300 F2.8 OS -- 50 F1.4 -- 85 F1.4  Tamron Glass   24-70 2.8 VC   600EX-RT X3

hkarlsen

Quote from: Stedda on November 24, 2013, 02:24:15 PM
How about trying a more recent Nightly build.... you guys are running builds that are almost or over a month old. Things move fast around here.

Sure, will do. But do people using this for work actually use the very latest nightly build at any given time? I would think you stick to the ones that works safely..?

Stedda

That would be fine but I don't think the DEVs look at bugs from month old builds.... you could also go back to your old build anytime.
5D Mark III -- 7D   SOLD -- EOS M 22mm 18-55mm STM -- Fuji X-T1 18-55 F2.8-F4 & 35 F1.4
Canon Glass   100L F2.8 IS -- 70-200L F4 -- 135L F2 -- 85 F1.8 -- 17-40L --  40 F2.8 -- 35 F2 IS  Sigma Glass  120-300 F2.8 OS -- 50 F1.4 -- 85 F1.4  Tamron Glass   24-70 2.8 VC   600EX-RT X3

hkarlsen

Ok, thanks.  But is it possible this is a "bad sector" on a cf card? I would think the entire recording would stop if that was the case..right? Always nice to narrow down the causes..

Stedda

Could be.. I've never seen anything like it on my 5D III but I stay up to date every couple of days...
5D Mark III -- 7D   SOLD -- EOS M 22mm 18-55mm STM -- Fuji X-T1 18-55 F2.8-F4 & 35 F1.4
Canon Glass   100L F2.8 IS -- 70-200L F4 -- 135L F2 -- 85 F1.8 -- 17-40L --  40 F2.8 -- 35 F2 IS  Sigma Glass  120-300 F2.8 OS -- 50 F1.4 -- 85 F1.4  Tamron Glass   24-70 2.8 VC   600EX-RT X3

Midphase

Quote from: hkarlsen on November 24, 2013, 02:31:13 PM
Sure, will do. But do people using this for work actually use the very latest nightly build at any given time? I would think you stick to the ones that works safely..?

People shouldn't use a hack for work and the developers are very clear about this fine point. However I can see how ML's stability can tempt people to feel overly confident about it.

Here's the thing...nowadays the nightly builds are being updated, but not at the same frantic pace as a few months ago. Most of the new improvements concern optimization for .mlv files.

Some people are still using builds from July, and that's plain stupid. However if you're using a build from November 1st, it shouldn't be too different than one from today.

I don't think it's a build issue (although if you're going to re-install, you might as well update to the latest). The way to re-install (at least the way I do it) is this:

1. Format the card in camera
2. Run the Boot script to prepare the card for in-camera booting
3. Drag manually onto the card the ML folders and the auto exec.bin

You'll need to re-configure your ML settings and modules, but it's the safest way I know to make sure that the ML install is clean.

The only time I've seen this type of problem on my 5D3 has been when I was shooting in Crop mode....I suspect the errors are results of data not being written quite fast enough on the card and becoming corrupted here and there.

Tai.Fighter

Quote from: Midphase on November 24, 2013, 06:42:04 PM
The only time I've seen this type of problem on my 5D3 has been when I was shooting in Crop mode....I suspect the errors are results of data not being written quite fast enough on the card and becoming corrupted here and there.

Thanks, I see, so basically there's no hope in salvaging these?

Tai.Fighter

Quote from: Kharak on November 24, 2013, 02:15:24 PM
Have a 5d2 and never seen that.

I think you should try reinstalling ml and see if that takes care of it.

What build are you on? Are you recording the same specifications as me? Also what card are you using? I think if I could narrow it down it would help! Thanks

reddeercity

Quote from: Tai.Fighter on November 24, 2013, 10:12:07 AM
Hi everyone,

I tried shooting RAW with my recently acquired 5D Mark 2, the problem is that some frames from my DNGs have strange pink bands going across them.

Shooting 14-bit RAW 2.35:1 at 1872x796, Komputerbay 64GB 1050x Card, November 1st Nightly Build I believe. Unpacked through a few methods, including raw2dng.exe
Here is a sample of a DNG and what it's supposed to look like
If anyone knows the solution to this, I'd greatly appreciate it. Thank you!
I had this problem once on my 5D mark ii, back in July
I recommend that you use the Oct 24 built from A.D. Hack built for the 5D2
https://bitbucket.org/a_d_/laboratory/downloads/af9a60fe144d.zip
This one is the Best built to date and completely  Stable for Paid Work.
Plus I always Format my card in Camera before I shot raw, every time.
and use raw2cdng to extract.
Use Lexar 1000x 32,64 GB Max. size 1872x1250 23.976p for about 800 to 1000 frames
also Sandisk 90MB/s at lower frame size(1872x936 @23.976p ;)

Edit: I noticed that in ACR, the dng is set to 8bit, change that to 16bit in advance pulldown menu.

Tai.Fighter

Quote from: reddeercity on November 25, 2013, 02:34:28 AM
I had this problem once on my 5D mark ii, back in July
I recommend that you use the Oct 24 built from A.D. Hack built for the 5D2
https://bitbucket.org/a_d_/laboratory/downloads/af9a60fe144d.zip
This one is the Best built to date and completely  Stable for Paid Work.
Plus I always Format my card in Camera before I shot raw, every time.
and use raw2cdng to extract.
Use Lexar 1000x 32,64 GB Max. size 1872x1250 23.976p for about 800 to 1000 frames
also Sandisk 90MB/s at lower frame size(1872x936 @23.976p ;)

Edit: I noticed that in ACR, the dng is set to 8bit, change that to 16bit in advance pulldown menu.

Thank you so much! Hopefully this works out for me. You are a good man, sir.

hkarlsen

Quote from: reddeercity on November 25, 2013, 02:34:28 AM
I had this problem once on my 5D mark ii, back in July
I recommend that you use the Oct 24 built from A.D. Hack built for the 5D2
https://bitbucket.org/a_d_/laboratory/downloads/af9a60fe144d.zip
This one is the Best built to date and completely  Stable for Paid Work.
Plus I always Format my card in Camera before I shot raw, every time.
and use raw2cdng to extract.
Use Lexar 1000x 32,64 GB Max. size 1872x1250 23.976p for about 800 to 1000 frames
also Sandisk 90MB/s at lower frame size(1872x936 @23.976p ;)

Edit: I noticed that in ACR, the dng is set to 8bit, change that to 16bit in advance pulldown menu.

So youre not using exFAT formatting on the card? The camera formatting will make it FAT, right? Its a big one for me to have non-split files. If the problem is related to write speed as someone mentioned, Im still thinking its a card issue. What is peoples go to method for card testing outside of the benchmark function? Just shoot a card full, and speed through it in Resolve etc?

reddeercity

5D Mark II is Fat32 system, it can not work with exfat!
That why the file are in 4GB spanning.
I never test my cards with benchmark test,
Start at smaller frame size full card then
Keep moving up on frame size until you the
Size you want.
Some times you need to brake in the card.
I had to do this on my 64GB Lexar 1000x .
There is no other way to record raw on
5d2. If you do not want spanning file
Then stop before the 4GB mark :)


hkarlsen

Quote from: reddeercity on November 25, 2013, 06:19:32 PM
5D Mark II is Fat32 system, it can not work with exfat!
That why the file are in 4GB spanning.
I never test my cards with benchmark test,
Start at smaller frame size full card then
Keep moving up on frame size until you the
Size you want.
Some times you need to brake in the card.
I had to do this on my 64GB Lexar 1000x .
There is no other way to record raw on
5d2. If you do not want spanning file
Then stop before the 4GB mark :)
Forgot to mention I am on 5dmrk3 where exFAT works beautifully. But will keep your tip in mind :)

bnvm

I have seen this type of thing several times when I get a full memory card during recording. Other than that I have not seen anything like this since the very beginning of raw. I am tempted to say it is a faulty card that is corrupting some of the data.

pascal

Quote from: Midphase on November 24, 2013, 06:42:04 PM
Some people are still using builds from July, and that's plain stupid.
Cannot agree with this statement. If an old build works perfectly fine for someones purposes then there is no urgent need to upgrade, since new iterations can also introduce new issues.
However since the topic starter posted this issue I recommend changing ML version and try different CF card.

spnsir

I've been using the latest nightly builds for the 5D3 and had this happen to me a couple times. Have not isolated why yet...

hkarlsen

Quote from: spnsir on November 27, 2013, 10:09:30 AM
I've been using the latest nightly builds for the 5D3 and had this happen to me a couple times. Have not isolated why yet...

Interesting. What type of cards are you on?

bnvm

Number all of your cards, then when copying your files to the HD include the card number somewhere - folder name / text file. I suspect this is due to a faulty card, if you can keep track of which files came from which card you may notice a pattern.

Midphase

Quote from: pascal on November 26, 2013, 09:36:02 PM
Cannot agree with this statement. If an old build works perfectly fine for someones purposes then there is no urgent need to upgrade, since new iterations can also introduce new issues.

I can agree that if you're using Nov. 26 build, maybe you don't really need to update to Nov 27, but since July there have been such substantial improvements which have made the code much more solid and less error prone. If an bug is accidentally introduced, it's usually taken care of fairly quickly.

Since July, ML has improved the was the buffers are handled resulting in better data writing even with less than great cards, it has improved playback substantially adding not only near-real time playback but also color high res playback. ETTR has been added and improved, peaking has been optimized, support for the new .mlv format has been perfected, the interface has been substantially refined resulting in an easier to read and access GUI. The list goes on and anyone sticking with a build from over 3 months ago really should consider updating to a newer build.

spnsir

Quote from: hkarlsen on November 27, 2013, 03:03:13 PM
Interesting. What type of cards are you on?

Komputerbay 64GB 1000x.. Using two cards.. Haven't isolated the problem to either one yet

a1ex

I believe it's caused by our incomplete understanding on how EDMAC works. I don't think it has anything to do with the card.

Lowering CPU usage might help (for example, turn off GlobalDraw).

1%

besides cpu, I think AE might touch the edmac too. turning it off helps a bunch with write speed on this generation of body. WB didn't have as much of an effect. of course the side effect of killing AE is you lose all exposure control while recording.

I got the cut up/pink frames from CPU more or less or canon drawing refreshing... i.e. turning on color preview full time.