Pink hue/Accidentally had "Fix Black Level" selected/Already converted to RAW 8(

[DAK29]

I am using a MARK 5D2.

Did the options for fix black level from the September build to the new Oct8 build change?

Because either there is a glitch or my wife messed up the settings.
The first time she was handed the camera, she accidentally hit the joystick and sent it whizzing through the
options... And when you bump that joystick you can accidentally hold it down too so not only does it fly
through all the options but it actually selects shit as well as it goes! When I saw this, I was like...
"Okay. Do you see what just happened? Do not ever go near the joystick again."
And I set it so she just can open the iso and change the iso/shutter settings with the top dial and never
have to go into menus or use joystick because for uncoordinated people, this REALLY can be dangerous!
She has been using the camera all summer and not messed up at all and she denies having messed with any
controls or settings and she says she has been very careful never to do so. (Even though I know sometimes
people who aren't as into computers as I am can accidentally hit shit and not realize they did it but I don't
really think she did anything this time, I think it is a glitch. However, I will tell this story so people know it's possible.)

I don't recall ever having selected fix black level but I upgraded to Oct build and shot
today's footage on it. I converted the MLV to RAW, loaded into PREMIERE and all the RAW stuff is pink hued.
However, I further verify that the original MLVs were pink also because of the "fix black level" option being on.
So this is not an issue which occurred during MLV to RAW/WAV conversion.

To discover why things happened, I went back into the camera and filmed more and then played it back on the camera it's self.
Again... Pink Hue! I knew at that point the issue was in the recording process it's self and not in the conversion.
So I started looking through options... And "fix black level" was turned on!

I turned "fix black level" off and shot some footage and then played that on the camera.
It played back perfectly! Everything is recording normally now... But the option "fix black level" is gone!
(After I turned it off, the camera is now recording properly plus the option "fix black level" does not appear.)

Did the new build remove the option and this cause a glitch?

And here is the part I'm really worried about - I already converted my MLV to RAW and do not have the original MLVs now.
So all my RAW stuff has the pink hue!!! Help! I remember people talking about fixing this... How?
I import my RAW stuff into PREMIERE using Mediareactor's plugin and it works awesome.
My normal workflow is I shoot RAW and have it save to MLV then I use MLV MYSTIC to convert to RAW and WAV files.
Then I import them into PREMIERE directly.

This has always worked awesome for me and will continue to work now that I shut the fix black levels option
off but I need to know if there's a way to fix the RAW footage that has the pink hue issue?

Anyone?

I hope this is not off topic! Please move to correct location if so, I do not mean to be off topic!

Also, this could be a bug report and if so, the bug is likely that if you had "fix black level" off
on the last September build and then upgrade to the October build, it comes up on.
Then if you turn it off, the entire option it's self seems to go away. (But it does turn it off)   

Thanks for any help!

I posted a long time ago when I was a noob, have been shooting RAW all summer.
Have Terrabytes of footage, have learned soooo much! Shooting RAW is so awesome!

I am so grateful to the Magic Lantern team, you guys have done a spectacular job.

[Audionut]

If you have the DNGs http://www.magiclantern.fm/forum/index.php?topic=11664.0

If you've converted to some intermediate format, you may be SOOL.

[DAK29]

Have the DNGS? How would I have DNG? I would have had to convert the MLV to DNG.
I never use DNG. I use MLV and RAW. I have the RAW files as stated above.
Nothing has been "converted" in that, in my understanding, the data in RAW and MLV is the same except
for the fact that MLV has WAV included in it and a few different headers.

I will check the link though, doesn't it show how to fix in one of the adobe programs?
Because I can import RAW directly into any of my Adobe software. (Premiere/Photoshop/After Effects/Etc.)
I have just never used anything but Photoshop & Premiere.

The link says to use MLVDUMP - Is there a way to take RAW & WAV files and mux them back into MLV?
My problem is that I need to perform the procedure on RAW files. I really don't want to have to convert
RAW to DNG either as I have never been able to get DNG working good on my workflow & can't stand having
all the frames be in individual files. I really don't like DNG at all and hope to avoid having to convert to it.

EXIFTOOL claims to support RAW but it is not working - What does it mean, "this type of RAW" ?
It's a RAW file that came from an MLV from MLV MYSTIC.

C:\OCT26>"j:\EXIFTOOL\exiftool(-k).exe" -BlackLevel=1024 M20-0406.RAW -o OUT
Error: Writing this type of RAW file is not supported - M20-0406.RAW
    0 image files updated
    1 files weren't updated due to errors

[Andy600]

@DAK29 - It's probably because the DNG files from MLVMystic are not properly validated. You could try running the DNG files through Adobe DNG converter first, then try EXIFTool again.

[Audionut]

Supports raw, as in raw image files, not raw ML files.

Try mlv_dump, instructions in the link I posted above.

[DAK29]

andy: For the millionth time - They are NOT DNGS. Are you people reading what I posted?

Audionut: Again, I do not see instructions on dealing with RAW video files. If I've missed that part, please point it out to me.

MLVDUMP = DOES NOT DEAL WITH RAW FILES

Everyone keeps assuming I use their workflow, I guess because most people can't import RAW into Premiere.

I need an expert here to explain what exactly happens to the video when the "black fix" is applied and how to fix it.
(Does it change meta data or the actual data? Does RAW have any meta data fields? Etc.)
My RAW video files are screwed because it was turned ON somehow, either accidentally or because of a bug which I've
already described above. ON means it's supposed to fix the range level, not break it.
Nevertheless, it was turned on at the camera level and the exported MLVs had the error.
I converted (It really is more of an extraction, no conversion is done with MLV Mystic) those MLVs to RAW.
I don't have the original MLVs now but RAW data is the same as MLV data except just not in the MLV "container"
and put together with WAVs. All MLV is is RAW + WAV plus some different headers/meta data.
If you look at an MLV file in a hex editor and outline the chunk where the video is, it is the SAME DATA as the RAW file.
If you don't understand this stuff, please don't comment.

Why do I use MLV only to "convert" (again, not really a "conversion") back to RAW & WAV?
- spanning suport (write to CF and SD in parallel to gain 20MiB/s <--- PRIMARILY BECAUSE OF THIS AWESOMENESS!
THANK YOU MAGIC LANTERN GENIUSES!

Perhaps there is a way to fix the video in After Effects? I think I remember someone posting how to do that.
But it is not going to help if people keep talking about DNG. (The format I have not converted anything into,
do not like, is not really lossless, creates a million individual still files instead of being a true video format,
is not reversible after you've converted to it and is, not to mention, the format I've specifically stated I want
to avoid dealing with if at all possible.)

RAW is RAW. There may not be a way to fix the "black fix" issue with RAW but if so, it is only because no one
has figured out how to do it yet because RAW is higher than DNG and is actually the same data that is in
an MLV file except for the fact MLV files also carry audio. SO... If it can be done with an MLV...
It can be done with a RAW video file. And there should already be a way to "mux" RAW+WAV to MLV since
this is essentially what Magic Lantern does in the camera it's self when it creates the MLV file.

A utility that "muxes" together an MLV file from RAW & WAV files should be easy enough to make if it doesn't already exist.
If the setting is in the meta data and not actually in the RAW video it's self then it should be even easier to fix as
long as I can get the RAW back into MLV which has more meta/header options.

What you guys are saying is I need to convert from a RAW format to something else in order to change data in the images.
That is ridiculous. That data can be changed within the RAW format. You're not looking at it from a binary level.
I edit everything in RAW. I load RAW directly into Premiere. RAW is the mother of all the formats.

AGAIN - Is there a way to fix this issue from within one of the CS6/CC suite's software? Like with After Effects? Or Premiere?
I have no issues importing my RAW files into any of those programs and wish to continue using pure RAW as the format
I do all my stuff in, I have Terrabytes of information which I wish to edit together into a documentary this Winter.
It's all in RAW & WAV files and that is exactly how I need it and want it. I'm simply trying to fix 1 day's fuck up.

[Audionut]

Heh, I thought mlv_dump could process the original raw files, but looks like it only outputs to original raw from MLV.

Looks like you're only option is to dump the raw file to DNGs, then process the black level.

[DAK29]

There has got to be a different solution. I am not converting my stuff to DNG.

It can be done with MLV right?


EDIT - For shits and giggles, I did a test of the DNG method. Just to verify that it can be fixed at all.

I converted one of my RAW videos to DNG, which creates a folder with tons of DNG images in it, right?
Then I went into that folder and attempted to fix one frame:

C:\temp\M20-0406>"j:\EXIFTOOl\exiftool(-k).exe" -BlackLevel=1750 M20-0406000000.
dng -o TEST.DNG
    1 image files created

The resulting image was the exact same. The output "TEST.DNG" file still has the pink hue over it.
I tried 1024 as well. Neither seem to have done anything to the image at all.

Now if the "fix" applies these numbers already and turning the "black fix" ON in Magic Lantern is
what caused these issues in the first place then maybe the numbers I need to be using for "BlackLevel"
should be the default ones that would be there before the option "black fix" is turned on?
Because that's how my stuff works. With it on, I get the pink hue. Off and all is great.
This footage is screwed because it was turned ON.


And how complicated is that header to MLV anyhow...
You can create MLVs with no WAVs in it.
I wonder if I create an MLV with no audio and then simply clip that header and append it to the beginning of a
RAW file if that is all that's needed to "convert" RAW to MLV? RAW doesn't have any header I don't think.
I don't think there are checksums that would need recalculated are there?
I didn't wanna hafta get into it at this level.

The basic premise is if I could "convert" RAW to MLV, I could then run that MLV through MLVDUMP with --black-fix option.

But until I get black fix option working right on the DNG method with exiftool I won't bother with what I'm pretty
sure I could do since I'm sure both mlvdump and exiftool are changing the exact same setting.

I don't understand why mlvdump doesn't have the reverse function of going from RAW to MLV. It should.

Is there a programmer's breakdown of the MLV header somewhere?
Like what each byte represents and the general layout? I'm sure it contains the size of the video data,
I would need to change that to represent the correct length.

[Audionut]

C:\temp\M20-0406>"j:\EXIFTOOl\exiftool(-k).exe" -BlackLevel=1750 M20-0406000000.
dng -o TEST.DNG
    1 image files created

The resulting image was the exact same. The output "TEST.DNG" file still has the pink hue over it.
I tried 1024 as well. Neither seem to have done anything to the image at all.

Check your workflow.  Changing the black level tag will unquestionably alter the image.

[DAK29]

C:\temp\M20-0406>"j:\EXIFTOOl\exiftool(-k).exe" -BlackLevel=9999 M20-0406000000.
dng -o 9999.DNG
    1 image files created

C:\temp\M20-0406>"j:\EXIFTOOl\exiftool(-k).exe" -BlackLevel=0 M20-0406000000.dng
-o 0000.DNG
    1 image files created

Neither setting the switch all the way to 9999 or all the way to 0 produces any change in the image.

0000.DNG and 9999.DNG look the same as the source file M20-0406000000.DNG.

Exiftool shouldn't complete and create the file if there's a syntax or other error so I don't get it.

Isn't there a function that alters this in one of the Adobe products? Couldn't you change this in Photoshop?

EDIT - It seems it changes the meta data and not all "viewers" display based on meta data.
So all 3 images are identical and they only have different meta data.
View them with a viewer that only shows what is there and does not listen to meta data and they are all the same.
If I load them into Photoshop, they are different. Need to do more research.
Want to figure how to just do this in Premiere.

[Audionut]

Mind uploading one of the problematic DNGs?

[DAK29]

Is there a place to upload to? A hassle free place where I don't need to fill an account out at?
Cuz I would just throw it on usenet if you know how to download that.

If I load it into Photoshop I can fix it by using the "blacks" option.

Can you alter "blacks" in Premiere? That would be great. Looking for options now.

[Audionut]

https://www.google.com/search?q=file+host

[DAK29]

Those aren't hassle free. Do you use usenet? I can put it there.

By the way it is the "blacks" function in Photoshop.

EDIT - Woo hoo! I can do it in After Effects!

I can import my RAW stuff in Premiere PRO, then import the Premiere project into After Effects.
THEN I can alter the levels I need and save back out!

THIS IS THE FIX!!!

The only thing I need now is to know the exact function I should use and exactly how many numbers I need to reduce it to.
Because right now, I am only doing it by eye.

P.S. - And here is the technical solution I was asking, it appears you can just fix it in hex -
http://www.magiclantern.fm/forum/index.php?topic=6580.msg52480#msg52480
It appears RAW files do have meta but it's stored at the EOF not the beginning! hahah

[Audionut]

You're kidding yourself.  TinyUpload allows you to select the file, hit upload, copy/paste the link.

Way less time then the walls of text above.