Uncompressed 14-bit RAW video testing - 5D Mark III

[Phillight]

Hey there!

I've been playing for a while now with the RAW stuff and it amazes me. That being said, I wanted to "simulate" a real project under real conditions. I'm gonna make it short, what problems I ran into:

i) Some of the clips I took were recorded in 29.976 fps, even though I specifically set the fps override to 23.976. I shot about 30 clips, 4 are in 29.976 fps. That's weird, what could it be??

ii) If I took shots within the same minute, ML seems to keep both files, but overwrites them, so that only the last one taken can be decoded. It's in that format: M16-1510. The other folder has  the exact same name and the file it contains seems to be an exact copy of the other file, named M16-1510 as well. So one of the two will always be missing. What can I do?

I shot everything in 1920x1080, 23.976 fps, I used a Komputerbay 64 GB card as well as the latest ML version. Any help in solving these issues will be highly appreciated!

Phil

[Midphase]

Hello Phil,

I realize you're new here and I would highly recommend that you consider spending some time not only just messing around with your camera and learning by doing, but also reading as much as you can of these forums. While they might seem like a real time suck, it will help you by providing you with a lot of insight and ideas so that you can self-troubleshoot your issues.

I would also advise that you include some critical information along with any questions, such as your firmware revision, the nightly build that you're using, and whether or not you're shooting crop or non-crop, and .raw or .mlv.

I can answer question #2, and I don't know if it's a ML lantern but the way Canon names the files is based on the day and time, so for instance M16-0950 would be the name of any file shot right now. Unfortunately I don't know if ML has control over the name of the file, and the way to avoid such issue might be to make sure not to record two takes within the same minute.

[Audionut]

The filename thing is ML.

Code: [Select]

snprintf(videoname, sizeof(videoname), "M%02d-%02d%02d", now.tm_mday, now.tm_hour, COERCE(now.tm_min + number, 0, 99));
When you shoot crop mode, the framerate switches to 30fps.  You need to use FPS override. 

Midphase has good advise.

[a1ex]

See also...

It should never happen. Find a way to reproduce and create a proper bug report.

[Phillight]

Hello there and thank you for your replies, also thank you for developping ML, I don't wish to come across as someone who doesn't appreciate the effort others make in order to realize such a project. However, I did read quite a bit and I have so far shot only in "unreal" conditions. It means I did some testshots, played a little with this and that, liked it and decided that I now want to explore a proper workflow for "real" projects. So I jumped on YouTube and looked for a tutorial. I found one and liked it, so yesterday was the day for me where I started a little project in order to finish it from A to Z. The problems I came across I managed to solve some of them myself, some of them I couldn't yet fix and some of them are above me; about the latter I was hoping to gain some insight by asking about them. The problem with the green cast I managed to solve by fixing the BlackLevel. About the other problems I don't know where to go anymore, even though I tried to find threads in the forum about similar problems, I did find one concearning the fps override issue, but it wasn't (at least to my knowledge) solved.

So when I tried to reproduce the issue with the fps override, I didn't succeed. I had several guessings, though. I will list them:

ii) ML lets you choose between writing files into one directory or just in the DCIM folder. In choosing the option with the seperate directory, ML creates a folder that is named Mdd-hhmm.MLV. If you were to record two or more clips within the same minute, ML puts two (or more) folders on the CF-Card, all named the same. So in windows the path leads to only one file, even though all the files are there. My workaround is that I rename the files that came double myself, these aren't a lot usually. It doesn't happen too often that you record several clips within the same minute. A batch renamer would do the job as well I think. What also works is cut the folders on the CF-Card and paste it one after another, because (that's my amature-guessing) windows will clear one file and frees the way to the next file that was under the same path. Like M16-1234 exists 4 times on the CF-Card, cutting and pasting will clear the first one, then the next one and so forth. When prompted you just choose Copy, but keep both files and everything is there.

So I figured out issue ii). I don't know, as I said, much about programming and stuff, but I counted the characters that ML uses to name the files and folders, and they are 8 Characters long. That number is the same that canon uses. So I'm guessing that it is obligatory to name files 8 characters long? Would it be possible to leave the hyphen aside and replace it by upcounting number, so that the filenaming would be like:
M1611234
M1621234
M1631234
M1641234
M1651234
M1661237
...
M1791532
M1701532

So that the number counts from 1 to 0 no matter in what difference of timeintervalls clips are recorded, but ensuring that clips within the same minute will never be the same. And 11 clips in one minute is something not to consider.

I thought the option take in the mlv menu would do smth like that but it doesn't.

So far for the 'Recording several clips within the same minute with the option directory enabled'.

If the option create several directories is not enabled, the files count up and in that case the indicated time is off.

i) For the problem with the fps override I couldn't find a fix. In my previous post I literally spelled it out that I did set the fps override specifically on 23.976 in order that the regular recording and the crop recording match.
I couldn't find a way to reproduce this, I tried several things, though:

I tried recording with mlv_sound enabled, with it turned off. I tried it with a lot of different ISO settings, Aperture-values. I never changed the exposure time, I left that at 1/50th of a second.
It seems to me that it occurs arbitrarily with no recognizable cause. It happened so far on both of my CF-Cards, with all lenses I used.

I have tried all morning since 9am to reproduce that error that occured yesterday a couple of times, today everything seems fine.

Here's exactly what my setting was:

Canon 5D Mark III
Firmware: 123
ML Version: Date: 2014-07-04 00:01:21 +0200
Canon Sound: Disabled
(yet dim-wittedly on my side) MLV_Sound turned on.

This moring when I tried reproducing the tha fps override error I turned Canon sound on, I then, after turned Canon sound on I executed my testing with MLV_Sound on and off.

The error never occured again so far, so could it really be that this whole problem is due to the fact that I had Canon Sound disabled? I was recording the audio with my external recorder so I didn't need it, I didn't even need a synchsignal, that's why I didn't bother at all.

In the meantime I have insalled the latest version as of yesterday and tested a couple of recordings, in everyone of them the fps override did its job, Crop, Non-Crop. All is well so far. I will later in the day redo the testing with Canon sound disabled again, so if the error occurs again, I think I have found a reproducable cause for the default.

If you want me to provide you with more information, I will be happy to do so.

Thanks again for developping the ML Hack, that is really awesome of you guys!

Phil

[NedB]

@Phillight: I think the reason you have not been able to reproduce the error is that it can't/didn't happen. I'm pretty sure that a1ex and the other devs thought about this problem when they developed .raw and .mlv. As far as I know, ML uses the current time to name the clip unless that name has already been used, in which case it is incremented by one (unless that name has also been used, etc.). This is a quick fix and I guess a bit of a hack, but given that they only had the 8 characters, there wasn't much choice. So the "time stamps" for .mlv (and .raw, I think) clips are more-or-less accurate, except for this caveat that they may be off by a few if you shoot multiple shots beginning in the same "wall-clock" minute. I think this is what a1ex meant by "it should never happen". No offense meant, hopefully none is taken. I just don't believe there were any "hidden files" with the same name, I don't think ML allows it. Not sure to make of what happened (or seem to have happened) in your case... But, cheers!

Edit: Good effort at debugging and trying to find the source of the "problem", but I think the chances are very small that this file-renaming issue (if it really exists) had anything at all to do with whether or not you had Canon sound disabled.

[Phillight]

Hello NedB, just to quickly clarify, Canon sound disabled and Filenaming were not the two issues I linked/connected. I connected the fps override issue with the Canon Sound issue. I will now post a couple of pictures by which I try to illustrate the aforementioned:

I uploaded a series of pictures to Google. Please read the captions and the appended notes to them. I hope I could describe the issue I have here as thorough as possible.

Thank you very much for taking interest!

https://plus.google.com/u/0/photos/103593325688024422825/albums/6037026448453372961

[a1ex]

Confirmed the bug and solved it. The keyword was the separate directory option, which I've never tried, and the first screenshot made it obvious.

(that's why I still use raw_rec if I ever record anything - much less stuff that can go wrong )

[Phillight]

Does that mean we can be sure that this also causes the fps override issue? And is that a "real" bug or are the fps just wrongly written into the meta data, so that rawcdng or DaVinci get a faulty reading? Whatever it is I can do to help, I will do it as long as I am able

[a1ex]

Find a way to reproduce it first. Without that, I have no idea what it might be.

[Phillight]

Ok. I tried. I did try to find a way to reproduce the 29 fps issue. I did not succeed. I will post the image right here, so you can see for yourself.



I've been recording in every way imaginable to trace down what causes the fps to switch from my desired 23.976 to 29.976. I don't know what else to do. Everytime when I thought I know now what it is, it was not possible to reproduce it, and when I thought the settings I have now are fine, it happened again. Until I gave up, that is, a couple of minutes ago. Arbitrarily it happens... at least in my humble opinion. Its not the time its not the Preview mode, its not the lens, its not the ISO, its not the audio, its not the intervall between shots, its not the preallocation of memory, its not the extra hacks...

How can I provide you with something you smarter guys than me can work with?? Am I really the only one having that problem??

BTW, this afternoon I switched to the newly released version of ML, the one from yesterday.

From 1817 to 1858 I was changing settings like an idiot, I did everything, but obviously not a single clip came out 29 fps... that is very weird in my opinion...

SHORT EDIT: I manually changed (using Phil Harvey's ExifTool) the Framerate with that string: -IFD0:FrameRate=23.976 on all the clips that had different framerates. I couldn't synch the audio, though. But then again, I had Canon audio switched off and tried it by simply lining it up by timecode. Which I thought it should work if the time in camera and the audio recorder's time were synched.
I hope I'm not getting off topic here. Will continue to look for the cause right now.

Phil

[dd2020]

Can I ask a question here?

I have been using ML on 5d2 for a while and have problem when recording in crop mode... the shots always come out not quite correct position (the subject always be on the right instead of being center)as seen in the camera's monitor  But anyway it seem a little bit ... not too much.

But now... when I started shoot on 5d3 with ML latest Nightly build.... it seem the problem still occur..
when I shot in crop mode .... after processing the subject will go to the right ...(when seen in camera's monitor , I made the subject at the center)and this time the position are much different.

 I don't know how to fix this...    I had checked whether the zoom rectangle is already positioned in the center.

Thank you..

[Phillight]

If you enter the submenu of RAW recording module, you'll find your answer to the problem under 'Preview'. If you use the ML Greyscale preview, it will literally tell you that even though it looks ugly, at least the framing is correct.

RAW -> Preview -> ML Preview.

[dd2020]

If you enter the submenu of RAW recording module, you'll find your answer to the problem under 'Preview'. If you use the ML Greyscale preview, it will literally tell you that even though it looks ugly, at least the framing is correct.

RAW -> Preview -> ML Preview.

Thank you for your answer... but.. it still doesn't work.
Anyhow I have figured out how to fix this by going into menu Prefs> Focus box settings > Snap points > Center to x5 RAW

but I still have some question in mind.. may be ask later... thank you again...

[chmee]

@Phillight
please do a look on your files with mlvdiag. maybe theres a bug in my converter? do a screenshot of the MLVI-block. as seen in the picture, the last two values on the right side are fps as fraction (23976/1000).

regards chmee

[itsDPmikey]

Hey friends,

I'm hoping you good people would be kind enough to help me with an issue i'm having with my 5D MARK iii.

Ever since I got my camera I've been getting these terrible flickering on all my shots(after installing magic lantern) And I cannot figure out why. I first downgraded to 1.1.3, had flickering, then upgraded to 1.2.3 and still flickering occurs in my videos. PLEASE HELP, PPLLEEAASSEE!!

Here is a footage of what i'm talking about

Modify message




By the way, In my ML settings under video TAB, I have both RAW VIDEO(MLV) and RAW Video on. Should I only have one of them on? I know one is with and without audio. But does it make a difference if both are set to ON?

[Midphase]

Yeah, you shouldn't have both MLV and RAW going at the same time. That might fix your issue right there. If that doesn't do it, I would probably blame it on too much CPU stuff going on at the same time, try turning off Global Draw.

[itsDPmikey]

Yeah, you shouldn't have both MLV and RAW going at the same time. That might fix your issue right there. If that doesn't do it, I would probably blame it on too much CPU stuff going on at the same time, try turning off Global Draw.

THANK YOU FOR RESPONDING MIDPHASE

I thank you very much for that information sir. I will turn off one of them and record video on both of them(one at a time) and see which one works best. By the way If I turn off global draw, won't I lose focus peaking and those cool features that are useful in video? I'd hate to lose that because I just got my Tilta follow focus+carbonfiber mattebox  and I love love love using it, well I better for 1200 bucks.

[Midphase]

Yeah, you lose peaking and zebras etc. Experiment and see what works. There is also the Canon native peaking function which doesn't add any strain on the CPU.

Experiment, worry about getting solid footage first, and the other bells and whistles last. And finally, understand that you're working with a hacked solution, what that means is that there are quirks and other workarounds needed. If you are looking for a no-compromise way to shoot in raw, I advise that you look into other camera solutions.

[Phillight]

@chmee, that wouldn't be of no use if I did what you suggested, because there (very, very oddly) is no further use for troubleshooting the problem since it no longer exists. I haven't then known what was causing the issue, I don't know now and I probably shall never know. All I have observed since is that it never occurred again like Harry Potters scar just never hurt again. (sorry for the stupid methaphor.)

I upgrated to the latest version of your awesome converter and to the latest verision of ML, so if there was a bug in any of the previous versions, I'm pretty sure that it was fixed.

As for now, I don't see a point in downgrading both versions and trying again to find it. Do you? If so I can do it as soon as I have the time.

Have you heard of anybody else encountering that particular issue with the latest versions of MLV RAW and cdng converter?

[Phillight]

itsDPmikey, I would like to ask you if you are using DaVinci resolve to render your footage, because if you do I just might know what's causing that flickering. I once had a "beta-testing" (if you know what I mean) version of DaVinci Resolveinstalled , (mind I didn't write DaVinci Resolve lite) and that was the exact same flickering I got every time I rendered something, especially when DaVinci's NR was applied. And your footage does look like it was treated with a faire amount of NR in DaVinci. I might be wrong, but that flickering bar at the top is very characteristic.

Anyway, try the lite version if that is in fact the case.

[anandkamal]

hi DPmikey, I once had the same problem. The possible reason could be that Resolve color corrects certain frames and certain remain. This causes the flash (corrected and original flashing). Click "ouput field as single source" in General Option under Master Setting. This would work as it worked for me for a while. I got the problem again. It is definitely to do with frame rate per second settings. Now, after I start using Resolve LITE, I face NO problem. It is all fine.... Hope that helps.

this is a wonderful thread for 5d3 guys, I hope there will be more conversation in here.

[Satis]

hi DPmikey, I once had the same problem. The possible reason could be that Resolve color corrects certain frames and certain remain. This causes the flash (corrected and original flashing). Click "ouput field as single source" in General Option under Master Setting. This would work as it worked for me for a while. I got the problem again. It is definitely to do with frame rate per second settings. Now, after I start using Resolve LITE, I face NO problem. It is all fine.... Hope that helps.

this is a wonderful thread for 5d3 guys, I hope there will be more conversation in here.

I used to have the same problem with Resolve 10 - regardless of what frame rates I rendered and etc - , ever since I downloaded 11 the problem has been fixed.

[danielson]

Last build from:  2014-08-07

Hi

I've a issue with the last file on the card:
the film is perfect, but the last 10 sec of audio is missing.

I've set "reserve card space" to "on". The camera writes the file before recording the first file, but with the last file, the "mlv.rec.tmp" is deleted or overwritten.
The last mlv-file is also distributed in to a lot of small files (around 9 files like "M13-1317.M08" with 52Byte and the "M13-1317.MLV" (the file with the missing audio on the last 10sec)).

I tried different settings with the same result.

Any idea to prevent this?
Thanks.

[STEVENPIX]

Coming from 113 with my 5DMK3 and moved to Nightly built 123 but as before when I go to MODULES Menu I press SET to activate the raw_rec then

Only have the message ON, will load
and This module will be loaded at next reboot

What can I do to active RAW Module and access to Submenu ?

thxxx