Messages

Whew .. crazy .. the script can´t really care about the content of the files BUUT - is the naming of the files much different? This could explain it! But very cool it works for you now!!

Did you also test it with another directory structure with other DNGs? Can you test the SAME files/dir structure on another AFX computer? Can you send a list of the files? in a DOS BOX, enter  DIR *.* /S >filenames.txt    which creates a textfile with the filestructure and the filenames, maybe I can find something.

Hm .. sometimes I have the script not creating the comps automatically and don´t know why yet, but THIS problem occuring so persistent is new to me.

Three more ideas to check:
- Which language version of AFX are you running? Something other than English?
- In EDIT / PREFERENCES / GENERAL, is "Allow Scripts to Write Files And Access Network" ticked on?
- Do you have full user rights on the folder you´re reading from AND outputting to?

Steffen

Yes, it´s working flawlessly with VisionLog, as VisionLog works "under the hood" in ACR and delivers you the flat "visionlogged" video sequence right in AFX, so the script just batches these, no matter what curve/settings you use in ACR.

@timbytheriver: Do you have any other files in the directory (besides WAV files) and ALL its subdirectories? Like a folder-preview-jpeg? Those can cause such an error. If there are only DNG file sequences and maybe a WAV it should be fine .. are all file names okay? No special characters?

@5DDaniel: Yes, you can easily create your own template version. For example, search for "framerate: undefined" in the script, it´s towards the end where the audioOffset is also defined. Here you can pre-set all your variables, "undefined" means the field is empty, but if you set it to a hardcoded value right here (just change "undefined" to your desired value) you have created your own template which you can still change on runtime if you need to.

Don´t we already know that the SD-card slot in the 5DMK3 tops out waaay before 100 MB/s as it´s not UDMA7 connected internally?

Hi, 5DDaniel!

Yes, absolutely, it does everything you´re asking for.

"Generate Compositions" is ticked automatically, so it will create a comp for every item you import. AND add it to the render queue with the output module(s) you selected on the bottom.
And you can set (comp AND footage) framerate, sizes and even any transform options (scaling etc.) for all image sequences within the comps. All pretty self-explanatory once you go through all selectable fields.
The Cinelog dialogue has nothing to do with that script, it is embedded in the ACR dialogue, if you set Cinelog as standard (explained in their well written doc file) then every DNG sequence automatically opens in CINELOG and you can basically hit okay for every shot and then check the comps later all after each other and make changes if necessary.

@pobox:
The audio offset is a value I came up with after testing MLV with sound on my 5DMK3. It might definitely be different for camera models or even for future builds, but I don´t have another camera to try. This is why I´d suggest you do a quick test with your camera (a clap with the hands), figure out the frame offset in AFX or an editing program and then edit the offset value in the script file.
To do this just open the script in Wordpad, search for "-0.08" and replace it with your new value, done in less than a minute!

Happy batch encoding!

We had this topic recently. Yes, MLV is very stable and reliable by now IF you have the right cards. No, I would not use ML on a paid shoot, only as a B-camera which can be left out of the game if it breaks, gives errors, problems etc. - but an awesome B-camera.

I am sometimes even using it as a A-cam on highend commercials, but only if the client is not watching! ;-)

Are your cards fast enough? Try to make your vertical resolution smaller (crop) and check if the errors still occur.

[a After Effects script which should make the lifes easier for everyone who converts ML raw via AFX]

Hi, community!

After reading along for a long time I can finally contribute with some software as well - a After Effects script which should make the lifes easier for everyone who converts ML raw via AFX that was written for my needs of converting MLV files to 10bit Quicktime files from the talented programmers in my company.

It starts with the "Smart Import" functionality from the original AFX script which goes through folders and imports file sequences, but adds some new interesting features.

It automatically syncs audio WAV files which are in the folders from a MLV file! It even considers the offset which is in the MLV files at the moment (might change in the future, that´s why it´s not hardcoded).

You can set the footage framerate for all the shots you want to import, you can set width and height of the comps (as you might´ve shot vertically cropped but need full-HD to transcode to ProRes or DNxHD), you can add several output modules for transcoding and set a folder for all the files. All you have to do is click OKAY for every ACR window that pops up and in the end you have to hit RENDER in the render queue - that´s it!
I use it together with CineLOG (http://www.magiclantern.fm/forum/index.php?topic=10151.0) to create flat LOG DNxHD 10bit quicktime files for further editing and grading.

Download it here:
www.alexandsteffen.com/smartimport2.zip

Just place the JSX file in your AFX/Scripts directory, go on FILE / SCRIPTS and execute it!

Happy encoding!

Steffen

At first they always seem to help quickly .. had a card that was unusable from one day to the other, their support helped a lot, sending a check tool etc., then when nothing worked I was told to send it in with an RMA number, exchange should come within one working week. That was two months ago and nothing ever happened.

Cool! Remember the game! Awesome music for that time!  Very nice, well done, especially bringing some of the known old styles (lasers, enemies) to the project!

Err .. no .. the other way around. You want to comp in rec709 color space and convert back to log in the end. So your log footage that needs to be mixed with other material needs to come from log->709 and then in the end the finals go 709->-log.
If the values are set right (no clamping) and if you´re working in 32bit there should be no visual loss to your footage.

With the Cineon Converter you can go "backwards" as well, from rec709 to log (there´s the linear to log preset for it), so you can mix your footage (apply LUTs or the cineon converter effect), comp it and output it back to log in the end. Don´t know how BMD film log behaves (normally I´m using Alexa LOG C material), but you can create your own preset for it by putting the same shots side by side and using the cineon filter forward (log to linear) and immediately backwards (back to log) after another and then tweak the values until it loos very similar to the unaltered file. Will be hard to match it exactly, but you can get close.

Well, easy .. then you just output the vfx shots from Resolve as log 10bit DPX files, so you have the same basis as for color correction. And on top of the Premiere timeline you can add a adjustment layer with a LUT to see "proper images" when editing.

If it´s too much of a hassle outputting shots then think about CINELOG (http://magiclantern.fm/forum/index.php?topic=10151.175;topicseen)  - I am mixing BMDlog and Cinelog in my current project and with the right color correction/LUTs it matches perfectly.

Steffen

Well .. studios (big ones) would go with a workflow that involves linear EXR´s and/or a log DPX output .. but that´s a little bit too much for the beginning.

Try it like this:

1. Create proxies in resolve in case you don´t want to use ACR in After Effects for the same task.
2. Edit Proxies in Premiere.
3. Replace VFX shots with AFX comps in the timeline to create your VFX. Replace the footage in AFX with the DNG sequences (this is where you did the color correction first, but it should not be done before.
In case you can´t work on the colors as they are .. use LUTs on GUIDE Adjustment Layers that don´t render out.

3. Export finished film and EDL from Premiere (in case you´re using resolve and not Speedgrade which does not need an EDL).
4. Grade the film, export as uncompressed file sequence and ..
5. Compress the sequence to your end format with Media Encoder.

cheers,
Steffen

So far I don´t see an advantage in formatting the card in camera. Especially on a bigger shoot day with several people involved the DP or camera operator should not think about card formatting etc. - he gives a full card to anyone dealing with the data/copy process and gets a fresh exFat formatted card back once it´s properly backupped. Everything else - in my opinion - can just cause problems.
And never problems with a card that came formatted from the computer.
The SD card with ML on it stays untouched anyway. Of course this only applies to the 5DMK3 with its dual slot.

Regarding the question of the highspeed cards - I don´t own them so far, but as we´ve heard from numerous sources in this forum, after around 100 MB/s the camera´s internals seem to become a bottleneck, so you can´t expect it scales endlessly with the speed of the cards. We´ll see, hopefully it works with future faster cards, but I doubt it.

Glad you like it! Learned so much in this community that it was about time to give something back!

Oookay, let´s start.   Can´t answer everything, though.

- The 1.2.3 is definitely not as stable yet as the 1.2.3. Problem is that most people did not shoot hours of footage on 1.2.3 firmware yet, on the 1.1.3 with latest ML builds they did. On my tests 1.2.3 worked fine, but I did not use it yet on set and I won´t very soon as I´m just about to be satisfied with the MLV implementation´s "buggy-ness" of MLV on 1.1.3 which you can fully use - but RAW is still more reliable. Get your cards, your build, your camera - and shoot tons and tons of footage. Long takes, short takes, everything inbetween, that´s the only way to find out if 1.2.3 works for you.

- On 1.1.3 there has been a time where new builds introduced new quirks, but at least for my testing abilities this has long been gone. Feels safe to always use the latest. But never go into a shoot with a untested build. Before my last week of shooting a feature on ML raw/mlv I did not update the build for two weeks prior to shooting and was only testing the stability of the actual one.

- mlv vs. raw - if you don´t need metadata and audio it´s still faster and more safe to use raw. On Komputerbay cards (which not always give you the full speed they are advertised for) it´s especially tricky as video+audio recording needs a little bit more overhead. On mlv you could theoretically record to SD simultaneously, but I did not find this feature useful in the field for recording.
If you need audio - for syncing with an external source for example - then go with mlv, but all the time if I just need the raw videodata for a project, I still switch to raw.

- You can realistically record the full 12 minutes until the card is full - on a 64GB card. I do not own 128 GB ones. Shot a lot of 10 to 12 minute takes WITH sound and had only two corrupt files in a week of shooting.

- Yep, exFat is the way to go. Everything else (merging the spawned files) works, too, but you´ll need that extra step.

- No LUT. You can´t apply a LUT to raw data, it just does not work with the concept. The camera´s hardware is not fast enough to debayer the images AND apply a LUT. It can´t even debayer in realtime. That´s the concept of ml raw - you push all the processor heavy stuff into post, on the computer side. Feel free to add any LUTs in DaVinci or ACR.

- Yes, you can playback your shots in camera. And even delete them. Not realtime, though, but now even in color.

- On 1.1.3 you might not want to use an external monitor for focusing etc. as the resolution of the HDMI stream is too low. 1.2.3 fixed that but I read a lot of problems that the image on the screen was shifted, don´t know if this is fixed. Anyone?

- As for the HDMI splitter .. never used it. Believe me - you don´t want to use ML with a client involved. If we talk about a real client. I´m a commercial director and I LOVE having my 5DMK3 with ML raw with me on set, but I would never ever use it as the 1st camera. Not because of the quality, but for safety issues. Unless you shoot only 5 minutes of footage a day you WILL definitely have stopped takes at one point, some errors here and there and it will need some sort of acting to show to the client that this is normal and you´re just swapping cards.

ML makes a 5D the perfect stealth highend second camera which you can easily intercut Alexa oder RED (or blackmagic or whatever) material to .. but it´s not an A-camera for a paid job where the client watches every of your step and you might have to explain delays. Might be all fixed one day, but it´s still not as easy as pushing a button and stopping and playing back. What if the client wants to review a take? How do you tell him it´s not 25fps he´s seeing on the monitor?

If there´s clients involved, get a A camera and equip someone with your ML camera of choice to get closeups, additional footage, whatever, and you will have a much happier shoot than with the fear in your neck that your camera might stop in the middle of the take.

Then your workflow .. this looks .. well .. strange. First of all - don´t shoot so heavily cropped (2.4). What if you want to stabilize? Or reframe? Not saying you should shoot 16:9, but think about 2.0 at least.
For my taste you´re doing way too many steps which might be fine for a small project, but that´s definitely not the smartest way. Why not going directly into Resolve with the DNGs? You can apply your log curve (CineLOG, see the thread in the forum here) there as well. From there you can go to ProRes files for editing and you saved the whole Lightroom thingie.

I do it differently .. I don´t use Resolve but ACR in After Effects to create quicktimes (DNxHD, on MacOS I´d use ProRes) and apply the log curve at the same time. Have a script for importing, render times are long but done overnight. Then I edit the quicktimes with a LUT/fast color corrector on top and use whatever I want for finishing - Speedgrade for example.

I would consider the TIFF files you wanted to make an unecessary step. You loose the benefit of the raw file the minute you´re converting the DNG to a TIFF. You´d need to convert it to linear EXRs to hold all information, but there´s no way editing with that. That´s why 10bit LOG video files have been invented, to store the full data like in the 10bit cineon files. If you apply a log curve to your DNGs and export 10bit ProRes HQ or 444, you basically still  have all information - you just need grading to bring it back to a "viewable" form.

cheers,
Steffen

Yes, that´s normal that the video file is not fit exactly. on my build audio starts 2 frames into the video for correct sync.

We once used Lagarith and huffyuv but in 64bit environments they produced problems when we switches to Win7 x64. Before using those "older unmaintained" codecs make sure they work in your application even in native 64bit environments, otherwise it´s painful to convert.

I´d go with DnxHD on Win or ProRes on a Mac, but of course there are some artifacts in comparison to Lagarith.

For the histrogram: Ist Global Draw turned ON for recording?

Yes, the RAW is stable now. 99% I´d say. I did shoot hours of test footage and did not have a problem. MLV with audio is 95% stable .. shot one week of footage, several hours a day and had a few errors, but only lost one complete take, the rest was just stopped takes which the camera tells you. Do your own tests. Do you need sound? Then MLV is worth using it. No sound needed? You´ll love RAW!

Steffen

You can try the HDMI-image for focusing right away - it´s the same quality without magic lantern. The HDMI output of the 5DMK3 with 1.1.3 firmware was never Full-HD, so you can judge even without ML or MLraw running what the quality will be like and if it is enough for your focusing needs - for me the onboard monitor was the better choice.

But if the new firmware works fine for some - great. My personal choice was rather take some quirks into account (like broken files that are completely unreadable, but I would restart the take anyway once I get a strange error which I got several of in a shooting day) and have sound to get PluralEyes working, because at our speed of shooting a slate would not have worked.

Definitely don´t use one of that ipad apps .. I love smarthphone/ipad stuff on set .. but they are unusable. Either too small for the distance or not loud enough for sound ... not working in the real world.

Hi, Michael!

Just shot half of a feature a few weeks ago on ML - 95% of the time it worked fine.

1. Komputerbay cards are definitely not the best and accident-free choice, but if you are on a budget (like I was) the only choice. From time to time they would drop frames or suddenly don´t start properly recording .. so we just switched cards (had five of them) and dumped and formatted the strange behaving one, most of the time it was fine afterwards again.
2. Shoot MLV with sound, so you can sync your external sound much easier with PluralEyes for example.
3. If the write speed is not enough (it is not on SOME of my KB cards) I cropped the image vertically slightly (1:85 to 1) as I am going for cinemascope anyway. Full 16:9 sometimes did not work with the audio recording on.
4. Make backups on set and have someone quickly looking into the MLV files randomly. It´s an overhead person, but in case a file gets corrupt you can reshoot. We had only ONE single file getting corrupt in a week´s shoot, and this was a very long take of about 10 minutes. Don´t know what happened, but generally everything was pretty reliable with the (back then) latest version of ML and of course the 1.1.3 firmware.

Now to your questions:
1. Test the card in camera. ML has a card test feature, it will give you the write speeds to see if it delivers.
2. yes!
3. I detached my monitor as it was behaving strangely. HDMI out definitely works, even on 1.1.3 firmware, but the image will be SD (bad for focusing!), so we rather went on using a hood loupe for the MK3s screen.
4. Dump the raw files and look into them with MLVrawviewer quickly, don´t do conversion on set, not needed at this stage. You just need to know if the file plays back fine, everything else can be done later without small external harddrives and a bigger RAID etc.

cheers and good luck!
Steffen

But how did you manage to get exactly the same framing from the H.264 and the crop mode? You would need to use another lense to correct for the crop mode .. much much wider. From the picture I´d assume that this is not crop mode .. and this is why you see binning artefacts.

blakekimmel, I had the same errors in the image. Very latest built (three days ago), and all the files were corrupt like this.