50D Raw video

[1%]

heh, I tested and I can't get 2000x1080 or 1920x1080.. the speed is still jumpy.

[Andy600]

I deleted configs, reformatted etc. Speed is holding up (buffer method 3) and DNGs playing nice with ACR

Needs the rec display thing added because it's still old style buffer bar etc.

I'm liking this

[1%]

Yea, I went back to 3 as well... maybe need to clean the card up, I just popped the new one on and started recording. 7D is holding up tho.

[jamesd256]

Just checked the latest nightly with supposedly mlv_rec.  Alas, module does not show up once camera boots.  Also, ML still doesn't have the raw capabilities that Andy600's TL build has on my 50D

Sorry if this is a boring question, but are the latest features (AETTR seems missing in TL) coming over to TL as they happen?  What are the enhancements that allow such high raw speeds on 50d with TL, and will they ever be pulled back into ML?

Thanks for all the effort to all those who contributed.

[1%]

The nightly + my builds should have raw_rec + ettr. Nightly doesn't have newer edmac channels (the old ones still gave ~80MB/s) and maybe some raw_rec/mlv_rec features

[jamesd256]

Raw_rec is in the nightlies and aettr.  I don't see mlv_rec in the nightlies, even though it's in the changelog.

I don't know what the edmac channels are, but I get ~75MB/s in Andy600's 50D build, but only ~64MB/s in the latest nightly.

No big deal, happy with what I've got, but looking forward to having aettr with dual_iso support + the speed I'm getting now.  Right now, it's one or the other. 

Playing with mvl_rec would be a bonus.  I'll wait patiently.

[1%]

Heh, well andy's builds should have all of those... no video DUAL for 50D/7D/5DII they only use one register period.

Hehe, so not to bypass MLV any but:
http://u88.n24.queensu.ca/exiftool/forum/index.php/topic,2750.0.html

download exif tool gui.. look at cr2, save camera model.. .ie "Canon EOS 7D" or 50D, etc. Select all DNGs... watch it added in about 20 seconds.

Also don't forget uniquecameramodel... its the same as camera model

[Andy600]

I'm still testing mlv_rec (all resolutions, speeds etc) and mlv_play but will upload a new alpha build later with everything (that is stable) included.

[Andy600]

@1% - that looks useful for legacy dng files. I guess we now have an option for pre-filled meta but slightly slower recording (mlv_rec) or faster original raw plus a little exif trickery before post (raw_rec). Nice to have choices 

[1%]

Yea, I dunno whats up with it on 50D... its like 60/70/80/83, 30 and then predicts 2k frames but stops at 1/2. I've diffed it and I've only taken out the 2nd writing thread so it is like it is. On 7D its slightly less jumpy but I got a random stop in 1x in between recording 2 videos where it showed continuous. Haven't tried method 3 yet on 7D so probably it will be almost the same as raw_rec there unless you're recording crop. 50D you can use it if meta data > extra res.

[jamesd256]

Ok cool, well thanks for the FB.  Ain't life a bitch.  My damn 550D does dual iso raw video, but only at postage stamp sizes.  Sounds like a final answer you gave on the 50D, but then again, I've heard final answers before around here

[1%]

Its pretty final, 7D has *some* tiny hope but I've tried setting the reg on 50D manually and nothing happened.

[jamesd256]

Ah well, seems fair really.  Shooting raw on a £250 camera is already a little unfair.  Now, if I sold ALL my gear, plus my road bike and my guitar, could I afford a 5d III? ...hmmm 

Austerity sucks.

[Andy600]

Ok, I've uploaded a new alpha build of Tragic Lantern for the 50D.

IMPORTANT DISCLAIMER: Please remember this is an alpha build of a fork of Magic Lantern SPECIFICALLY FOR THE 50D and is NOT directly supported by other devs. The plan is to eventually add thoroughly tested and permissible code from Tragic Lantern to the Unified build but as it stands it is purely for testers who want a little bit extra and who understand the added risks. Please do not hassle other ML devs if you experience any problems with this or any other build I upload. Post your findings/problems/bugs in this thread only!

This build includes:-

Tested and working mlv_rec module. The corrupt frame issue seems resolved so now you can experiment with recording mlv raw video which has embedded exif/metadata. This should make using mlv recorded DNGs play better with ACR and allow the use of correct DCP (Digital Camera Profiles) in your editing app.

Use the mlv_rec module in the same way as the raw_rec module but do not have both modules loaded simultaneously or you'll be in a world of pain.

mlv_rec has slightly different configuration settings than raw_rec and does not yet have the nice selectable buffer gui of raw_rec. You also have a choice of buffer methods in this still 'experimental' module. I have found methods 3 & 4 work best.

To convert mlv raw files you need to use mlv_dump (A command line only tool included in the zip) or you can use mlv2dng to drag and drop convert, however, the latter app seems to no longer be supported. You can also playback on your desktop (PC) using g3gg0's mlv player.

For more info on how to convert and playback mlv files please refer to the following threads:

http://www.magiclantern.fm/forum/index.php?topic=7122.0 - g3gg0's thread with info and instructions for converting and playing mlv files

http://www.magiclantern.fm/forum/index.php?topic=7802.0 mlv2dng.exe - old thread


MLV Play module

The MLV player module (integrated as part of the file manager module - both mlv_play and file_man modules must be loaded!) has been supercharged. You can now play back any raw or mlv file in color, though not in realtime. There is a transport bar accessible with your Picture Style button (under the LCD). You can also now playback a raw or mlv file immediately after recording by pressing the play button (also under the LCD). When play is activated you can skip through your raw files using the top wheel. It will build indexes for mlv raw files before playing but this is very fast.


As usual, there are various other little tweaks and fixes but you may have noticed correct shutter readout is broken.  I'm not sure if/when this will be fixed.

Download the latest alpha build (including the latest raw2dng.exe and mlv_dump.exe desktop converters) here: https://bitbucket.org/andy600/tl50d/downloads/Tragic_Lantern_50D109-Andy600.2013Nov11_mlv_rec_and_player.zip 50D ONLY


UPDATE: mlv_rec may still have issues. I've recorded a lot of footage without a problem but I also had a 3gb file that would not dump to dng with mvl_dump and would also crash the player. It did however convert to legacy raw but there seems to be an issue with the headers/footers as debayering is not correct. Needs investigating further so I recommend only using mlv_rec for experimenting ATM

[jamesd256]

Wow, awesome, thanks Andy600.  I won't have time to play with this now for a couple of days, but will be sure to give some feedback.

[pulsar124]

Great work, thanks! I'll be one of alpha testers for this MLV stuff.

A question: does each frame include things like aperture stop and focal length? This is critical, as that's what lens correction module of ACR needs to get read of those annoying aberrations, distortions and vignetting. (Also, having the lens model would be very helpful; but if not, one could always add it later with exiftools.)

I am trying to come up with a "photographer-friendly" workflow for 50D RAW video PP. Specifically, I want it to be built around Adobe Camera Raw (packaged with PS). It actually works very well, except for one deficiency: my version of ACR (6.x) doesn't provide the upscaling option to get from uncropped video to 1920x1080. (It does provide a few sizes, but no way to customize that - weird!). If not for that, one could have a complete solution in ACR - it'd do all the grading, lens and perspective  corrections, then upscale to 1080p and sharpen the frames. No such luck - the last two steps (upscaling and sharpening) would have to be done elsewhere.

Virtualdub is probably the only video tool I know well (have been using it for ages), so I was trying to use it yesterday to supplement ACR. So the workflow is:

• Convert RAW to DNGs
• Load DNGs into ACR, do the corrections, but set the sharpening to 0 - I think this is important, as any sharpening (even the default one) has to be reserved until the end, after upscaling - am I right here?
• Save the frames in ACR as JPEGs - this seems to be the only way to directly bring them to Virtualdub (it doesn't know TIFFs).
• Open the JPEGs in Virtualdub, load two third party filters - Smart resize (I use exact bicubic resize to 1920x1080) and MSU Smart sharpen (I am debating between "Small overall" and "Overall" levels of sharpening), and then save this as an AVI video with the codec of your choice (I use Xvid for now)

At the end, I do see moire/aliasing artifacts, but they seem to be no worse than in the original size video. No flickering AFAIKT. And apparently no new artifacts due to upscaling - e.g. a high contrast diagonal line doesn't appear jagged. I think that's very crucial.

Does this look reasonable?

[jamesd256]

Great work, thanks! I'll be one of alpha testers for this MLV stuff.

A question: does each frame include things like aperture stop and focal length? This is critical, as that's what lens correction module of ACR needs to get read of those annoying aberrations, distortions and vignetting. (Also, having the lens model would be very helpful; but if not, one could always add it later with exiftools.)

I am trying to come up with a "photographer-friendly" workflow for 50D RAW video PP. Specifically, I want it to be built around Adobe Camera Raw (packaged with PS). It actually works very well, except for one deficiency: my version of ACR (6.x) doesn't provide the upscaling option to get from uncropped video to 1920x1080. (It does provide a few sizes, but no way to customize that - weird!). If not for that, one could have a complete solution in ACR - it'd do all the grading, lens and perspective  corrections, then upscale to 1080p and sharpen the frames. No such luck - the last two steps (upscaling and sharpening) would have to be done elsewhere.

Virtualdub is probably the only video tool I know well (have been using it for ages), so I was trying to use it yesterday to supplement ACR. So the workflow is:

• Convert RAW to DNGs
• Load DNGs into ACR, do the corrections, but set the sharpening to 0 - I think this is important, as any sharpening (even the default one) has to be reserved until the end, after upscaling - am I right here?
• Save the frames in ACR as JPEGs - this seems to be the only way to directly bring them to Virtualdub (it doesn't know TIFFs).
• Open the JPEGs in Virtualdub, load two third party filters - Smart resize (I use exact bicubic resize to 1920x1080) and MSU Smart sharpen (I am debating between "Small overall" and "Overall" levels of sharpening), and then save this as an AVI video with the codec of your choice (I use Xvid for now)

At the end, I do see moire/aliasing artifacts, but they seem to be no worse than in the original size video. No flickering AFAIKT. And apparently no new artifacts due to upscaling - e.g. a high contrast diagonal line doesn't appear jagged. I think that's very crucial.

Does this look reasonable?

You'd be better off exporting PNGs than Jpegs.  VirtualDub will use those too, and obviously it's lossless compression vs lossy for Jpegs.

[pulsar124]

You'd be better off exporting PNGs than Jpegs.  VirtualDub will use those too, and obviously it's lossless compression vs lossy for Jpegs.

It won't work - ACR can only export TIFF, JPEG, DNG, and PSD. I am using maximum jpeg quality (essentially uncompressed), and I don't see any difference in the video when compared to the one made from PNGs. (I did test another workflow, where I did ACR->TIFF->ImageMagick tools (upscaling + unsharp mask)->PNG->Virtualdub, but it involved more steps, and the ImageMagick step was very slow.)

[jamesd256]

OK, didn't know that about ACR as I've never used it.  Sounds to me like you've optimised your workflow.

I use Aftershot pro for raw conversion, but sadly it won't open the DNGs produced by the tools I've tried.

I did get a satisfactory workflow using raw2dng -> DNG -> ufraw-batch -> PNG -> virtualdub -> Cineform 10bit -> Vegas 12 for edit and grade.  What was nice about that was that the denoising in Ufraw was good, but it was slow, and I obviously had to grade on 10bit footage rather than raw.

I have now settled on a workflow based on raw2cdng -> Davinci Resolve -> proxies and round tripping with Vegas (grading after editing for a more joined up approach). 

Have you considered Resolve?

[Mkolaj]

I started testing the new release from Andy600. Everything seems to be working better mlv-wise. MLV viewer in the camera works 50% of the time. Sometimes it just gives an error and freezes the camera, so I have to remove the battery. My bigger issue though- I cannot open the mlv file on my PC. I can view it in MLV viewer, but MLV2DNG doesn't work (not recognized header "STYL"), and MLVdump doesn't do anything.


How do I get dngs?


Tried MLV_dump with different parameters, it stops working/gives one frame, generally speaking- my file is useless now

[dsManning]

Great work on this!  MLVViewSharp.exe will work on Mac/Linux through mono. MLV2DNG/MLVdump do not sadly, but still nice to quickly see focus in the field if you have a macbook.  Message if you need a hand setting this up.

MLV works in continuos at full frame.  At cropped, I get crazy jumping write speeds.  Sometimes drops from low 80s to 50s to 40s in just a second.  Compared to raw_rec on the same build, MLV is jumping all over the place.  When in RAW, I get a few mbs/sec jumps, not the 30mb/s jumps in MLV.

Formatted my card in camera, removing ML, and reinstalled to the card from scratch.  Cleared out everything, and using all speed squeezing techniques, but I can't get it to stabilize.  Tried a few of the buffer settings in MLV, found 4 (or maybe 3) might be best for me.  Any tips?

In the meantime, RAW recording is stable at 1920x1080 for me hanging around a nice looking 82mb/s on this build.  Can't wait to see MLV become my daily driver.  With all that metadata at the bottom of MLV Viewer, I'm pretty excited!

[Andy600]

I started testing the new release from Andy600. Everything seems to be working better mlv-wise. MLV viewer in the camera works 50% of the time. Sometimes it just gives an error and freezes the camera, so I have to remove the battery. My bigger issue though- I cannot open the mlv file on my PC. I can view it in MLV viewer, but MLV2DNG doesn't work (not recognized header "STYL"), and MLVdump doesn't do anything.


How do I get dngs?


Tried MLV_dump with different parameters, it stops working/gives one frame, generally speaking- my file is useless now

It's not 'useless'!

I'm having a similar problem with a large mlv file but can still dump to the legacy format using something like this:

Code: [Select]

mlv_dump -r -o M11-0920.RAW M11-0920.MLV

[1%]

Yea, you have to give it more parameters than raw2dng.. there is also mlv2dng somewhere here.
Its spanning for me fine, I'm trying the compression out to see what it does.

[Andy600]

re: mlv_rec - see update at the bottom of this post http://www.magiclantern.fm/forum/index.php?topic=5586.msg87952#msg87952

Problems 

[1%]

Was it crop mode? That seems to be where the main issues are at.

Yea, crashes for me dumping to dng for some reason.. maybe some of the changed functions? I'd have to try a pre-built one from g3ggo to see.

Wonder how much meta data you need to have raw = mlv, I know the 2 cam models are necessary but wonder about ISO too, etc.