Magic Lantern Forum

Using Magic Lantern => Raw Video => Topic started by: Andy600 on July 20, 2013, 02:40:19 PM

Title: Magic Lantern universal raw video conversion app
Post by: Andy600 on July 20, 2013, 02:40:19 PM
I'm sure this idea has been in the thoughts of many users but wouldn't it be great to convert raw files with the controls of ACR/RawTherapee, the file output and proxy options of the current convertor apps and real-time debayering (for a live preview to enable selective conversion before hitting the convert button), all bundled together into one, universal, cross-platform application specific to Magic Lantern raw video.

All of these things are possible through open source code. It just needs developers to take a collaborative interest to make it happen.


Every NLE or color grading app has it's benefits and drawbacks. The use of plugins like GingerHDR in After Effects and Premier Pro produce amazing results so plug-in integration would be amazing too.

One of the issue's I've had is finding work-around's for problems which usually results in command line or script editing (not my strong point) and/or using completely different editing apps than I am used to.

For instance, Davinci Resolve is wonderful to work with but can produce some nasty mosaic patterns in highlights thanks to it's solitary demosaicing algorithm. Thanks to A1ex I discovered Rawtherapee which has options to use the much better DBL or AMaZE algorithms, producing significantly better results, however, Rawtherapee is an app (a very good app) designed for stills processing and has a lot of bloat that we don't need, especially as we tend to color in dedicated video apps/NLE's. Processed footage can also suffer from flicker in the same way as ACR. Not ideal!

Realtime raw file previewing is not possible in any of the conversion apps (though I believe it is possible with GingerHDR in AE and PPro?) but The issue of real-time debayering, decoding and colorspace conversion for (at least) previews has been shown to work using open source code: http://vimeo.com/67542994 and Soyoyo Fujita has released his source now for Linux and OSX (still needs PC): https://drive.google.com/folderview?id=0B8AXAVUQ9BiKVXN1cDh3dmw1VGM&usp=sharing

Conversion of raw files is currently possible with a1ex's raw2dng app and the extended functionality provided in BATCHelor, raw2cdng (for Cinema DNG conversion), RawMagic (on the mac), EyeFrame etc etc. All of these apps too have advantages and disadvantages but are all developed by individuals with a common purpose.

Putting all of these developments together into one package would, I'm sure, benefit every Magic Lantern user. Think of something similar to Cineform's GoPro Studio app. Uncomplicated and efficient.

With the proposal of a dedicated Magic Lantern video format now in discussion http://www.magiclantern.fm/forum/index.php?topic=7122.msg59525#msg59525 It's the perfect time to do this. The ML raw video format might not get everything that is needed (because of firmware/hardware limitations) but imagine being able to add-back metadata (reel no's, WB settings etc etc) for your own needs or apply a perfect LOG lut before taking your footage into your NLE and then choose the codec you prefer to work with and all savable in your own presets list. The possibilities are vast.

Managed in the same way as Magic Lantern development (using Mercurial/Bitbucket/Github etc) would allow for other 'outside' devs to contribute and I'll hazard a guess that there are more of them working in the field of image processing and app development than there are in the more specialized ARM field that ML devs are in.


So, why don't you app devs put your heads together and create a real Magic Lantern sister app? 

BTW, call it ZOEtrope ;D

(there you go, that's my contribution  ;D).


p.s. sorry Mods, I should probably have put this in the Post Processing section  ::)
Title: Re: Magic Lantern universal raw video conversion app
Post by: Andy600 on July 20, 2013, 03:27:47 PM
On my wishlist would be:

Realtime preview of raw file (as shown in Soyoyo Fujita's demo) with in and out point selection for quickly removing excess or unwanted footage (as with GoPro Studio)

ACR/Rawtherapee-type exposure/highlight/contrast adjustments but without the flicker that those apps can cause.

Choice of Demosaicing algorithms (AMaZE etc)

Upscaling/Downscaling with selectable algorithms for minimal aliasing

Ability to apply/bake true LOG luts

Choice of output codecs without the need for scripting (ProRes, Cineform, DNxHD, H.264, TIFF seq etc)

Dead/hot pixel removal

Auto frame rate conform

Sharpening (Luma/Chroma - for simple preview or render. I tend to sharpen in an NLE)

Batch raw conversion to DNG/CDNG at selectable bit rates

Modular view so you can enable/disable features

Metadata - auto and manual addition (see my idea for getting full EXIF data: http://www.magiclantern.fm/forum/index.php?topic=7122.msg59712#msg59712)

Preset storage

Dual ISO processing



Title: Re: Magic Lantern universal raw video conversion app
Post by: deleted.account on July 20, 2013, 04:20:08 PM
Seems like a lot of effort and duplication for a handful of opps that have to be done from raw data to RGB.

Most stuff specific to raw development is done once and not a lot even under user control, apart from White Balance and personally I don't use that as a creative color control.

All other opps are done in RGB within any NLE or grading app and a decent one at 32bit float.

Admittedly the tools that work in Lab may be a bit lacking but then there's YCbCr at 32bit in an NLE or grading app as an alternative.

A small batch script using libraw or dcraw will give flat 'open' gamut 16bit output to tif, admittedly intermediate storage but at least quick and full control of raw development, very good debayer, no sharpening and no camera curve applied.
Title: Re: Magic Lantern universal raw video conversion app
Post by: Andy600 on July 20, 2013, 04:37:59 PM
Quote from: y3llow on July 20, 2013, 04:20:08 PM
Seems like a lot of effort and duplication for a handful of opps that have to be done from raw data to RGB.

Most stuff specific to raw development is done once and not a lot even under user control, apart from White Balance and personally I don't use that as a creative color control.

All other opps are done in RGB within any NLE or grading app and a decent one at 32bit float.

Admittedly the tools that work in Lab may be a bit lacking but then there's YCbCr at 32bit in an NLE or grading app as an alternative.

Maybe, but I think this, developed parallel to Magic Lantern's proposed new format would enable the development of an open source alternative that has all the features and not just some of the features that we need.

Yes, you can do it all using a variety of NLE's and photo editing apps but in the case of ACR for example it is both slow and prone to flickering. Rawtherapee too.

Having correct metadata in Resolve is both useful (and essential for round-tripping) and yes, there are apps that can add reel numbers etc but it would be nice to add more metadata to enable Resolve to work as intended. It's guesswork most of the time.

Scaling footage is crap in After Effects unless you use a costly Red Giant plugin. Resolve is better but could be better still and as I said, demosaicing in Resolve has issues which means I have to revert to Rawtherapee again for AMaZE or DBL.

I can think of many more examples of  one app/NLE's abilities over another but that's the point. So many apps are needed because they all lack something that would give a better end result. Maybe the developers at Black Magic, Adobe etc will eventually introduce more support for Magic Lantern raw video but why rely on them when there is a perfectly capable open source dev community who can bring it all together into a unified conversion app that will give us the best possible starting point for color grading and editing in the NLE and codec of our choice?

Give me AMaZE in Resolve and I would be happy but that wont happen because of licensing costs.


I bet the same was said of Magic Lantern at some point. Seems like a lot of effort! Why bother when you can buy a camera that can do it all already etc etc and now look where we are  :)


One other thing. This is something the ML team CAN accept donations for without pissing off Canon.
Title: Re: Magic Lantern universal raw video conversion app
Post by: mageye on July 20, 2013, 05:39:33 PM
@Andy600

I totally agree that it would be really good/useful/helpful if there was a more well rounded general application to deal with the RAW files coming from the Magic Lantern project.

Indeed most things have some workaround but it would certainly be useful to have a one stop shop.

I especially agree that a preview would be very, very, very, very useful. Also I totally agree with the ability to have elementary cutting tools (basic in/out as you mention).

I posted something on this myself http://www.magiclantern.fm/forum/index.php?topic=6773.msg55180#msg55180 (http://www.magiclantern.fm/forum/index.php?topic=6773.msg55180#msg55180) and had absolutely no response whatsoever?. I sometimes think that I am alone in wanting such tools but I am sure there are many people that would benefit from such tools.

I certainly support what you are saying and I hope there are more people out there that can see this too.

Oh and the ability to have optional NON LOSSY COMPRESSION in DNG/CinemaDNG would also be a pretty useful too.

I wish I had the ability to code these tools myself but my head turns to noodles when I try to comprehend programming. :(
Title: Re: Magic Lantern universal raw video conversion app
Post by: Andy600 on July 20, 2013, 05:54:18 PM
Thanks Mageye. I guess sometimes topics fall off the edge. Sorry, I didn't see your post or I would have contributed to it.

I'm trying to learn the basics of coding and trying to understand the interaction of some of the other apps which is why I feel confident to know this is feasible but I'm nowhere near being able to contribute code.

I'm hoping Chmee, Fatpig and co will start looking at this and maybe some of the ML devs too. I think it's needed and would easily be the most popular app for transcoding and converting ML raw video especially if it were cross-platform and developed specifically for the new proposed Magic Lantern Video format (.mlv) and included 14bit debayering etc.

Sometimes good ideas need a little pushing to wake the individuals who can make it happen. This is a no-brainer idea to me and maybe it is something Magic Lantern can benefit from by way of contributions. Who knows?
Title: Re: Magic Lantern universal raw video conversion app
Post by: deleted.account on July 20, 2013, 06:37:11 PM
Andy you misunderstand me, I wasn't saying don't bother. I find it rather irritating to read naive calls to arms for open source software solutions from people who have little or no experience or knowledge of the history and immense effort to create such involved apps and think its so easy and a no brainer.

Fixing raw demosaic in Resolve is a small simple step especially for a large commercial enterprise. Creating an app as you describe is not like creating some dumb glorified script of a batch conversion app.

As some one who uses and has used open source software (Linux) for twenty plus years and not seen one decent NLE or grading app come in that time not to mention higher than 8bit precision color grading or even a GPU based motion analysis library for deflicker, denoise and motion compensated sharpening and such its a bit of a joke to here those recently aquainted in passing to the open source world who are not even on Linux to start talking of such big projects as easy and no brainers.

Much of what you mention are not processing done on raw at all but the RGB output from a raw decoder. Either consider building on an open source app like ufraw, rawtherapee etc or just badger the commercial enterprises to fix their shit you pay for. :-)
Title: Re: Magic Lantern universal raw video conversion app
Post by: Andy600 on July 20, 2013, 07:14:59 PM
@y3llow - You seem to have misunderstood me too. I never said it was easy and my comment about it being a  'no-brainer' was simply the idea and not the process.

I was working with Cobol 25 years ago and oversaw my IT dept's (then called data processing dept) move over to Unix. I have used Linux on and off for about the same period as you but, like you, found it lacked in comparison to commercial OS and apps. This is the reason why I stuck with Steinberg Cubase and pro tools through a 15 year professional music career (which is why I lost touch with coding) and I chose Adobe software for my hobby which is photography and video.

I understand the huge efforts involved in bringing these applications to fruition having spent time working with games developers, albeit as a musician, but with experience of the way the game devs worked so I do know it's neither 'easy' nor 'simple'.

Yes, my proposal goes beyond the a simple conversion application and may step on the toes of Adobe, Black Magic etc but Open Source development across multiple applications have achieved all that I suggest. No, I don't think for a second that it would be easy to cherry-pick snippets of code and throw it all together as an app. It's a huge undertaking! BUT there are talented individuals here and outside of Magic Lantern with way more knowledge and experience than you or I, who 'could' make it happen.

Regarding your point about the large commercial enterprises, well yes, of course, they will have the resources. Companies such as Black Magic have introduced support for DNG as (I think) a direct result of Magic Lantern raw video. Why else would they do it? They already support the Cinema DNG format in their cameras and software so why bother? Maybe they will go further and yes, maybe they will add better demosaicing etc (maybe that will come in Resolve 10) but they have their own brand and cameras to think of without fully optimizing their software for ML raw video.

Incidentally, if you have followed the Cineform conversion thread you'll have read why these companies cannot simply use just any algorithm/code they wish, even if they do find a better one because of costs involved in licensing and/or royalties. However, many of these advancements, developed in open source, CAN be used for something like this.

Your last line is exactly what I am suggesting minus the "badger the commercial enterprises to fix their shit you pay for" line.

Title: Re: Magic Lantern universal raw video conversion app
Post by: deleted.account on July 20, 2013, 07:35:09 PM
My last line was building ON an app like.... not build from scratch.  Discuss with the developers of those apps in existance, mock ups, info about ML raw. Darktable devs are considering support for video already. But again so little is on actual raw data, so raw editor is not necessarily the starting point. What about raw support in Blender for example, probably not a good example. :-) Shotcut maybe, that's cross platform.

A lot of start from scratch, reinventing whats already out there goes on including things like Openshot NLE a misguided attempt that didn't go towards solving the problem from twenty years ago in the days of Kino and now up for complete recode from crowd sourced finances.

Yes of coarse there are talented open sources coders out there but don't you think that if they were both talented and had endless time we'd have a decent open source NLE or grading app, that we'd have all heavy lifting libraries already.

Open source works generally by a developer needing to find a solution to a problem THEY have, then it might like minded developers join, it doesn't work by naive posts on forums from non coders. It would take more than one talented coder to build one aspect of an app like you mention in any reasonable amount of time.

And really I shouldn't have mentioned Linux as if it were the only open source route, I use many cross platform or Windows based open source apps. Just to avoid any misunderstanding.

Title: Re: Magic Lantern universal raw video conversion app
Post by: Andy600 on July 20, 2013, 07:50:29 PM
Quote from: y3llow on July 20, 2013, 07:35:09 PM
My last line was building ON an app like.... not build from scratch. A lot of that goes on including things like Openshot NLE a misguided attempt that didn't go towards solving the problem from twenty years ago in the days of Kino and now up for complete recode from crowd sourced finances.

Yes of coarse there are talented open sources coders out there but don't you think that if they were both talented and had endless time we'd have a decent open source NLE or grading app, that we'd have all heavy lifting libraries already.

Open source works generally by a developer needing to find a solution to a problem THEY have, then it might like minded developers join, it doesn't work by naive posts on forums from non coders. It would take more than one talented coder to build one aspect of an app like you mention in any reasonable amount of time.

And really I shouldn't have mentioned Linux as if it were the only open source route, I use many cross platform or Windows based open source apps. Just to avoid any misunderstanding.

Lightworks and Resolve. Both free. Both professional grade and both can (or will soon) work on Windows, OSX or Linux. Maybe an OS NLE or grading app is not needed so much.

As for devs, yes it will need more than one, more than 2 and preferably a whole swimming pool full of them :) Which is why I suggested Fatpig talk to Chmee and other current app devs to get the ball rolling.

Naive? really? We obviously differ in our experiences and points of view but open discussion has been, imo, a good way to spark ideas.

Maybe because of this and other posts it will happen, maybe it wont but if no one suggests these things we will have to rely solely on developers to think of absolutely everything.

...yes, I know ML devs are usually way ahead of the curve with ideas but talking about things can spark new ones.

I think this is a valid discussion, don't you?

Title: Re: Magic Lantern universal raw video conversion app
Post by: deleted.account on July 20, 2013, 08:53:23 PM
Valid discussion, well talk is cheap as they say. :-)

I think first identify open source apps that form a good base like Blender with its 32bit float nodal OpenCL compositor, VSE and color managed (OpencolorIO) support for LUTs or Shotcut with its 16bit GLSL grading tools and basic in/out editing and playlist functions a bit like Bulletproof then approaching devs of those apps with structured ideas, mock ups, information and hopefully a couple of ML raw related Devs like you suggest with patches for raw support and EDL output for example and see what transpires. Blender devs would be complete dicks not to accept raw support in Blender.
Title: Re: Magic Lantern universal raw video conversion app
Post by: Andy600 on July 20, 2013, 09:16:28 PM
Quote from: y3llow on July 20, 2013, 08:53:23 PM
Valid discussion, well talk is cheap as they say. :-)

I think first identify open source apps that form a good base like Blender with its 32bit float nodal OpenCL compositor, VSE and color managed (OpencolorIO) support for LUTs or Shotcut with its 16bit GLSL grading tools and basic in/out editing and playlist functions a bit like Bulletproof then approaching devs of those apps with structured ideas, mock ups, information and hopefully a couple of ML raw related Devs like you suggest with patches for raw support and EDL output for example and see what transpires. Blender devs would be complete dicks not to accept raw support in Blender.

It is I agree and I've got plenty of it  ;D

That all sounds like a good idea in principle ::)

Rawtherapee is closest to what I had in mind so I guess I'll go hassle the devs there.
Title: Re: Magic Lantern universal raw video conversion app
Post by: mageye on July 20, 2013, 10:21:09 PM
Yes I thought it would be a good idea to get some communication with the developer of RAWMagic. OK so maybe I didn't search deep enough but I never even found the persons name who coded it?.

I then decided to try to make some contact through the Rarevision site. The pages on that site don't really seem too active but I thought I would send an email to Rarevision (ideally I wanted to post to the developer of RAWMagic).

I sent what I would consider to be a polite email with a suggestion for RAWMagic. I suggested that a simple edit option (IN/OUT points) be included in the great little tool he had created.

It was just a suggestion and I didn't want them to feel like I was pestering them. I got no response at all. This was about three weeks ago. I didn't even get any 'automated acknowledgement' as a confirmation that they had got the email. I doubt that who I was trying to get the email to ever read it.

I understand that developers are gonna be busy like anyone else (probably more so if they are talented). What they do is valuable. I am sure that they have bigger fish to fry.

What I can say though is that; at the very least, I tried. :-X
Title: Re: Magic Lantern universal raw video conversion app
Post by: deleted.account on July 20, 2013, 10:29:09 PM
Andy best of luck, although I think you may be approaching it from the wrong end ie: image manipulation vs video. I'd not underestimate what's involved getting an app built for image processing at high precision to playback video or image sequences in realtime with color processing applied vs an app already built for video playback that needs ability to import raw and decode to RGB.

As mentioned the amount of opps that actually use raw data are a handful, majority of color processing, temporal NR and MC sharpening are not done on raw data.

The need is to decode raw into a 16 or 32bit processing chain.

Yes there are some nice Lab based tools in these raw editors which would be good to have in a video app but as said YCbCr color tools are available.