ML .RAW plugin integration OS X

Started by cosmo.lv, November 15, 2014, 01:55:02 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

cosmo.lv

Hi there!
I come up with idea with a ML RAW video plugin to integrate it like all plugins works under OS X

I`m not a programmer, but somebody with knowledge can investigate its work on this.
since MiRAW has capability to play .RAW video files without converting them to .dng - i think we can integrate MiRAW code into OS X system like a plugin to play .RAW natively. (similar to RED RAW .R3D) or how any other plugins works.

5D Mark II


dmilligan

OS X is moving away from QuickTime (which supports plugins) to AV Foundation which does not (AFAIK, the perian project has pretty much been abandoned because of this [citation needed]).Even if you write a plugin for QT or OSX to be able to playback the files, that doesn't necessarily mean that third party apps will be able to use that capability. Many third party apps (Premiere, AE, Resolve, etc.) have their own completely different plugin API. You'd have to re-write a new plugin using the particular API for each different third party app.

This is why I went the direction I did with MLVFS. The API MLVFS uses basically just implements low level file system calls, which are pretty universal and consistent, unlike any plugin API you might use which is probably unique and would limit usefulness to that specific program. All programs use the same calls to talk to the file system, so if we implement at this low of a level, any and all third party programs can use the data we spit out of MLVFS. In fact because these calls are so low-level and fundamental, we aren't even limited to just OSX. MLVFS works on Linux and other *nix type OSs, and it can be fairly easily ported to Windows as well.

It would also actually be possible to transcode to some other format than CDNG with MLVFS, and this would give you OS X support and also support in any third party app that can read that format. However, this would be a tremendous amount of work. The main issue here is that we are dealing with raw (linear and 'bayered') image data, which is a fundamentally different type of data than typical video or picture media. It's usually pretty easy translate video data from one container or format to another, but here we are talking about more than just changing formats, it's a complex and fundamental change to the type of data. You would certainly need GPU acceleration (which is a bit above my knowledge and skill level) for this to have any reasonable speed.

It is true that MLRawViewer has the capability to playback raw files in realtime, but it is written (mostly) in python. To work with MLVFS it would need to be re-written in C. The same would be true of implementing any other type of plugin API. Most of these are typically some flavor of C (C/C++/Objective-C). That's not to say that MLVFS couldn't use any of baldand's work on MLRawViewer, in fact MLRawViewer code would likely be a very helpful source for getting started implementing something like this, but it would still probably be a massive amount of work.

The first step would be something simple and relatively easy like JPEG previews. I've thought about adding this capability to the webgui in MLVFS, but haven't yet had time.

DeafEyeJedi

Great point @dmilligan and looking forward to your future updates as always and Thx!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

cosmo.lv

Thanks dmilligan for reply and explanation :) i hope somebody will take on this :)

i remember in past when i get idea about DSLR RAW somewhere here on forum and i get replied that it is not possible, after 6 months of researching somebody finaly get done it. so.... I think this story about RAW Plugin and API integration for third party apps will come to life. :) :)  8)


Maybe admins can Pin this thread? Alex? :)
5D Mark II

dmilligan


chmee

its not about impossibility, its about the will to code it. and sorry, dont take your opinion so vital, it should be sticky. if you wanna help, do some work on ffmpeg-bug-reporting.

regards chmee
[size=2]phreekz * blog * twitter[/size]

cosmo.lv

chmee - don't really understand what you mean by that. I been supporter for ML for long time and i post here not so often, and ideas always been challenging and i do not say that it is easy task. ;)

i don't have any problems with FFMpeg and anyway, this is not thematic for this thread!
5D Mark II

chmee

as you say, you re not a coder - im not a professional coder either, and i dont know mac-world - but from win-world i can clearly say, there is not "one" code-point to plugin a codec for all apps. one code doesnt work. if it would be so "easy".. mlvfs is the interesting point, because it doesnt need processing power and leaves all data "nearly "untouched. just saying, dmilligan told, i do, this "idea" is not worth to be a sticky.

regards chmee
[size=2]phreekz * blog * twitter[/size]

cosmo.lv

chmee dont panic, everything is fine ;) this forum is for sharing ideas and work. And there we are, to inspire each other, without that ML was impossible. Look how many things and tools, improvements are found here.

if we starting to stop each other, then we lost the idea of this community. :)

and we don't need codec for all apps, we can start with little try. if someone want do it. No problem with that. it is just idea... :)
5D Mark II

TKez

Eventually I hope we can get some sort of native support in fcpx  and davinci. MLVfS is a great idea but I can't preview in real time in davinci with it so I don't find it practical. Batching to cdng or proves with MLRV  is still works best for me.

I do really miss quick look though. I have my thumb permanently resting on he space bar when I'm working as I find instant preview integral to fast workflows when rummaging through files.

Here's a quick look plugin that already supports most of ffmpeg via vlckit so it just needs the mlv implementation in ffmpeg to mature and then our mlv icons will show snapshots. See here for the devs comments https://github.com/Marginal/QLVideo/issues/10

No video unfortunately due to aforementioned avfoundation migration but there's chance it could work in a hacky way if we can work out how to find the memory address of the quick look window.

reddeercity

I like this Idea ! I think ML dev. Should look in to this .
All other workflows take too much time , no matter what
NLE you use. 

chmee

i can say, for win-world again, getting data into the explorer-details-pane is a pain.. but when .mlv and .raw is bounded to raw2cdng, a simple doubleclick opens a player and exit is only a ESC or x. so, from my position i did, what i could (better: what i need)..

but: sorry, this thread is about mac-world.
[size=2]phreekz * blog * twitter[/size]

reddeercity

Yes you have done very well and raw2Cdng is my main tool .
I just would like to see better support in FCPX with raw.
I don't mind working cross platform pc->mac .

DanHaag

Quote from: reddeercity on November 22, 2014, 05:26:48 PM
I just would like to see better support in FCPX with raw.

I don't think support for a certain format is a task anybody else but Apple could enable within their FCPX. 

reddeercity

I can Edit Cdng's within FCPX as a compound clip, but it take some time to setup ingest etc.. can be a real pain sometimes.
I would like to see a ArriRaw type file support which by the way FCPX has native support with Raw adjustments.


 

DanHaag

Second that - still something Apple would have to implement. I'd suggest posting the request on their official support forums for FCPX.

TKez

QuoteI would like to see a ArriRaw type file support which by the way FCPX has native support with Raw adjustments.

http://www.arri.com/camera/alexa/learn/arriraw_faq/

Looks like they made it pretty easy for Apple with an SDK and all. MLV would likely need the same though Apple's not likely to want be seen supporting it when you consider their stance on modding firmwares.

dmilligan

Send me a specification of the arriraw file format and I will implement it into MLVFS (if it's not too different from CDNG, which I suspect it isn't => mostly just different metadata format I think).

TKez

Considering the wide support for ARRIRAW, batch converting from MLV to ARRIRAW would certainly handy workflow solution though it's
hard to tell if it's open on the input side (encoding). Probably not as it would be counter productive.

Does MLV use any form of encoding? I wonder how different the data is besides headers and data structure. (ML module possible?)
Then u have native QT including thumbnails and quick look, native FCPX support, native Resolve support and plugins for pretty much everything else.

From the website:

QuoteIs ARRIRAW an open format?
We recently submitted the ARRIRAW format and header specifications to the SMPTE, requesting the initiation of a new Registered Disclosure Document (RDD). Once the document has been accepted, the SMPTE will publish the RDD on its periodic distribution media and on the SMPTE website. This will ensure that the format is open to anyone who wants to develop applications around it.

QuoteARRI offers a software development kit (SDK) for ARRIRAW processing that software vendors can incorporate into their application. ARRI also supports vendors who wish to implement the ARRIRAW processing procedure on their own. Depending on the implementation, the following processing settings can be adjusted:

Exposure Index (ASA rating).
white balance.
tint (green-magenta shift).
current or legacy debayering mode.
output color space for HDTV, digital cinema (P3), ACES and Log C wide gamut.
different standard aspect ratios.
output resolution.
sharpening of the image output.

With the ARRIRAW SDK, you can also apply a custom ARRI Look File, which offers primary adjustments through printer lights, saturation and additionally lift/gamma/gain (or slope/offset/power).

No info on where to get such in SDK though. I imagine they expect direct contact from developers.


Andy600

I think you're probably barking up the wrong tree regarding ARRIRAW. The RDD will describe how to decode but it's highly unlikely they will give away any significant encoder info, especially with the likes of the Apertus on the horizon.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

reddeercity

@dmilligan the arriraw is just Cdng's , i have 4k arriraw files will look in to the spec. And get back to to you.
This is a good way to go for all non adobe products as Arri is very well supported .

chmee

@reddeercity .not really. this is a samplefile from an alexa. about 2.8K width. as i tested months ago, it didnt responded to the resolution changes. but i only played with it one evening.. had more important things on my converter..


regards chmee
[size=2]phreekz * blog * twitter[/size]