ML's Goal?

[AnotherDave]

I've been using the ML Raw hack now since May, and I'm wondering what the ultimate goal of the project really is?

Obviously it is about adding features to these cameras that weren't included and even limited for marketing, but what is the end goal?  Are WE (the users and developers) attempting to make these more professional, practical, and useable so that smaller budgeted projects and independent filmmakers have access to high image quality?

To me, it seems like ML has gotten stuck on just pushing numbers lately when there are serious functionality issues that need to be solved.
Issues like...

- moving the record/buffer information to a place that doesn't block the screen
- storing WB, Picture Profile, & image finetuning information so these images are WYSIWYG right after conversion
- adding free run timecode jam syncing capabilities so that external audio recorders can be used more easily
- internal audio recording/syncing
- addressing the issues with HDMI output, playback
- standardizing the .MLV file format so that applications can be created to read the recorded information without the need for conversion to .DNG

To allow this camera to be used more widely on productions, it seems that all of these issues should be address before experimenting on speed increases for slower cards and dynamic range hacks (a1ex you rule, btw!).

I love what the developers have done, and I want to use the 5D3 with the ML Raw hack on everything I do.  But the perception of being a "hack," "nerdy," complicated thing that will "make your camera explode" is a hard one to crack when these issues aren't being addressed.

Am I wrong?

Thanks.

[1%]

moving the record/buffer information to a place that doesn't block the screen

I did this to an extent.

Other stuff you have to wait. Its useless if the cameras can't give you a decent resolution to record.

Some of these things aren't trivial to implement either and require reverse engineering. Ie the sound/genlock/timecode.

storing WB, Picture Profile

Completely pointless. I like to shoot UNIWB but any old white balance you just correct out in post.

Aperture/Fstop/iso/etc are good ideas tho and I think g3gg0 is implementing it.


The goal for me is to have a camera I can work with with actually features related to production... not this consumer oriented crap they shovel in. I'm not just taking pics of my non-existant kids on vacation here with the money spent and I'm sure nobody else is either.

[AnotherDave]

Some of these things aren't trivial to implement either and require reverse engineering. Ie the sound/genlock/timecode.

Is genlock/TC even being worked on?  I understand that it is complicated, but iPhone apps can do it so it can't be impossible.

Completely pointless. I like to shoot UNIWB but any old white balance you just correct out in post.

Aperture/Fstop/iso/etc are good ideas tho and I think g3gg0 is implementing it.

If I'm handing in footage to a client, they shouldn't have to be a colorist to understand how to make the footage look professional.  Couldn't WB info just be stored in the header?

I get how aperture/f-stop/iso would be useful information for special effects, and compositing... but that doesn't help for giving people good images out of the box.

The goal for me is to have a camera I can work with with actually features related to production... not this consumer oriented crap they shovel in. I'm not just taking pics of my non-existant kids on vacation here with the money spent and I'm sure nobody else is either.

Me too!  I haven't seen the Red Dragon in the flesh yet, it sounds like the only thing that could even compete with what you've given us...  But, since we're not just taking pictures of kids... shouldn't we try to match the Red on other levels?

Dave

[Andy600]

WB as shot is included in the new .mlv format.

As for giving clients footage "straight from the box" isn't that what dallies are for? To the best of my knowledge clients rarely (if ever) get to see raw footage without at least some minor post work. Clients wouldn't be able to watch raw footage anyway without first converting it to a watchable format and WB adjusting is perhaps the easiest part of that process.

Raw footage on any camera looks crap until it's tweaked a bit.

[a1ex]

Well, there's no hard "end goal", it's more like a hobby project where we try to push the cameras beyond their limits, while also sharpening programming skills and sharing knowledge. Everybody here is working on their spare time (and I happen to have a bit more time because my daily job is not a 9 to 5 one). Also, everybody is free to work on whatever he feels like; there's no hard to-do list.

I come from a research background, so I like to experiment with cutting edge techniques. And of course, I like sharing these tools with other people that may find them useful and can improve these things.

For metadata (WB and a ton of other settings), just read the MLV thread, this is work in progress. Synced audio is pretty hard (but not impossible), and I don't have any use for it myself, for example. On the other hand, dual ISO is something I'd use in over 80% of my pics (since I like to shoot in very tricky lighting conditions). I don't shoot professionally, but I shoot some events for friends every now and then (not paid work, just hobby level, and mostly photos and sometimes timelapse).

Anyway. For me, the end goal is not to provide a finished product that everybody can consume. I'd rather see it as an open software platform where others can program their own enhancements and share them with the community (which actually happens to some degree). I prefer it to be a little on the nerdy side and attract knowledgeable people and enthusiasts; after all, ML is a tool for power users.

(and while we are at it, this reminds me I should revive the scripting support)

[1%]

For me, the end goal is not to provide a finished product that everybody can consume

This. Most professional cameras aren't like this either. You have to know what you're doing, there is no way to get around this.

Clients requesting raw and then not being able to set WB means they really shouldn't be getting raw.

[AnotherDave]

@ a1ex
You rule!

@ Andy
Clients often hire myself and other DPs out as a shooter.  At the end of the day, I hand over the hard drive and I don't see the footage again until the finished edit.  Lately, I've gotten into offering additional services as a final colorist for footage that I've shot - those jobs are few and far between though.

It is very easy for a client to "blame the DP" for problems on their shoot, and I like to minimize those claims as much as possible.

I'm frankly amazed with what you guys are doing in your spare time.  Testing/posting about this is a spare time project for me too.  My goal is to assist and help guide this to being very functional and adding to the positive outlook of this firmware.

[Redrocks]

I'd like to see all these issues ironed out, but a lot of the problems you mention didn't exist till a few months ago, Dave. As for handing off footage to clients, that's what you are getting paid for in all fairness. You could always shoot in h.264 and hand that off? From watching an interview with Trammell Hudson, ML's original goal was making the dslr more suited to independent film and video. It's legacy will hopefully be putting professional level tools in the hands of more film makers. If Stanley Kubrick told you to run wild with a camera, ML taught you how to tame it.

[Andy600]

@AnotherDave - That would make me very nervous but I'm just an amateur. You could argue that it's the editor/colorists job to fix WB. Even when WB is in the metadata it will still be tweaked 90% of the time.

I love the rolling development of Magic Lantern. You never know what's gonna happen next. As a user I can play with things that only a couple of years ago were only available on hugely expensive cinema cameras and it's also boosted my own interest in coding so it's educational even though I don't have a clue what 99.9% of the code means or does.

I'd personally love to see what would happen if the devs started playing with Black Magic cameras too

[1%]

It is very easy for a client to "blame the DP" for problems on their shoot

Nightmare/uninformed clients like: http://clientsfromhell.net/

[g3gg0]

Obviously it is about adding features to these cameras that weren't included and even limited for marketing, but what is the end goal?  Are WE (the users and developers) attempting to make these more professional, practical, and useable so that smaller budgeted projects and independent filmmakers have access to high image quality?

To me, it seems like ML has gotten stuck on just pushing numbers lately when there are serious functionality issues that need to be solved.

hello AnotherDave.

after you got some generic answer, i will respond to the backgrounds of your post.
as you will see later, i am a bit upset about your post - so excuse me if it may attack you a bit.
at least i am trying to stay calm.

you already explained your personal needs in magic lantern several times in the other threads.
a few things you blame not being in our code was already implemented, but still you call us "just pushing numbers"

our try to thoroughly testing features and performance before releasing is just a reason for you to call us not to care about "end users".
all things you mentioned what is not perfect for your needs, is about a feature we called EXPERIMENTAL and NOT DONE YET a dozen times.
every thread says: "HEY! This is us playing around! Its not done yet!"

as alex already explained, this is a spare time project for all of us. we are not making any money from it and we dont want to do that.
i do not know how he feels, but i tend to implement requests who are asked for politely and are worth the effort.
whereas you tend to explain something that is just your personal needs, is a fundamental deficit of ML and the devs are careless.


as already explained:
 - we are aware that the raw_rec module (which is experimental btw and the only thing you question) is not perfect for a cinematographer's set yet
 - we are aware that post processing isn't very supported yet. why do you think we implemented a new format? to sacrifice write speed for nothing?
 - we are aware that recording a new hobbit movie with ML's raw_rec might be a bit time consuming and not straight forward

i can ensure you: we implement anything that is requested, but we have to priorize, as we are no company with
nearly unlimited resources for development. we have 1-4 hrs a day, minus the time needed for writing posts. not n*40 hrs.

feel free to support us in implementing.

[VisualPursuit]

I am also one of the people who have stated their wishes, and I realize you
seem to feel this is done as demanding. This was never my intention.

I did so in an attempt to explain how ML is used in my work and give feedback.

Your work is much appreciated and I am in no way asking release dates,
or try to push you towards anything. You guys decide.

What I did do was ask in several places about the support of the current
firmware of the 5D MkIII, and I never received any answer nor could I find
any postings on the background.

There must be a reason why current development is done on 1.3.1, maybe even
a problem.  A quick sentence on that might help others understand what the
problem is and maybe others can step in and help solving it.

Please: I am not trying to push you to do anything, I am just trying to understand
a little background.

You do this for a reason - fill us in and maybe others can help better.

Also, anybody posting builds: A simple "for firmware 1.3.1" makes everything
perfectly clear and stops questions in advance. I have received a warning for
asking if the build announced in a posting would work with 1.2.1.
Did the search before and read everything associated with this build folder,
but could not find out if or if not. So I asked, and a simple "no" would have been
much better than issuing a warning. Lourenco was so friendly to answer my
question by pm. The answer is no and that s fine for now.

[g3gg0]

thanks for your feedback.
about firmware revision: i didnt have time to dig out the wiki page we post usually.
Audionut just posted it in another thread, so there you are:

http://wiki.magiclantern.fm/faq#when_will_you_release_the_next_version

this answers exactly your question. i.e. i wouldnt have said anything different from what's written there.

we do not deny switching to v1.3.1, we just didnt spend two days to do that stuff.
especially when there is stuff with a lot more "pressure" from the users.
if someone digs out all the stubs (stubs.S, const.h, hardcoded stuff in modules) that are necessary to work on v1.3.1 and makes a pull request, we will most likely switch to this revision.
until there is no help, you have to wait till alex or anyone else takes their time to do this.

[VisualPursuit]

if someone digs out all the stubs (stubs.S, const.h, hardcoded stuff in modules)

Now THAT is a great answer, and it might help getting the thing going.

Thank you very much!

[AnotherDave]

hello AnotherDave.

after you got some generic answer, i will respond to the backgrounds of your post.
as you will see later, i am a bit upset about your post - so excuse me if it may attack you a bit.
at least i am trying to stay calm.

you already explained your personal needs in magic lantern several times in the other threads.
a few things you blame not being in our code was already implemented, but still you call us "just pushing numbers"

our try to thoroughly testing features and performance before releasing is just a reason for you to call us not to care about "end users".
all things you mentioned what is not perfect for your needs, is about a feature we called EXPERIMENTAL and NOT DONE YET a dozen times.
every thread says: "HEY! This is us playing around! Its not done yet!"

as alex already explained, this is a spare time project for all of us. we are not making any money from it and we dont want to do that.
i do not know how he feels, but i tend to implement requests who are asked for politely and are worth the effort.
whereas you tend to explain something that is just your personal needs, is a fundamental deficit of ML and the devs are careless.


as already explained:
 - we are aware that the raw_rec module (which is experimental btw and the only thing you question) is not perfect for a cinematographer's set yet
 - we are aware that post processing isn't very supported yet. why do you think we implemented a new format? to sacrifice write speed for nothing?
 - we are aware that recording a new hobbit movie with ML's raw_rec might be a bit time consuming and not straight forward

i can ensure you: we implement anything that is requested, but we have to priorize, as we are no company with
nearly unlimited resources for development. we have 1-4 hrs a day, minus the time needed for writing posts. not n*40 hrs.

feel free to support us in implementing.

I'm sorry if we got off on the wrong foot.  I think it is more that my tone doesn't come across in text.

I just want to help.

I really dig everything you guys are doing.  For me, every new version is like Christmas! 

There is only one thing separating this camera (5d3) from the "high end" cameras and that is marketing (and a whole lot of money/hardware). 

I'm not asking for features that I want.  This is already what I want.  I can make this work for me in a snap.  It's great!

All I am doing is offering suggestions for what I know will open this firmware up to more uses in businesses where not everyone is as technologically proficient.

I'm sorry if I came across as a jerk.  You rock!

[stevefal]

From thread http://www.magiclantern.fm/forum/index.php?topic=7139.700

I'm going to commit blasphemy and say I think Dual ISO is a dead end for video. Some of the samples out there have amazing dynamic range, but the aliasing is just terrible.

Having HDR with no time-based artifacts may be worth it for stills, but even there, bracketing is an option with even more range, and it works fine in many cases.

I say this because I really wish that RAW video was perfected and truly field-ready. RAW video was already drop-dead awesome before dual-ISO came along, and yet there's still plenty of work before it's really usable. Solving audio, external displays, cropping overlays, status indication UI, file format, time-code, exposure/focus aids, workflow and, of course, stabilizing the system are all worthwhile if not critical. Finishing them is a job.

I think dual-ISO video is intriguing, but a huge "feature-creep" on RAW video, and it would be best to ship the first before becoming consumed by the second. ML RAW video could mean a revolution in film-making, but it's essentially still a prototype.

How about getting RAW video done? Think of the films, the press, the accolades! It's that good.

I'm happy to help any way I can.

Dual ISO for video is a very usefull tool. And like every tool it has specific task to do. It is not usable in every situation though, for me it is great in cooperation with HDR function.
If I want to have high dynamic range with lots of detail - i use HDR. It gives motion artifacts so i use it when there is no fast camera movement. If I want high dynamic range in motion - i use dual ISO. Aliasing and moire can be less visible with motion blur and of course when background is out of focus, i tested it also with some chroma blur and it gives quite good results in decreasing artefacts.
It is great to have such awesome tools, they are still work in progress and i don't feel it is a dead end.
Even RED's HDRx has some quirks with motion so seems there are no way to have an ideal solution.

Even if dual-ISO video has value in those cases, I still wish that RAW video could be finished and shipped first. Shooting RAW provides massive value compared to in-camera video, and without the huge caveat and complication of aliasing.

In your opinion!

Besides, it's not like work has stalled on raw recording, g3gg0 is doing some awesome work on it here.  That's a massive job going on there, coding an entire container format.  How long does it take the MPEG to deliver a new format.  Proposals, drafts, deciding who gets the biggest percentage of the patent pool.  Years!
That includes a ton of stuff you mentioned in your previous post.

P.s.  Take further discussion into another thread please.

Of course my opinion! Also agree the file format etc. work is fantastic!

Guys listen, I think all RAW video development should be put on hold for now because I personally don't need it. I know this might be a controversial point of view, but if you'd stop making video so much, I'm sure you'd see things my way.

More seriously, dual ISO is awesome. It's perfect for shooting stills outside where one exposure sometimes doesn't have enough range but stuff is blowing around too much for an HDR bracket. I've already started using it during every shoot. This feature is infinitely more important to me than anything that involves video.

Fair point, I did imply bracketing was good enough for stills, even though I hedged on it.

But really I am talking about dual-ISO *video* and it's value/priority relative to RAW video. Supporting dual-ISO video has lots of development and workflow implications, and I just think it's less important given the artifacts. It's not that I would NEVER use it, just that I'd default to RAW and bite the bullet on dual-ISO video if the shot really really demanded it.

I don't dispute the value of dual-ISO for stills, even with whatever aliasing you might end up with. But with video, the aliasing crawls and vibrates like crazy so it's much more objectionable when it happens, which is often.

Of course, I'd never bet against a1ex or someone else wiping out the video aliasing issue, but I haven't heard of anything on the horizon. If it does get solved I'll be jumping up and down with everyone else.

So my opinion isn't that dual-ISO should be killed, but I do use video and many would love a stable, refined version of RAW video that's suitable as a workhorse. I also think that widespread adoption of ML RAW video has incredible industry-rocking potential.

Just sayin'.

[Audionut]

stevefal I've merged your topic into this one.

Solving audio, external displays, cropping overlays, status indication UI, file format, time-code, exposure/focus aids, workflow and, of course, stabilizing the system are all worthwhile if not critical.

Audio, file format, time-code, workflow, stabilization is being worked on here, here, here and here

Cropping overlays and other raw recording work is in the todo list.  You get honorable mention in that todo list also. 

Finishing them is a job.

ML is a job?

Well, there's no hard "end goal", it's more like a hobby project where we try to push the cameras beyond their limits, while also sharpening programming skills and sharing knowledge. Everybody here is working on their spare time

Seems the developers think otherwise!

I think dual-ISO video is intriguing,

So does a1ex!

On the other hand, dual ISO is something I'd use in over 80% of my pics (since I like to shoot in very tricky lighting conditions).

How about getting RAW video done? Think of the films, the press, the accolades! It's that good.

Anyway. For me, the end goal is not to provide a finished product that everybody can consume.

This. Most professional cameras aren't like this either. You have to know what you're doing, there is no way to get around this.

Clients requesting raw and then not being able to set WB means they really shouldn't be getting raw.

This one from g3gg0 where he is answering another user who thinks their priorities are the most important.

you already explained your personal needs in magic lantern several times in the other threads.
a few things you blame not being in our code was already implemented, but still you call us "just pushing numbers"

our try to thoroughly testing features and performance before releasing is just a reason for you to call us not to care about "end users".
all things you mentioned what is not perfect for your needs, is about a feature we called EXPERIMENTAL and NOT DONE YET a dozen times.
every thread says: "HEY! This is us playing around! Its not done yet!"

as alex already explained, this is a spare time project for all of us. we are not making any money from it and we dont want to do that.
i do not know how he feels, but i tend to implement requests who are asked for politely and are worth the effort.
whereas you tend to explain something that is just your personal needs, is a fundamental deficit of ML and the devs are careless.


as already explained:
 - we are aware that the raw_rec module (which is experimental btw and the only thing you question) is not perfect for a cinematographer's set yet
 - we are aware that post processing isn't very supported yet. why do you think we implemented a new format? to sacrifice write speed for nothing?
 - we are aware that recording a new hobbit movie with ML's raw_rec might be a bit time consuming and not straight forward

i can ensure you: we implement anything that is requested, but we have to priorize, as we are no company with
nearly unlimited resources for development. we have 1-4 hrs a day, minus the time needed for writing posts. not n*40 hrs.

feel free to support us in implementing.

Developers do this on their own free time.  It is NOT a job.  Think of all the great features we have to play with thanks to the ML team.  We get these features for free, not free as in free speech or free beer, free as in free.
In the last 2 months or so alone, we got, Auto ETTR, Dual ISO, Auto Exposure, Raw based overlays, Raw Recording, DNG silent pics, Post Deflicker, Automated DotTune AFMA, File Manager, Mem_Spy, Pic Viewer, Arkanoid, Custom text editor, File prefix adjustment, the Module System and whatever else I have forgotten.

You didn't get charged to use ML, and you don't get charged for updates.

Emotions and intent don't convey well with just text, generally posts that say, "I want", or, "You should be doing this", only serve to piss the devs off.  And I personally like the devs being happy, it means we get more cool features to play with, for free!

[stevefal]

ML is a job?

No, an expression, as in "finishing them is a challenging task"

------------

Ha, another perilous thread There are so many personal factors and assumptions going on. As much as some people may get pissed hearing such requests, I think they should understand that not all onlookers can tell:

 - how many contributors there are
 - who does what
 - how many may come and go
 - what each's priorities, capabilities and interests are
 - what policies and goals are official, if any
 - what is possible
 - who "controls" ML, if anyone

Given that the project is run on a community, I don't think it's fair for one contributor to say "we" get pissed at this or that, or "our" priorities are such-and-such. How do I know there isn't some contributor who will be convinced by my views and decide to act on them? I can't know that, so I put my views and requests out there hoping it will resonate.

I totally understand that people work on ML on their own time. I also understand that people have their own motivations. My motivation is to use the tool, to provide others with a great tool, and to help shift the industry towards excellent affordable film making products. I don't expect everyone to share my motivations, but that doesn't stop me from evangelizing them, nor do I think it should.

I think that "What is ML's goal?" is a great rhetorical question. That is, I think for anyone to answer it absolutely is a disservice to the community and off-putting to people with different hopes. Who is anyone to claim what ML may and may not be? It can be whatever a motivated group or individual can make of it, right?

If I say "more beginners would use ML if blah blah blah", I don't want someone replying "No, because ML is for power users". I'd rather hear, "MY goal is to help power users". This way I understand that he is speaking for himself and his own investment of time. It short-circuits a contentious discussion that makes people feel unnecessarily defensive or disenfranchised.

I'm guessing that most ML contributors don't chip-in for ego-stroking, and most wouldn't stop because of some dissatisfied user. And I'm also sure that ML users don't make requests in order to berate and demoralize contributors.

So it would be great if users didn't feel the need to sugar-coat their heartfelt wishes, and contributors didn't feel the need to defend their progress in them, or lack of interest in them. I don't take any of this personally, and I don't expect anyone else to.

All that said, I would echo AnotherDave's first post above. Not as a complaint, but as a personal statement of hope for a tool I'm eager to use. "Pushing numbers" mirrors my comment about dual-ISO video versus RAW video. I personally hope that people will decide to downplay dual-ISO video for the sake of advancing RAW video faster, and shipping a field-ready version sooner. But if they can't, won't or don't, I blame no one. No one owes me anything.

Finally, I agree that typing stuff sucks. 99.9% of the people here are well-intentioned, even if not perfectly informed. We should remember that when we feel insulted or snarky. It's just a waste of time because we can bet the other guy is actually a good chap.

[jrumans]

I think the workflow isn't that bad actually, there is an extra step but that will hopefully be shortened with the Raw support coming to Premiere Pro in October (for those using PP).  I love being able to go into ACR and haven't found a better way to recover the shadows / highlights.  Just my humble opinion.

Example of what ML Raw has done for me:

[a_G]

Well, there's no hard "end goal", it's more like a hobby project where we try to push the cameras beyond their limits, while also sharpening programming skills and sharing knowledge. Everybody here is working on their spare time (and I happen to have a bit more time because my daily job is not a 9 to 5 one). Also, everybody is free to work on whatever he feels like; there's no hard to-do list.

I come from a research background, so I like to experiment with cutting edge techniques. And of course, I like sharing these tools with other people that may find them useful and can improve these things.

For metadata (WB and a ton of other settings), just read the MLV thread, this is work in progress. Synced audio is pretty hard (but not impossible), and I don't have any use for it myself, for example. On the other hand, dual ISO is something I'd use in over 80% of my pics (since I like to shoot in very tricky lighting conditions). I don't shoot professionally, but I shoot some events for friends every now and then (not paid work, just hobby level, and mostly photos and sometimes timelapse).

Anyway. For me, the end goal is not to provide a finished product that everybody can consume. I'd rather see it as an open software platform where others can program their own enhancements and share them with the community (which actually happens to some degree). I prefer it to be a little on the nerdy side and attract knowledgeable people and enthusiasts; after all, ML is a tool for power users.

(and while we are at it, this reminds me I should revive the scripting support)

I dont know if you are start to doing this for fun but you did a great work for the cinematography world. And I was wondering if ever Canon gave you a generous donate to the team of ML guys. A big part of their sales is due to you guys. There is no other dslr and/or mirrorless capable to record video with the 180 degree rule of shutter speed except of a new panasonic.

Well done, revolutionary greetings from Greece, and keep up the good work for the MK IV, 80D etc

[Walter Schulz]

Canon never officially acknowledged ML exists. No contacts/relations of any kind, either.

[rexorcine]

Canon never officially acknowledged ML exists. No contacts/relations of any kind, either.

Fantastic.

They're the Leanburn avoiders of the camera world. We could have cars that do 400 miles to the gallon now, but we don't because there are a gaggle of arseholes making a fat load of cash out of our stupidity.

[Audionut]

but we don't because there are a gaggle of arseholes making a fat load of cash out of our stupidity.

Before you attack a legitimate business, you may want to focus your off-topic attention onto flat-earthers, anti-vaxers, wellness centers, non-gmo products, superfoods, organics and the like.

We respect Canon, because without them, this development project would be nothing.