Messages

I was able to narrow it down to Raw Magic 1.2.2. (MacbookPro Retina mid 2012 and MacPro late 2013)

Patrick, this should obviously not be happening. If you wouldn't mind contacting me directly, I'd like your help to get this resolved.

If you can send me the DNG frames that are corrupted, I'm more than happy to look at them.

By the way, Rarevision now has a support forum for RAWMagic: http://rarevision.com/forum. I'm not sure the ML devs want RAWMagic discussed on their forum, so it's probably better if you post there instead.

You're correct to use the "BMD Film" setting, but the problem is that it assumes output from a Blackmagic Camera. From what I understand, these cameras use some type of log curve to pack their data into 12 bits efficiently. The output from RAWMagic doesn't use a log curve. So right away, by using BMD Film with Magic Lantern footage, incorrect assumptions are being made by Resolve because I'm fairly certain the RAW output from Canon DSLRs is, for all intents and purposes, linear (Alex can perhaps confirm this). That pretty much invalidates any LUT, if accuracy is a factor. Regardless, this is the only setting in Resolve that renders CinemaDNG files from RAWMagic properly without discarding a bunch of picture information.

Now, that doesn't mean you can't select any random LUT and get something that looks okay. People do that all the time. It's just not "correct" in that it is not properly transforming log data into linear data as designed. And since it's not correct, then why bother with a LUT at all? Why not just grade the footage yourself to get the look you want? That's basically all that's happening anyway when you cycle through the LUTs in Resolve. At that point, you're just doing the equivalent of Magic Bullet or Instagram or any photo filter app. It may look cool, but there's no regard for technical accuracy. That's why I rarely use them.

I'd just pick a LUT you like, and stick with it. You can try downloading some LUTs designed for other cameras and see if they give you the look you want. Unfortunately, there's no right way to do this yet, since as far as I know there's no specific setting in Resolve for Magic Lantern footage.

[RAWMagic is currently in full GPL compliance.]

I've read enough libelous posts here that it's about time someone responded with facts instead of hearsay.

RAWMagic is currently in full GPL compliance. I asked, begged, pleaded with you to release the vertical stripes correction code as LGPL because I wanted to link directly to the code from RAWMagic for performance reasons, but you refused. I therefore had to implement the VS code as a separate binary, which hurts performance but is at least GPL-compliant.

It is a simple matter to read the RAWMagic release notes or inspect the software on a Mac to know this. So, I really don't know what Alex is talking about when he says "ML development is stopped because of this issue." Since RAWMagic is GPL-compliant, I fail to see how this is an issue at all.

Sadly, as a response to my good faith gesture I've been called a thief, ignorant, a liar, greedy, a black sheep and other mean-spirited things. You've also discussed petitioning Apple to remove RAWMagic from the App Store. Come on, guys. How in the world would you justify a nasty, vindictive move like that that to your users?

All I tried to do with RAWMagic was give the community a tool many of its users were asking for. Nobody else seemed interested in meeting these users' needs. Yes, I charge what I think is a fair price for my time. I even asked the forum before adding MLV support if people would be willing to pay for such an app, and the response was YES. So it is your own community that requested a paid piece of software!

Go ahead an delete the RAWMagic thread if you think that's good for the ML community. If you want to ban me, well, I guess that's OK too. I probably won't post here anymore anyway because of the really, really negative energy. You indeed do good work, but the negativity and name-calling is really not something I want RAWMagic users exposed to.

As far as I'm concerned, Thomas Worth has ignored all reasonable requests by the development team, and now expects the development team to assume his application abides by the GPL, based solely on his word.  Support for his applications should be immediately ceased from these forums.

I have not ignored any requests. In fact, I went to great lengths to try to appease the ML team. Anyone who reads this forum can figure that out.

It seems there's nothing I can do to make you guys happy. So, if you think it's good for the community to delete the RAWMagic thread, I guess I can't stop you.

How do you think your users would feel about that? Perhaps you should take a poll before making a decision.

Therefore, I suggest enforcing GPL compliance on all the postprocessing apps based on ML command-line tools.

As stated previously, RAWMagic is not based on any ML command line tools, nor does it require any GPL code (either linked to the application or calling a separate binary) from the Magic Lantern project to operate. Its core functionality comes from 100% proprietary code, just to be clear. It is not a "derivative work." Far from it.

RAWMagic is certainly not a wrapper over a GPL program, although some, if not most, of the other post-processing apps on this forum are.

Only for vertical stripes correction, and only to stay consistent with other ML tools. So, only four GPL functions which are related to stripes correction.

I'm not sure what you're implying. Are you saying you think RAWMagic's core conversion engine is based on your code, e.g. raw2dng?

It relies on my GPL code to do its basic functionality.

If one converts RAW or MLV files with RAWMagic that don't suffer from the vertical stripes issue, no GPL code is employed whatsoever (external binary or otherwise) since vertical stripes correction isn't needed. So no, basic functionaity does not rely on GPL code. I assume that's the code you are referring to.

I was wondering, since Premiere Pro CC 2014 now supports 5D3 Raw video, what is the most efficient workflow for editing 5D Mark III footage in PP?

I just tried to drag the raw files from the 5D into premiere pro and that didn't work. Also when I converted the footage using RawMagic, but all of the footage appears green.

RAWMagic (paid version) should not be producing green CinemaDNG files. I spent a lot of time making sure the CinemaDNG files from RAWMagic render properly in Premiere CC.

Post a CinemaDNG file from the sequence that is giving you trouble and I'll have a look at it.

Here's how it would normally be done:

+ Convert RAW/MLV clips to CinemaDNG with RAWMagic
+ Convert GoPro footage to ProRes with 5DtoRGB or edit it natively in Premiere
+ Import footage into Premiere Pro
+ Edit DNG/ProRes/MP4 natively in Premiere
+ After edit is locked, export OMF of sound for sound mixer
+ Export XML of edit sequence to Resolve for color correction
+ Color correct in Resolve while sound is being mixed
+ Render color corrected media, export new XML from Resolve, import back into Premiere (the term for this is "roundtrip")
+ Back in Premiere, marry completed sound mix to picture, add titles, credits, etc.
+ Export master from Premiere to ProRes MOV, DPX, etc.
+ Create deliverables from master, from H.264 versions for the web all way up to a DCP.

Don't color correct before you edit. You need to color correct after editing so you can make all your shots match.

Which color grading program should I learn?

Resolve. Don't waste time thinking about this. Just go download it now and start learning it.

Instead of copying the source or object code into an app (like Thomas Worth's)

RAWMagic does not contain any GPL code. This is a misconception.

One of the benefits of GPL is that it protects the customer.  Suppose I became dependent on Thomas's tool, but then the GPL code were to fix a bug that I found desirable (or even if MLV format changed), all I would need to do is build a new shared library, swap out the one that RawMagic uses with that one, and as long as the code syntax and semantics hadn't changed, I'd be good to go.

You can indeed do that now.

I'm not a lawyer, but isn't this shared library approach permissible in GPLv2?

From what I understand, no. The code is still executed within the main binary, so this doesn't meet the requirements. A separate binary could be implemented, itself compiled from GPL code, which is what I've done with RAWMagic. This separate binary must itself be open source, of course.

LGPL would allow the shared library approach, which would give better performance and streamline development of commercial apps. I proposed the LGPL idea to the Magic Lantern developers, but they aren't interested.

I have complied with the wishes of the developers and have dealt with any GPL code in a way that satisfies the GPL. This is what I said I was going to do, after going to great lengths to assure the ML community I would follow the rules.

I refer you to this:

http://www.magiclantern.fm/forum/index.php?topic=11879.msg115399#msg115399

Yes, I'm fine with commercial programs that are also free software, working with ML file formats and being supported here on this forum.

I'm also fine with proprietary software (aka closed source) supporting ML file formats, but if they want to make a profit, that should come from their own code, not from a closed-source modified version of my code.

I'm not really sure why this is an issue anymore.

the simplest solution is of course corrective action on the part of RareVision

Already done.

[done out of respect for the ML developers.]

RawMagic seems not to be a good choice now because it doesn't support all a1ex and other coders' work.

There is no code in RAWMagic written by anyone else but me, save of course for the usual C and Objective-C libraries that you're forced to use if working with the Mac OS X API. Vertical stripes correction is implemented, but RAWMagic uses an external binary (which itself is GPL'd code) to apply it. This causes a performance hit, but is necessary to maintain compliance with the GPL and frankly, done out of respect for the ML developers.

I am a filmmaker, own multiple Canon DSLRs and use RAWMagic on my own projects. I know what is needed from the software to make it just plain work. If a feature is missing that makes the workflow impractical (based on real world testing), I add it. I need the software to work in a professional setting, as do other professionals. Filmmakers up against deadlines don't want to mess around with "entering DOS" into a Terminal.

Just thought I'd clear that all up.

I disagree, ACR is the only way to get the best and most acute  results .

This isn't necessarily true. I'm fairly sure ACR can't account for the curve baked into footage from Blackmagic cameras, for example. So results from ACR on Blackmagic footage won't be as accurate as Resolve.

Resolve still can't debayer ML Raw right at best its a big compromise ,

Can you cite a specific example where Resolve can't debayer Canon/ML footage properly? I'd like to see this if it is indeed a problem. I haven't had a problem. I believe Midphase uses Resolve, so perhaps he can chime in.

It you per-grade with ACR in A.E. I find the workflow is faster in the end.
I done the resolve workflow & find it not faster just more convenient and you still
have to spend the time to grade ,where as per-grading . Bulk of you grading work is already done.

reddeercity, with respect it sounds like you're not as up to speed as you probably should be with Resolve. Any professional colorist will tell you Resolve is one of the best tools for finishing. The only people I know that would even suggest color correcting in After Effects aren't colorists and don't know Resolve. I'm not saying you're one of them, I'm just saying in my experience those are the types that recommend an AE workflow. Furthermore, I have worked as both a visual effects and post production supervisor on feature films, and know both pieces of software well. I can say, without hesitation, that After Effects is the wrong tool for color correction and finishing. It is excellent for motion graphics and visual effects work, and I use it extensively for both. I wouldn't consider using it to color correct anything outside of matching elements in VFX compositions. Yes, technically you can use it for color correction. But then you can also write C programming code in Microsoft Word.

Let face it we are working with raw with photographic color space not video, so you must treat it as such and prepare it properly
to enter in the Video color space. There will always be short cuts to quality but can you live with the results , personally  I can't .
Just my two cents .

You should choose a workflow that suits your needs, but it's a bit shocking that anyone who knows both Resolve and After Effects would prefer After Effects. I highly recommend you spend some more time with Resolve. I think once you get more accustomed to it, you'll find it's much better.

In the long run, ACR is not a good solution.

Midphase is right. Don't waste your time with ACR. It uses an extremely slow, CPU-based high quality demosaic algorithm that does not work well with moving images. Besides, After Effects is notoriously slow at everything. While it is a fine app for compositing and VFX, you pay for its excellent flexibility by giving up performance.

Resolve, on the other hand, uses a GPU-based renderer, which is designed specifically for realtime demosaicing/debayering and playback. The "BMD Film" setting in Resolve will also give you the "washed out" RAW look you want.

Sebastian, I'm working on adding some new functionality to RAWMagic you may find useful. Feel free to contact me if you're interested in chatting. By the way, are you in L.A.?

So in order to bypass long rendering times I can edit directly dng sequences and render only the final cut. Premiere CC only on a computer with dedicated GPU?

Yes, and the media on a disk array or an SSD, ideally.

Very HDD space demanding at the moment if you are working on multiple projects and probably not convenient for me. Shame encoding of these sequences still takes so much time.

I know what you mean. If you do get around to encoding them to ProRes, it'll open up some options for editing and other work since the computer has a much easier time dealing with ProRes footage versus DNG sequences.

So from reading this thread, Premiere (not AE) has GPU-supported encoding and should thus run faster?

Premiere has a GPU-accelerated debayer engine. This will let you play DNG files back at normal-ish frame rates. What do you mean by "encoding?" Do you mean saving to a different codec/file? All of that is typically not helped by the GPU.

Keep in mind that GPU stands for Graphics Processing Unit. The key word being "graphics." These processors accelerate math operations that are commonly used with graphics. Data compression is something else.

I'm not sure about CC, but as far CS6 goes no media decoding is handled by Mercury Engine, not even h264 or AVCHD where cards contain hardware accelerators for such tasks

Adobe don't maintain their own H.264 decoder. They use MainConcept's under license, which is a shared library and runs on the CPU. "GPU," by the way, isn't a magic word. From what I understand, some types of math are just not a good candidate for GPU processing. The math used in codecs like H.264 and REDCODE (JPEG 2000) are examples. That's why special hardware DSPs have been needed to accelerate these codecs. As an example, a typical smartphone SOC has a GPU and a hardware H.264 decoder. Why not just use the GPU? Doesn't work that way, apparently.

As chmee explained, you're getting 1 fps because ACR (which renders DNG inside After Effects) is not accelerated and runs on the CPU. It's very high quality, and not designed for video playback. Remember, it was designed originally for still photography.

That said, an upgraded GPU will do exactly diddly squat. If you want to speed things up, use software with a GPU-accelerated debayer engine like Davinci Resolve. Or the latest Premiere, if that even counts.

Or render your DNG files to ProRes 4444 first, then composite in AE.

I'm going to add the ability to override the black level in the RAW/MLV file with a user-selectable one just in case the black level in the files are incorrect. I know different cameras use different black levels. Right now, the options are:

Autodetect - uses the black level in the RAW/MLV file
2048 - forces the black level to 2048
1024 - forces the black level to 1024

What other values should I use? Are there cameras that use black level values other than these? I only have access to a 5D Mk III and a 7D, so I haven't been able to test with all the Canons.

The DNG looks OK to me. I sent you a link to what came out of Photoshop (Camera RAW).

Send me or upload a DNG file that shows the problem. What version of RAWMagic are you using?

Tried it again, and I had slightly better results with the SD card. There was still corruption at 40 fps, just not as much.

Is this due to 40 fps approaching the limit of the camera's internal bandwidth or something like that? That would make sense since 40 fps sort of works (but not really), but 39 does seem to work pretty consistently. Perhaps 39 is just under this corruption threshold.

Was 40 fps the limit you picked based on your testing? If so, does that imply you could have increased the frame rate beyond 40, but limited it to 40 due to this bandwidth issue?

So, while the sensor and LiveView tasks can go to 40 fps, the bottleneck seems to be elsewhere (even the extra load on memory bus caused by fast CF writes can be too much).

I tried with a fast SD card (I originally tried with CF), but got the exact same result. It's only at 40.000 fps the corruption shows up. And it shows up in the first frame or two. It's constant corruption throughout the entire video, not just at the end because recording stopped due to slow media. Every other frame rate (less than 40) is fine. Can you confirm that you were actually able to capture at 40 fps without corruption, whether to SD or CF?