Messages

3. The newbies probably pounded him with questions and dumb questions to boot. He got fed up and decided to make some money off of it, understandable.

That's not the motivation.

EOSHD is free, I work on it all year and get pounded by questions every day. I make money because I have to. If I didn't, it just wouldn't be viable.

[coexist]

"This guide and contributions from other video related blogs fill a gap and make this amazing technology accessible for a broader public"

Cheers for the concerns from certain individuals here, even the troublemakers who are already known to me (not you Bart!).

I really do appreciate the spirit and ethos of an open source project and didn't want to go against that.

I wanted with the book to take Magic Lantern to the less-technically minded parts of the film industry and it is a showcase for the apps too, bringing them to an audience outside the forums. When RawMagic goes gold, if it becomes a paid app like 5DToRGB will the same people complain to Rarevision that they're simply profiteering off someone else's work? I hope not. They work bloody hard to give you such a useful tool.

EOSHD isn't an open source collaborative effort but I do put a lot of stuff out there for free because I believe in free exchange of information. I also believe capitalism and free exchange of info can coexist. In fact it goes deeper than merely coexisting. They compliment each other. I am not alone in requiring funds to survive. Magic Lantern needs funds to sustain it and day jobs and donations. The Shooter's Guides are my day job and I think it is unreasonable to expect me to do my day job for free AND provide an enormous blog about cameras for free also.

An anamorphic guide, a GH2 guide or a 5D Mark III guide is a donation to EOSHD. You get something in return - the book and my hard researched knowledge that I hope will be useful. I don't see a problem with it.

I put huge unpaid effort into the site not to profiteer from a community but to help build one.

I enjoy doing EOSHD and I'm passionate about filmmaking. If I was such a profiteer you'd think it would be plastered with ads.

I get requests and bat them off because I don't want ads to ruin the experience for the reader for the sake of an extra few pennies. That isn't just talk, it's action - show me the ads! Profiteering? Rare paid content goes into sustaining that service and if I didn't have the books - there'd be no EOSHD! End of story! Not even an Andrew Reid, no filmmaking. I'd be unemployed. I sometimes think that is what some people actually want, to see me fall flat on my face as a filmmaker? Completely unable to fund my creative work?

If not for these books I'd be shooting uninspired commercials for corporate clients instead of attempting to help the broader film community get to grips with Magic Lantern and anamorphic lenses. The book is 100% my own work. It didn't exist before I wrote it and of course wouldn't exist if not for Magic Lantern existing, but then everything sits on the shoulders of giants. No Dennis Ritchie. No C code. I am doing a brain dump onto the page of everything I did to familiarise myself with raw video, not just with Magic Lantern and the 5D Mark III but with Resolve and the Blackmagic cameras, and also my own cinematography with two chapters of advice on the creative side.

Here's the overexposed DNG John.

http://we.tl/31Mb24MVqb

Cheers

[128GB]

128GB

Your 128GB results are invalid as you ran the benchmark in playback mode. Run it in movie mode with live view enabled to get an accurate impression of how it will perform during raw recording. You will find it is slow - around 70MB/s, pretty much all the 128GB KomputerBay cards are. The 64GB cards are fastest.

John, great work.

At 1/2 res on the timeline I get fluid playback on my Mac Pro with CUDA accelerated Mercury engine, GTX 560 Ti in Premiere CS6.

At 1/2 res the magenta highlights disappear but they are there on the full res debayer.

There's something else wrong too as I can't get the same latitude as I can on the DNGs.

The shot above was overexposed and I was able to bring it down fine in ACR. When I do the same in Ginger with the Manual Exposure setting, or any of the colour grading controls under Filmic Curve I get clipped highlights as if a lot of the dynamic range just isn't accessible.

I also think the UI needs an overhaul for this to become a really popular tool. I think it should show the full metadata, and have graphical dials and sliders for kelvin, ISO, exposure, tint, shadows, highlight recovery just like Adobe. I can't get a 'feel' for my image as I change it by having to type the digits in, and the sliders are way too sensitive and not responsive enough in the rate at which they change the image in the monitor.

One other point - although I get a clip preview in the project manager thumbnail gallery, nothing happens when I double click the raw file to bring it up in the source clip monitor. It works fine on the timeline but not in the source clip viewer.

This is all on Premiere CS6 under OSX Mountain Lion.

Cheers!

I did wonder how the Dugdale Black Edition worked

I'm interested Alex, what is the technical reason for the live-view overhead on write speeds? Is it CPU or memory related? Do you think there's still more optimisation possible to get the 120MB/s people achieving with live-view off?

Guys is the pink fringing the same as what I reported to chmee? Here's how I see it...



The standard Raw2DNG frames don't have this. What is that code doing differently?

I think that is because 444 needs Go Pro CF Studio Premium and it can't find it installed, only the free version - but it doesn't give an error, it just carries on and doesn't produce a file at the end.

Can the pink highlights be fixed?

Here's an example from my most recent transcode...



Update - Resolve doesn't have the pink highlights problem. Only Premiere and Quicktime Player.

I just tested to changed .avi to .mov and I get the file generated as it should. And it opens fine in Vegas, don't have Resolve. Did you keep the quotation marks around the filename ("{OutputVideoName}.mov")? They are needed if file path contains spaces. I cannot reproduce your problem right now.

There is no big reason for me to make a clickable interface to building the parameters, at least not now when I don't know which parameters are "universally" useful to all. Is there a parameter setting that everybody uses?

I do not have anything to do with the installation of cineform and the tool (I have not license to do it), so I cannot control where things end up. RAWanizer tries to guess where it is, but if that fails you can manually set the location of the cineform tool by clicking the 'folder settings' button.

I decided to use .net 4.5 so I could use the new threading options there. I currently do not have time to go over and rewrite the threading with other tools so for now it will require .net 4.5. Sorry.

I understand the confusing thing with cineform and  the video option. I will disable that option for cineform. As described in original post, that option is for making a videofile out of tiffs which has no meaning in cineform workflow. Cineform = Proxy video.

Fixed the MOV problem - my copy of Windows in VMWare didn't have Quicktime Player installed. CineForm needs that in order to do the Quicktime wrapper during transcode. So not a problem with RAWanizer

Great tool, looking forward to see how it develops!

BTW - maybe someone can answer this (I have already told David Newman) - on my Mac the CineForm RAW or 422 gives a flat LOG-like curve when you open the clip in Resolve, Premiere or Quicktime Player, but 444 gives punchier colour and contrast like a DNG in Adobe Camera Raw. Metadata issue?

Some feedback from my test of RAWanizer and RAW2GPCF:

I can create CineForm AVI videos no problem, but Resolve needs MOV. At the command line with RAW2GPCF I can get MOV but not in RAWanizer when I change .avi to .mov in the CineForm parameters box - although RAW2GPCF runs it doesn't produce a file and the RAWanizer batch ends with 'aborted by user'. Any solution?

Rather than just a CineForm parameters text box can it be radio buttons in future for less advanced users?

CineForm Decoder and GoPro Studio give us two Tools directories so make sure RAW2GPCF is in the right one. With only CineForm Decoder installed and the RAW2GPCF.exe placed in the CineForm Tools directory, clips could not be produced. Does RAW2GPCF work with just the CineForm Decoder installed or does it require GoPro Studio?

I'm running Windows 7 in VMWare Fusion within OSX. Tried it in XP - doesn't work, and .NET 4.5 only runs in 7. Would be helpful if the app could be made to work in XP SP3 for those not running 7.

I cannot make a CineForm clip with the Video option selected, only Proxy. I get the TIFF error. This is confusing, can it be changed?

My research shows the OP was correct, the problem is -

(+) IFD0-Entry is the jpg-Preview in 128x96px 8Bit - thats the one you see in Resolve.
(+) Raw-Data are packed into a subIFD - as in the DNG-Definition described.

Stills DNG has support for multiple images in one DNG at different resolutions (subfiles) and Cinema DNG only has one 'primary-image'. The raw data in the stills DNG is in a subfile, and seemingly the primary file is a thumbnail. Resolve reads the thumbnail data in IFD0-Entry but cannot access the raw data packed into a subIFD since it lacks support for this file structure.

Could we change Raw2DNG so subIFD becomes ifd0? I.e. pack the raw data into IFD0 instead of SUBIFD? Is that a trivial metadata change or more complex? There is a DNG SDK and Cinema DNG docs, someone just needs to understand the file format and write the correct structure into Raw2DNG's code.

I asked a Blackmagic guy via John Brawley about Resolve supporting stills DNG, and Pete confirmed it doesn't yet support DNG only Cinema DNG. I am sure they are working on it but probably quicker to change Raw2DNG.

This thread at EXIFTool was useful in pinpointing the problem - http://u88.n24.queensu.ca/exiftool/forum/index.php?topic=4927.0

Here's how the ML DNG metadata differs to Cinema DNG (Blackmagic Cinema Camera)

http://www.eoshd.com/uploads/dng-metadata.png
(Blackmagic Cinema Camera EXIF on the left, Canon raw DNG on the right)

As you can see stills standard DNG EXIF is very different to Cinema DNG.

DaVinci Resolve seems to read the reduced-resolution metadata for 5D Mark III footage.

There seems to be other errors like JPEG compression for the full resolution image and uncompressed for the reduced resolution image?

Highest reliable res on the 128GB card is 1920x840, around 65MB/s.

1920x960 isn't too bad... but right on the limit of the card at 73MB/s. Sometimes it can't even do that.

Still the 64GB card pushes on for 92MB/s peak and can do 1080p no sweat.

Sometimes my 128GB card is slow off the marks - 50MB/s, then slowly rises to reach 70MB/s after around 10 seconds of recording.

In benchmark it always achieves around 73MB/s no matter what mood it is in 

Is it due to fragmentation?

With latest build I had strange issue where on my faster 64GB card it dropped back to 70MB/s and dropped frames at the start of the recording, then gradually sped up around the 1GB mark during recording, until it was back at 80MB/s+ again.

Rebooted camera, tried 1920x1280. 92MB/s sustained to the card. Under the magic 98MB/s really needed for that resolution but still - why such variation in card writing speeds going on here?

A little update regarding the KomputerBay cards... You may want to avoid recommending the 128GB version of the KomputerBay CF card. Evidence is building that the 128 version had slower real world write speeds than the 64. This prevents 1080p raw recording since the 128 doesn't seem to attain the 83MB/s minimum write required for 1080p raw recording. The 64GB version seems to test at 90+ MB/s write while the 128GB version tests at 60-70 MB/s write.  It turns out that Andrew from EOSHD was using the 64gb variety of the KomputerBay card for his Raw tests, which is why he was able to utilize even greater than 1080p raw without dropped frames.  This is all still unproven but I just returned my 128gb and ordered two 64gb to see for myself.

73MB/s on my 128GB card (best case).
95MB/s on my 64GB card peak sustained rate in ML benchmark - 85MB/s reliable during raw recording.

Barely 20MB/s on my fastest SD cards. They are: Sandisk SDHC-I 95MB/s (16GB) and Sony 94MB/s (32GB). They are unusable for raw, at least for me. Had expected better from those.

I am going to try painting go faster stripes on my 128GB card and see if it improves

Yep, if they could be compressed to JPEG all problems would be solved.

Is it a code problem or a CPU time issue? Is debayering and compressing to JPEG at 24fps just not possible?

Looks like the Transcend 1000x is faster than the Lexar. http://www.fredmiranda.com/forum/topic/1202766 They're cheaper too, B&H are selling the 64Gb for $200.

I've had good results with 1000x KomputerBay, even cheaper than Transcend. Was flawless in 1D C at 4K MJPEG, and that was 100MB/s to the card. Not a single dropped frame or buffer full alert. The memory in these cards probably all comes from the same manufacturer anyway!