Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Topics - Andy600

Pages: [1] 2 3
1
There are not many substitutes for having a proper white balance reference when shooting (white balance card, Colorchecker etc)  but here is one that will help you get accurate white balance and it's very cheap.

Polytetrafluoroethylene or PTFE - Teflon.

PTFE and is usually found in a plumbers toolkit. It is a white, stretchy, non-stick tape used for sealing pipes but it does have another very useful property. PTFE is an almost perfect (~99%) Lambertian reflector i.e. it's ideal for providing an accurate white balance reference.

You can pick up a roll of PTFE tape at most hardware stores for a couple of dollars/euros and make your own white balance reference card by wrapping the tape around some cardboard. You need to wrap several layers as the tape is a little opaque but a small roll should give you a portable 6"x4" card that you can keep in your camera bag. It's not as robust as a commercial reference card but it's cheap and is far more accurate than a lot of the cheaper white balance cards that you can find on eBay.

Lambertian Reflectance https://en.wikipedia.org/wiki/Lambertian_reflectance

2
This is a long post (yawn) but most of it is summed up in the first couple of paragraphs. In the first reply to this post is a simple exercise for any willing participants to provide normalized matrices using DNG metadata and Microsoft Excel.

In an effort to achieve a basic, uniform rendering of color from Magic Lantern Raw Video across many different applications there is first a need to standardize color related metadata in the DNG and CinemaDNG files output by Magic Lantern raw2dng apps.

Presently, there are multiple raw2dng apps that provide users with DNG or CinemaDNG files from original Magic Lantern Raw (.raw) and 2nd gen Magic Lantern Raw Video (.mlv) container formats with each app embedding color related metadata (matrices) according to whatever level of meta support the app developer chooses to include. The variations in default color rendering between post production apps proves the need to begin standardizing this important part of Magic Lantern Raw Video.

Enhanced color and look management using embedded DNG camera profiles and/or .ICC camera profiles would be an obvious next step and the (remote) possibility of any future ACES support for MLV will hinge on this small but important change to metadata.


Why?

It is important to understand the differences between DNG and CinemaDNG standards in relation to color in raw video workflows. CinemaDNG is a subset of the DNG Standard (both developed by Adobe Labs). CinemaDNG has several addition video related meta tags (timecode, shooting information, reel numbers etc) however some recommended DNG specification tags are omitted in CinemaDNG. While both standards 'can' be used/mixed it leads to actual color appearance being rendered differently depending on the particular standard that the raw reader (i.e. your NLE or color grading app) is based upon.


To simplify things we should look at the most basic purpose of .raw and .mlv and that is to produce moving images.
 
DNG standard is intended for stills photography with raw photography apps  like Adobe Camera Raw, Photoshop, Lightroom, RawTherapee etc making use of any/all DNG standard meta content.
 
CinemaDNG standard is for raw video. Color grading, compositing, DI and NLE apps (Resolve, Premier Pro, Scratch, Nuke etc) are not required to parse certain DNG tags (i.e. they can and will ignore some tags even if they are included in metadata) and therefore, as .raw and .mlv is primarily intended for video production and editing with video-centric post production software, magic Lantern Raw Video should ideally be conforming to CinemaDNG standard. Some raw2dng conversion apps already do this to a lesser or greater extent.


What this means in relation to color:

The key color related difference between these 2 standards is the inclusion or omission of Forward Matrices. A forward matrix (part of DNG standard but not part of CinemaDNG standard) acts as an additional raw level color calibration function. A forward matrix maps normalized sensor RGB values (after white balancing) to a connecting colorspace or PCS. In this case the PCS is CIE XYZ with its D50 white point - PCS is used when transforming from one RGB colorspace to another i.e. to translate your DSLR sensor's wide gamut primaries to fit a much smaller display colorspace like sRGB or Rec709.

The color matrix tags which are included in both standards map a sensors non-white balanced RGB values and white point to PCS using a simple 3x3 least squares matrix. In CinemaDNG, the color matrices perform both the white balance and initial color calibration however, color is only a secondary, fallback function and will usually require additional color calibration after debayering to a colorspace.

Presently there are 2 ML supported raw2dng apps that embed 2 sets of color matrices and 2 sets of forward matrices per camera. These apps are raw2cdng (for Windows) and MLVFS (for OSX but can be configured for use under Windows).

The CinemaDNG files produced by these apps will tend to produce a 'nicer look' by default (similar to Adobe Standard color) but only when the files are debayered in an application that is built to honor the DNG standard and/or forward matrix tags. However, most video-centric color grading applications (Resolve, Premier Pro's native DNG support, Nuke etc) do not support forward matrices i.e. the forward matrices are ignored -  only the color matrices will be used. Resolve may have supported forward matrices in the past but v12 does not.

CinemaDNG standard does not support embedded 'look management' - stating that this part should be deferred to post production to enable a very basic, low level default rendering of the color across multiple apps and workstations.

So what needs to change?

The first change is very simple and is dictated by both DNG and CinemaDNG standards - We need DNG files with normalized primary matrices.
 
As CinemaDNG is the more relevant standard for MLV to follow, and because CinemaDNG does not support forward matrices, the color matrices used should be normalized (hard-coded normalized). The idea behind this is where a pixel value is recorded as white it should remain white (or neutral) throughout the transformation process to a display space. The normalization of color matrices in photographic related apps like Adobe Camera Raw is typically an automated function i.e. the color matrix will always be scaled at input regardless of the hard-coded scale of the matrices in metadata. The forward matrices are always normalized and so the final output will always be normalized and scaled correctly ready for the PCS to transform to a display referred or scene referred colorspace.

In applications that are coded to support the CinemaDNG standard and using only the current color matrix sets in MLV (which are not normalized), this auto-scaling does not typically happen. The debayered primaries are therefore not correctly scaled/optimized for PCS mapping or translation to a colorspace and this may lead to minor inaccuracies in white balancing and/or color rendering.

The most likely issue is with default saturation levels being slightly higher than expected but there may also be other, more significant issues where the same white balance multipliers (color temp and tint offsets) may behave differently across different apps. This part is somewhat theoretical but obvious differences can be seen when comparing 2 copies of the same DNG (one containing normalized matrices and the other containing non-normalized color matrices) and then analyzing the resulting trace on the Vectorscope in DaVinci Resolve.

Possible issues with normalized color matrices:

Although I am not posting the full normalized matrix set here (see the first reply of this post for the reason why) I have checked most normalized matrices using DNG_Validate.exe (v1.4 found in the Adobe DNG SDK) and they should be ok for use as-is (without forward matrices) but it is understandable that developers may wish to continue with the inclusion of forward matrices for raw processing apps that can make use of the extra metadata. I have also tested the normalized color matrices with forward matrices (using Exiftool) and these too validate using this method but I strongly recommend app developers test privately with their own apps before release as there may be validation issues when compiling (if using the Adobe SDK) that I have not personally experienced when changing meta with Exiftool.



My personal view is that MLV should follow CinemaDNG standard and omit forward matrices completely. This approach 'should' then produce the same default color rendering of Magic Lantern Raw Video across raw post production apps with any remaining difference then being purely down to an individual application's color processing i.e. debayer algorithms, noise reduction, chromatic adaptation model etc.
 
This change will not perform any color matching between different Canon DSLR models. The differences due to color sensitivity and color reproduction between say a 5D mark III vs an EOS-M will remain. Color matching cameras is a whole other subject and in CinemaDNG is deferred to post production. Conversely we are not creating additional problems or imposing a look ahead of any such color matching or look application i.e. this proposal gives the user the most flexibility in post without anything degrading the image color at its most basic raw level.

If raw2dng developers agree with this proposal and want to continue with the inclusion of forward matrices (i.e. mix DNG/CinemaDNG standards) I would suggest making it an option, not a default and perhaps link the option to this post or inform users of the difference in color they may experience. I will add further content to this thread including some simple color comparisons for users who are less technically oriented or who may not speak English.




Disclaimer: I (Andy600) and my company (CinelogDCP) produce commercial color management products that can be used with DNG footage however the changes I propose here are not related to CinelogDCP and are intended to benefit the wider ML community regardless of whether or not CinelogDCP or other third party products are used.

3
General Chat / DaVinci Resolve 12 beta is now available
« on: July 27, 2015, 01:09:13 PM »
The latest and greatest DaVinci Resolve (v12) has been released in beta. I think this now rivals Premier Pro and FCP in it's NLE capabilities and with Resolve's color grading toolset, it's probably all you'll need for editing/color work - if you have the power to run it!

https://www.blackmagicdesign.com/products/davinciresolve


4
FYI - Blackmagic Designs has just released Resolve 11.3.1 (full and free lite versions).

This is good news for Windows users who now get DNxHR codec support in Quicktime wrapper (previously it was limited to .MXF).



What's new in DaVinci Resolve 11.3.1

    Support for DNxHR in Quicktime wrapper
    Updated color science for Alexa 65 RAW processing
    ALEXA Open Gate and Canon RMF clip resolution improvements
    Support for CUDA 7 and NVIDIA Maxwell GPUs
    Anti-aliasing improvements for title rendering with shadows
    General performance and stability improvements

Minimum system requirements for Mac

    Mac OS X 10.8.5 Mountain Lion
    12 GB of system memory is recommended and 8 GB is the minimum supported
    Blackmagic Design Desktop Video version 10.1.1 or later
    CUDA Driver version 6.5.45
    NVIDIA Driver version – As required by your GPU
    RED Rocket-X Driver 2.1.19.0 and Firmware 1.3.19.7 or later
    RED Rocket Driver 1.4.36.0 and Firmware 1.1.17.3 or later

Minimum system requirements for Windows

    Windows 7 Pro 64 bit with SP1
    12 GB of system memory is recommended and 8 GB is the minimum supported
    Blackmagic Design Desktop Video version 10.1.1 or later

https://www.blackmagicdesign.com/support

5
General Chat / New Blackmagic Products
« on: April 13, 2015, 06:47:56 PM »
So BM have done it again!

Not content with taking on the big camera manufacturers, they are now gunning for the likes of Atmos with https://www.blackmagicdesign.com/products/blackmagicvideoassist

The Micro Studio Camera 4K and new micro Cinema Camera are impressive and will likely impact the GH4, A7s etc. Paired with the Video Assist and you have a relatively cheap (ok, dirt cheap) 4K setup.

Even for me, a Canon fan, I'm starting to think the big C are just gonna leave the prosumer level kit to the likes of BM. It's a big shame as far as Canon goes but exciting times are ahead for low budget shooters.

6
General Chat / Alexa Mini - oh my!
« on: February 23, 2015, 05:23:57 PM »
No digital camera has ever made me drool - until now http://www.arri.com/alexamini/

One thing's for sure - this will not be cheap so I'm going to stop dreaming right away  ;D.

7
Blackmagic Design have just released DaVinci Resolve 11.2 with support for Avid's DNxHR codecs. This is great for Windows users as we now have something closer to ProRes (including 4444 XQ) with up to 4K output.

The Cinema DNG Softclip option may be useful for clipped footage but I haven't tried it much. I'm not sure about the 'improved CinemaDNG tone curve' - I think they may have increased precision but need to experiment more.

https://www.blackmagicdesign.com/support

What's new in DaVinci Resolve 11.2


    CinemaDNG tone curve improvements
    CinemaDNG soft clip option
    DNxHR encode and decode support
    Media Composer 8.3 round-trip support
    AAF import now sets the timeline resolution
    Invalidate clip cache option in Edit and Color page
    Option to select RGB pixel order for DPX version 2.0
    Flags and markers now supported in ColorTrace
    Support for Red SDK 5.3
    General performance and stability improvements


8
Hi guys,

Accurate color reproduction and white balance can be a little tricky for some users when it comes to DNG processing in apps like Resolve and this is nearly always down to incorrect or minimal meta data in the DNG files. Adobe Camera Raw at least offers a fall-back to Adobe Standard and common picture styles IF the UniqueCameraModel tag is filled but nearly all other apps (Resolve etc) rely entirely on the embedded matrices.

I know some ML Raw apps do include 2 color matrices which give better default color but it would be much better to include all useable matrices. I believe most MLV apps use DCRaw data for the current matrix info but it is a little outdated and most color matrices have since been refined by Adobe Labs.

I have put together a sheet containing the current Adobe standard color matrix coefficients (as found in the latest Adobe DNG Converter) for all MLV capable cams. You may want to try embedding the tags into DNG with your apps. We have tried these on several shots from different cams and it certainly improves things in Resolve. At the very least it will ensure the correct green/magenta tint at default settings and allow use of the default color temp presets.

The sheet includes the correct Exif tag names, color matrices (row scan order) and illuminant tags that should be embedded when your apps extract the DNG files. Alternatively, someone may want to create a simple Exiftool batch script (or app) that can re-write/add these tags to DNG files - this will be useful for older DNGs.

You can find the list here: http://pastebin.com/ZGpUm6Nx - Updated 29/11/14 (please discard previous sheet as it contained incorrect forward matrices)


9
General Chat / Apertus Axiom Beta
« on: May 09, 2014, 01:02:28 AM »
This slipped under my radar but Apertus announced their Beta/crowd funding intentions and it looks like a great deal for early adopters to help develop the system. https://www.apertus.org/2014-NAB-Axiom-Beta

Ok, it's not Magic Lantern related (yet  ;D) but they have referenced Magic Lantern when describing the Axiom's open source credentials. Definitely worth a look and worthy of ML input.

Plus there's an indirect dig at (I presume) Blackmagic in the write up  :P

10
@a1ex - how easy would it be for raw2dng or raw_rec/mlv_rec to generate a virtual white balance frame?

Disclaimer: This may be nonsense because I'm half asleep and haven't properly thought this through  ???

This simple idea might help with adjusting WB in post (especially in apps like DaVinci Resolve) when something was shot without balancing to an 18% grey card.

If raw2dng can generate a simple neutral frame, but apply the white balance that was used in the image sequence currently being converted, it would give a perfect WB reference frame to balance to. I suspect it might be tougher to do in-camera with the raw/mlv module?

i.e. read DNG WB exif > generate virtual 18% grey card or even a virtual Macbeth chart > apply Exif WB and save along with the DNG sequence that is being converted. The generated image could be a simple white box or have WB and other meta data printed. Not sure about gamma but I guess 2.2 would be best if it was something simple like an 8bit Jpeg and linear 1.0 if it could be saved as a DNG file.

I have a virtual, Macbeth DNG image if it helps.


11
Peachpit are offering Alexis Van Hurkman's brilliant Color Correction Handbook (Second Edition) in eBook format for $19.99. This is a 'must have' for anyone interested in color correction/grading and is a real bargain for $19 (it's usually $60). I think the offer is valid for this week only. I'm not associated with this in any way and I'm recommending the book purely on merit. This is great for newbies but many professional colorists also praise it as a reference guide.

http://www.peachpit.com/deals/?WT.mc_id=2014_03_02_PP_eDOW_Color_Correction_Handbook

12
Feature Requests / ACES for Magic Lantern Raw Video
« on: February 22, 2014, 11:04:42 AM »
As the DNG files produced by Magic Lantern Raw Video are predominantly used for moving images I would like to propose that Magic Lantern adopt ACES (Academy Color Encoding System).

ACES is an rapidly becoming the standard color science for motion picture and VFX pipelines and is aimed to standardizing color reproduction across cameras, regardless of manufacturer. It also offers the benefits of full Dynamic Range reproduction in the widest possible color gamut and accurate conversion to any colorspace (with an ODT) without needing to re-grade.

Incorporating ACES into Magic Lantern Raw Video will mean some minor changes to color matrix calculations in .raw, .mlv and probably raw2dng but beyond that, users should not notice an difference in their current workflows. The big benefits will come when using ML Video in an ACES pipeline as color reproduction will become both predictable and standardized. To fully utilize ACES we will likely need Input Device Transforms (IDT) for the cameras but these are relatively simple and (I suspect) we can base them temporarily on current knowledge deemed from DNG files, Canon IDTs and maybe Canon .ICC profiles.

OpenEXR will not directly be needed but may be the best option for DNG app devs who want to incorporate the highest quality OpenEXR output.

ACES and OpenEXR are both open source.

For more information on ACES:-

Presentation: http://www.oscars.org/science-technology/council/projects/pdf/ACESOverview.pdf
ACES: http://www.oscars.org/science-technology/council/projects/aces.html

Ampas/ACES sourcecode: https://github.com/ampas/aces-dev/tree/v0.1.1

ACES discussion group: https://groups.google.com/forum/#!forum/academyaces

OpenEXR: http://www.openexr.com/
OpenEXR sourcecode: http://www.openexr.com/downloads.html


Thoughts?

13
Hi guys,

Please excuse the shameless plug (I hope the mods are ok with me doing this  ::) )

We just launched a new product called Cinelog which works with Adobe Camera Raw and most raw video shooting Canon cameras, plus the Digital Bolex and Blackmagic Cameras.

Full info is on our website www.cinelogdcp.com

Unfortunately this release isn't a freebie (and yes we know there is something similar available for free) but Cinelog is true log space conversion for Magic Lantern Raw and other DNG based video and comes with a comprehensive LUT pack and guide.

If you have any questions or want sample frame conversions to test gradability let me know using the contact form.

14
General Chat / Magic Lantern HDR video - 2 years old today
« on: December 20, 2013, 06:46:18 PM »
I could be wrong about the date (maybe a day or 2) but I think today marks the second anniversary of Magic Lantern HDR Video. Thanks a1ex!

15
Raw Video Postprocessing / Film Punch LUT available to download
« on: December 11, 2013, 06:18:09 PM »
Hi guys,

I've been grading a project shot in Magic Lantern Raw Video and thought I would share a LUT I developed with you.

This is my Film Punch LUT (It's a kinda 'Blockbuster, blueish shadows, punchy but with more natural skin tones' look) and can be downloaded here: Film Punch LUT

I have included a basic guide on how to use the LUT correctly. It will work with any LOG footage but the guide details an Adobe Camera Raw/AE/PPro workflow.

Also included a sample DNG and JPEGs






Before (CDNG LOG):




After (with Film Punch LUT, a touch of extra contrast and a vignette):




Regards

Andy600



UPDATE: I have uploaded 2 more LUTs for those not using ACR (Raw-to-LOG for use before applying Film Punch and Raw-to-LOG-Film-Punch for an all-in-one look for raw or REC709 footage). Details in the zip: https://bitbucket.org/andy600/tl50d/downloads/Film_Punch_LUT_Update.zip note: the original method with ACR will give the best results but this is for those who can't do that!

16
Been having an issue with how ACR handles over exposure (or possibly over saturation?) on some shots. MLV files shot on a 50D, dumped to DNG.

Here's an example where red taillights can go pink with ACR (v8):



ACR (Visionlog - only WB and exposure adjusted) Color with Colorista in AE CS6





Resolve (V10.0.1)




I could obviously just use Resolve or use a secondary to recolor the pinks in AE but want to know what the cause is? It's only happening in the red channel (I think).

Shot on 50D.


17
Davinci Resolve 10.0.01 has been released for anyone that's interested. Apparently it fixes the roundtrip xml problems with Premier Pro.

http://www.blackmagicdesign.com/support

18
Share Your Photos / 50D Super Resolution Dual ISO
« on: November 14, 2013, 05:37:13 AM »
Although my main focus is video I'm starting to love my 50D for photography.

I've been experimenting with photo matrixing, taking multiple shots of interesting architecture around Riga, Latvia.

Here's one of my first efforts (click link below for scaled to HD image):



This shot is made up of 15 x 15mp images (3rows/5 columns). Original composite is around 19000px wide. Shot on an old, manual Auto Revuenon 50mm F1.4 but using ML Auto ETTR. Processed with ACR and Microsoft ICE (for stitching the images)

I hadn't heard of ICE until I went looking and it's fantastic. Beats Canon's photo stitch and Photoshop photomerge. Well worth the FREE download if you're on PC http://research.microsoft.com/en-us/um/redmond/groups/ivm/ICE/


View a scaled to HD size here: http://i.cubeupload.com/BIuArx.jpg


UPDATE: Here is a small gallery with a few more. Shot with ML Dual ISO (up to 70x shots per composite/up to 58k pixels wide) Images scaled to about 3-5% of their original size for web viewing ;)

http://imgbox.com/g/NMOHCoarPn













19
General Chat / Case Study: Ender's Game - worth the watch.
« on: November 11, 2013, 09:23:55 PM »
Interesting case study of the pipeline used for Ender's Game. Some useful info too regarding colorspace and intermediates for CGI.



20
General Development Discussion / Anti Aliasing - crazy idea (script?)
« on: November 03, 2013, 12:48:06 PM »
I usually shoot crop raw video on my 50D because aliasing can be pretty bad in non-crop mode and this got me thinking.

If the camera is line skipping (I presume it keeps line 1 and skips lines 2 & 3 etc) could it be possible to post-process a dng by splitting each vertical and horizontal line and space the resulting pixels apart as if they were not skipped i.e. pixel 1 - space -space - pixel 2 -space -space - pixel 3 etc then interpolate the missing data. I know 2/3rds of the data is missing and would need reconstructing which (I guess) would be very processor intensive and not accurate enough for full image reconstruction but it might be usable for compositing small areas where aliasing ruined a shot.

21
Share Your Photos / Central Market, Riga, Latvia - 50D Dual ISO
« on: November 01, 2013, 12:10:03 AM »
Some 50D dual ISO shots at Central Market in Riga, Latvia.

Processed with a1ex's cr2hdr-amaze-edge2 (Thanks a1ex, love it!) and ACR.

Not bad for a 5 year old camera ;)

More shots and larger size gallery here: http://imgbox.com/g/oDlbuS2Gcx







22
General Chat / Real-time HDR upto 18 stops from Canon DSLRs
« on: October 01, 2013, 11:45:58 AM »
WTF?

This guy claims up to 18 stops of realtime HDR / 32bpc :o

http://www.newsshooter.com/2013/09/15/ibc-2013-gohdr-hdr-video-from-off-the-shelf-dslrs/

How is he pulling 720MB/s (43GB a min) from 2x 5D MkIII's (at least, I think they are 5D3's??)


23
Raw Video / Mis-information by the pros about ML raw video
« on: September 16, 2013, 04:02:21 PM »
I've noticed a couple of blog articles from professional shooters recently that seem to contain mis-information about ML raw video.

In Nino Leitner's article http://www.ninofilm.net/blog/2013/09/16/kineraw-mini-vs-5d-mark-iii-raw-dynamic-range-test/?utm_source=feedly (posted today) he suggests "It seems like you can “bake in” a picture profile look into the 5D3 raw files."

and in videos that accompany an article by Paul Kell (also retweeted by @autoexec.bin on the front page of Magic Lantern) http://www.eattheblinds.com/2013/09/once-you-go-raw.html he clearly states that QScale VBR -16 has an effect on raw video aliasing.

Although I think these quotes are totally incorrect, what would make a professional come to these conclusions? Are they simply mistaken or is there any substance to what they are suggesting?

24
Feature Requests / Idea - Ansel Adams metering
« on: September 15, 2013, 05:40:57 PM »
Just throwing another idea out there for the devs.

We have some fantastic exposure aids in Magic Lantern and IF you know ire values and can read vectorscopes etc you can achieve great results but I have an idea for a more simplified metering system (maybe a new module) based on the Ansel Adams zone system http://en.wikipedia.org/wiki/Zone_System.

The basic idea is for something akin to zebras and false color metering but where a chosen zone (i.e. skin in zones 5 - 7) would pass through while everything outside is blacked out. Each zone (or a selected range of zones) could be selected for different exposure purposes. This would be a very fast way of setting exposure, especially for skin tones because skin would not be seen until it comes within correct exposure.

Ok, this is nothing new and of course, shooting raw lets you do most of the work in post but as the cameras are pretty much in the 10-stop range that the AA zone system uses it kind of makes sense to use it. It is, after all, a very widely used and respected system.


25
Raw Video Postprocessing / GoPro Cineform Studio V2.0.0.285 is out
« on: September 10, 2013, 11:02:43 PM »
Just a heads-up. GoPro Cineform Studio has just been updated to V2.0.0.285

http://gopro.com/software-app/gopro-studio

Key Features

    New: GoPro Edit Templates make it easy to create GoPro-style videos
    New: Trim, edit and mix video clips
    New: Add titles, music, audio tracks and more
    New: Fisheye adjustment control
    New: Adjust video playback speed for fast or slow motion
    New: Export video still frames at full resolution
    Create videos from time-lapse photo sequences
    Edit immersive 3D videos (requires 3D HERO System, sold separately)
    Supports GoPro, Canon®, Nikon® and other constant frame rate H.264 mp4 and .mov formats
    GoPro CineForm codec provides up to 1080p with deep color in 3rd party software

Not sure if there are any improvements for raw video as I've only just downloaded it.

Pages: [1] 2 3