Author Topic: The CinemaDNG Discussion (raw2cdng) (Read 1080485 times)

chmee · « **Reply #1200 on:** April 29, 2015, 06:34:05 PM »

tc to 0 is something i could implement, but i dont see any benefit on it.. does it really matter for you? second question is solveable with the filenamegenerator, try [F]_DNG_00001(_)

theartofweb · « **Reply #1201 on:** April 29, 2015, 07:01:10 PM »

Come to think of it I can set TC to 00:00:00:00 on Resolve. Everything is ok now.

Thanks

chmee · « **Reply #1202 on:** April 30, 2015, 11:15:44 AM »

Short Info. With ".net Core" raises the usage of this and other .net-tools to work under mac and linux.

Link [german] - http://t3n.de/news/net-core-fuer-linux-mac-os-607819/

regards chmee

KelvinK · « **Reply #1203 on:** April 30, 2015, 03:25:39 PM »

Quote from: chmee on April 16, 2015, 04:33:00 PM

@KelvinK
please do me a favor, convert the same "problematic" file more than two times - just to inspect, if these dropped frames vary.

regards chmee

sorry for delay, was away. tried convert problematic file 3-4 times, dropping same frame.

Jackie 4CHAN · « **Reply #1204 on:** April 30, 2015, 09:20:19 PM »

Hey Chmee,

First, thank you for everything you've done. This program works great at converting my .MLV files. However, I got a question about chroma smoothing. What exactly does it do?

The reason I ask is because I had it on when converting some footage shot on a 5dm3. When I import into AE, some of the highlights are pink. Even with using VISIONlog. I could not for the life of me figure out how to get rid of the pink without damaging the image.

After a day of searching for information on both the Adobe and ML forums, I decided to convert my .MLV files again, this time without chroma smoothing on. Whaaalaa! All of the footage is properly displaying. No more pink highlights.

So what's going on? Can you put it in layman's terms on what's happening? I read somewhere that the pink highlights are from the wrong black levels? But those threads were years ago, not sure what's the most up-to-date info.

I'm using RAW2CDNG 1.7.4

jankrueck · « **Reply #1205 on:** May 14, 2015, 07:23:57 PM »

hey chmee,

I just did some raw-shooting. (mlv with sound turned off to be correct)
and when starting to convert, I realized there is some kind of brigtness-shift between 12bit(maximized) and 16bit(maximized)
on raw2cDNG 1.7.4.

Is this intended?

12Bit(max)
https://dl.dropboxusercontent.com/u/26597584/12bit(max)_M14-1620_00000.dng

16Bit(max)
https://dl.dropboxusercontent.com/u/26597584/16bit(max)_M14-1620_00000.dng

16Bit
https://dl.dropboxusercontent.com/u/26597584/16Bit_M14-1620_00000.dng

Cheerio, Jan

chmee · « **Reply #1206 on:** May 16, 2015, 09:53:31 AM »

@all
sorry for doing nothing atm - i'm really busy.

@jankrueck
will check that. patience please.

jankrueck · « **Reply #1207 on:** May 16, 2015, 11:18:27 AM »

actualy I did some new comparison between your conversion
in different modes, mlRawViewer and mlv_mystic.
Because I trie to find the mlv -> cDNG conversion wich "does" quite nothing to my material.
dunno if this is stupid, but I want completly untouched files when I start to edit.

http://i.imgur.com/x6JccSg.jpg

I'm just wondering how different colors can exist.
Maybe I understand that whole mlv thing wrong,
but in my understanding it is just a container in wich some dng's are waiting to be let free

dng files will be here for download in some minutes.
https://dl.dropboxusercontent.com/u/26597584/compare.zip

Cheers,

theartofweb · « **Reply #1208 on:** May 16, 2015, 01:12:44 PM »

Why not just regular 16-bit no smoothing? It works great.

glubber · « **Reply #1209 on:** May 16, 2015, 03:34:13 PM »

@jankrueck:
In the end everything you see on your monitor is an interpretation of the sensor-data depending on your software (debayering).

I recommend reading the cinelog thread of Andy600. He gives a good insight of colorspace and the whole sience behind it.

Danne · « **Reply #1210 on:** May 16, 2015, 04:43:59 PM »

It,s a pretty interesting comparison. How is mlrawviewer lossless compression working in these examples? Check the sky next to the flower.

jankrueck · « **Reply #1211 on:** May 16, 2015, 08:34:30 PM »

Quote from: Danne on May 16, 2015, 04:43:59 PM

It,s a pretty interesting comparison. How is mlrawviewer lossless compression working in these examples? Check the sky next to the flower.

actualy there is no sky, its blue paper.

What you mean by lostless? dng export is always lostless in my point of view.
The Bottom pictures are all mlrawviewer. Just copy pasted them for comparison.

Quote from: glubber on May 16, 2015, 03:34:13 PM

@jankrueck:
In the end everything you see on your monitor is an interpretation of the sensor-data depending on your software (debayering).

I recommend reading the cinelog thread of Andy600. He gives a good insight of colorspace and the whole sience behind it.

I know how this works, but when you debayer dng's, in my opinion they should be all the same before I start grading.
Also cinelog is not embed into rawfiles. Its just kind of a LUT for makeing h264 files more flat to make grading easier.
This should only affect proxy files, like mov clips.

So if there is no colorspace changes while debayering, all pictures should look the same.
This comparison is a study about the debayering process itself, to ensure there is no lost or adding of information.

Quote from: theartofweb on May 16, 2015, 01:12:44 PM

Why not just regular 16-bit no smoothing? It works great.

Why no smooting? Because reducing noise on this level improves imagequality before adding Luts.

Aaaand like you see there is no difference between smooth files and not smoothed.
16bit (not maximised) is kinda dark, with smoothing and without.

----------------

mlrawviewer looks pretty much the same as I remember the scene looked like. (color & brightness)

Danne · « **Reply #1212 on:** May 17, 2015, 01:51:26 PM »

Mlrawviewer compresses the dng files. Does it show? Said to be lossless. Could you upload the sample files?

dmilligan · « **Reply #1213 on:** May 17, 2015, 03:30:15 PM »

Quote from: jankrueck on May 16, 2015, 08:34:30 PM

So if there is no colorspace changes while debayering, all pictures should look the same.
This comparison is a study about the debayering process itself, to ensure there is no lost or adding of information.

There is always a colorspace change during debayering, b/c raw has no colorspace and you must convert it to one to even be able to view the file. There is also always "lost and added" information during debayering. Debayering itself is a type of interpolation. You cannot view raw data directly, it must be synthesized to some real colorspace before you can even preview it.

There is a lot of metadata in a DNG file that tells the reader/debayer software how to read/interpret the raw file and how to convert it from the native raw "colorspace" to something standard like XYZ (this metadata can effect color as well as brightness and any manner of other image aesthetics). Also various software vary in how they interpret this data and of course debayering algorithms vary in speed/quality/artifacts/etc. Various tools that produce DNGs from MLV,RAW may vary in the particular metadata they put in the DNG file, which may affect the initial "look" of the DNG file when first opened, even if the raw data is the same. When you "edit", "grade", and "develop" the DNG file, you are likely going to be changing some of these parameters themselves, and your image is re-debayered, so even if images start out looking slightly different, you are modifying the parameters that made them look different and there will be no difference in the final output.

jankrueck · « **Reply #1214 on:** May 17, 2015, 05:53:45 PM »

Quote from: Danne on May 17, 2015, 01:51:26 PM

Mlrawviewer compresses the dng files. Does it show? Said to be lossless. Could you upload the sample files?

Samplefiles are downloadable in post above

@dmilligan: yeah sure there is some change or lets call it "import" into a xyz-colorspace.
it seems there is some misinterpretation or at least some problem with the metadata embed into the 16bit dng's.
Or lets not call it problem, but if there is a brightness-shift this strong just on 16bit. It seems to be wrong.

Just wanted to tell chmee what I've found

Andy600 · « **Reply #1215 on:** May 17, 2015, 06:08:18 PM »

@dmilligan - I couldn't have put it better myself!

To add to that, I have long suspected that the DCRaw matrices, and for that matter, all the Adobe DNG converter matrices that I recently gleaned are really only useful for Adobe apps and more specifically, stills/print processing - the reason being is that Adobe camera calibration allows out-of-gamut color in the color matrix tags which they have to 'manipulate' with forward matrices and HSV tables to get prettier or more 'accurate' and gamut safe color reproduction. FYI, the Forward matrix tags are also important to any .ICC dependent NLE or color grading app but they are not part of the CinemaDNG standard i.e. so we can't rely on them for any gamut 'correction' outside of Adobe apps.

Some of the work we (Cinelog) are doing for forum member @janoschsimon to calibrate his DIY MiniRaw camera, plus several other custom/scientific camera projects we worked on over the last year has highlighted to me that we can do much better to conform Magic Lantern Raw Video color to fit better in relation to the big raw shooting cinema cameras - afterall, Canon sensors have great color sensitivity so let's make the most of it.

I'm planning a 'Magic Lantern Raw Color' project and will publish details of what I want to do, how I will do it and how ML users can help sometime in the next few days. Hopefully a new set of ML color matrices I plan to develop will not only provide better default color that renders identically across all apps but, if/when accepted into ML source code, will allow me some confidence in developing proper ACES IDTs for each camera - at present, it's a crap shoot and unworkable with the single DCRaw matrix and differences between commercial and non-commercial apps. There is little point basing anything on the Adobe matrices for the reasons stated above. The resulting data will be published for any app dev to use under ML licensing!

To avoid any confusion - the DCRaw matrices (which actually come from Adobe labs) and Adobe DNG matrices are not 'bad' or 'wrong', they are simply intended for another purpose - they work fine for most things but IMO, it's time for Magic Lantern Raw Video Color to becomes it's own 'thing' without reliance on incomplete or borrowed data that 'might' have legal implications.

DeafEyeJedi · « **Reply #1216 on:** May 17, 2015, 06:51:32 PM »

@Andy600 -- great read once again ( along with @dmilligan's from few posts up

)

Looking forward to your much anticipated 'Magic Lantern Raw Color' Project and would be more than happy to help contribute and get these colors up to speed!

Keep up the excellent work and I appreciate your hustle on this -- it has definitely paid off on my part in getting used to the workflows around Adobe apps especially in AE (still can't resist from using ACR) so I must say THANK YOU for getting me on board, keeping me eager enough to stick through the bumps on the road and along with your patience I was able to absorb & learn to the fullest without giving up and still yet I'm learning!

Danne · « **Reply #1217 on:** May 17, 2015, 07:16:47 PM »

Quote

I have long suspected that the DCRaw matrices, and for that matter, all the Adobe DNG converter matrices that I recently gleaned are really only useful for Adobe apps

Thanks for calrification. I,ve been comparing these matrices with dcraw numbers and peeping dng files and came to the same conclusion recently.

Quote

I'm planning a 'Magic Lantern Raw Color' project and will publish details of what I want to do, how I will do it and how ML users can help sometime in the next few days.

How cool is that. Let me know if I can help.

chmee · « **Reply #1218 on:** May 18, 2015, 01:16:14 AM »

@jankrueck
let raw2cdng write the according whitebalance-data into the dng's.
(*) i assume all but AWB will work.
(*) in the beginning there was the way to shot a CR2 and name it like the raw/mlv-file. raw2cdng takes the whitebalance-data from this cr2 and put it into the dng. i didnt tested it the last versions

but finally all was said - the raw-data are nearly untouched - in simple 16 bit the least. its up to you and your app to get out most of it. the metadata inside dng are the slightest problem..

regards chmee

Licaon_Kter · « **Reply #1219 on:** May 23, 2015, 12:44:42 AM »

Heads-up, latest Lightworks 12.5.B beta imports cinemaDNGs and LUTs even in the free version: http://www.lwks.com/index.php?option=com_lwks&view=changelog&Itemid=207&id=194

Danne · « **Reply #1220 on:** May 23, 2015, 06:31:28 AM »

Quote

in the beginning there was the way to shot a CR2 and name it like the raw/mlv-file. raw2cdng takes the whitebalance-data from this cr2 and put it into the dng. i didnt tested it the last versions

Hi Chmee! Can I ask you what numbers you were using and applying to the dng files coming from CR2 files? I tried some with no success.

chmee · « **Reply #1221 on:** May 23, 2015, 10:52:04 AM »

exif MeasuredRGGB - calc.getRGGBValues() (line 1287) and calc.convertToFraction()
used in io.cs getMLVAttributes line 611 and getRAWAttributes() line 767

old discussion with informations

regards chmee

Danne · « **Reply #1222 on:** May 23, 2015, 11:22:54 AM »

Thank. I keep circling. Will take a dive

.

Danne · « **Reply #1223 on:** May 23, 2015, 08:20:05 PM »

So fiddled with dcraw. When writing

Code: [Select]

dcraw -i -v CR2 it puts out the camera multiplier from the CR2 file. It seems it is almost exact to AsShotNeutral. For instance.

Numbers from CR2 via dcraw
Daylight multipliers: 2.125175 0.943985 1.338680
Camera multipliers: 1722.000000 1024.000000 2032.000000 1024.000000

Dividing by 1024/1722=0.59465737514518 1024/2032=0.50393700787402

I put in following numbers with exiftool
AsShotNeutral=0.59465737514518 1 0.50393700787402

It gives a dng with settings in ACR Temperature 3900 Tint +7
CR2 gives Temperature 4050 Tint +8

Is it possible to get an exact measuring? It is close. I try to follow your links but I am lost

.

chmee · « **Reply #1224 on:** May 24, 2015, 01:15:47 PM »

so far this was the simplest way to get "good" (not perfect) body-data. dont forget, you can't count on body-measured data as well, it can differ from cutters opinion of "good" colors.

regards chmee

News:

Author Topic: The CinemaDNG Discussion (raw2cdng) (Read 1080485 times)