Author Topic: The CinemaDNG Discussion (raw2cdng) (Read 1056625 times)

mva · « **Reply #400 on:** December 05, 2013, 10:55:54 PM »

chmee, yes, although I do pretty much everything in English on my computer(s), all are enabled for Hebrew characters as well, I have Microsoft Office Hebrew language packs installed in them, etc. Could this be a factor?

Yes, I did try running the app from the root directory of a different drive, but without success.

chmee · « **Reply #401 on:** December 05, 2013, 11:30:05 PM »

my thoughts went to unicode/ascii (chracterencoding) and something like that. maybe it cant read/write/find the proper file because of differences in right that. wrong character-encoding leads to "FileNotFound". i didnt had such an issue, so finally cant help by now, but maybe "our" worldwideweb can.. have to search and read about it.

regards chmee

mva · « **Reply #402 on:** December 06, 2013, 12:19:55 PM »

Quote from: chmee on December 05, 2013, 11:30:05 PM

my thoughts went to unicode/ascii (chracterencoding) and something like that. maybe it cant read/write/find the proper file because of differences in right that. wrong character-encoding leads to "FileNotFound". i didnt had such an issue, so finally cant help by now, but maybe "our" worldwideweb can.. have to search and read about it.

regards chmee

FYI, I updated to .NET 4.5.1 and now raw2cdng works!

chmee · « **Reply #403 on:** December 06, 2013, 09:50:45 PM »

lol. really.. i have to check, what code-excerpt uses .net 4.5 (i'm kind of pissed, VisualStudio doesnt show up those things).
Thank you.

jordancolburn · « **Reply #404 on:** December 07, 2013, 05:15:07 PM »

Thanks for this awesome tool. I've been doing a couple of test shots just to get the workflow down.
I'm currently going: EOS-M(crop mode) -> focus pixel remover -> raw2cdng -> Premiere(editing) -> speedgrade (grading)

The latest raw2cdng w/ partymode is really great, and seems to have more lattitude in speedgrade vs h264, but I have noticed an issue where clipped values in raw show as magenta in speedgrade and premiere. Any help or tips would be appreciated, I've tried going through this whole topic, but maybe I missed something along the way. Thanks!

timbytheriver · « **Reply #405 on:** December 07, 2013, 05:59:25 PM »

@chmee Thank you for v1.4.8! I have much better results now in 'Partymode' and Gamma 1.0 getting rid of the pink going into PPro 7.1

Something else I have noticed is this chromatic aberration (?) in Cdng. I don't see it in straight dng. Two 100% crops from the same .raw file to illustrate here (You can see it clearly on the ripples):

https://dl.dropboxusercontent.com/u/21590714/raw2Cdng.jpg

https://dl.dropboxusercontent.com/u/21590714/raw2dng.jpg

Where is this coming from?

If this is old ground already covered / fixed, I do apologise!

rgds

PS Both the files here were debayered in ACR from the original .raws and then cropped.

chmee · « **Reply #406 on:** December 07, 2013, 11:18:59 PM »

@jordancolburn
appreciate that

canon raw have different black and whitelevel for the components (r, g and b) it seems. if you use the partymode, all values lower blacklevel (~1023 or ~2047) will be cutted and all values higher than 15000 will be cutted as well. So, yes, it seems, there is an under- or overflow of values (i already have switches for that, but your picture shows theres another thing i missed)

@timbytheriver
you're welcome. one or two sites ago, there was an example with the same problem. it seems (in your picture as well) there is CA in dng as well, but much weaker. by now i dont have any clue to avoid that. did you try to set a according modelID? Maybe ACR is reacting well with the right Body ID?

so, sorry i cant help by now. but the tool isn't finished yet (g3gg0 just offered his help to some questions) and i hope i can throw out a nutty version till christmas

timbytheriver · « **Reply #407 on:** December 08, 2013, 12:33:28 PM »

@chmee Good news about the xmas version! Can it have chocolate, brandy and cream included?

So model ID/no model ID seemed to make no difference to the chromatic aberration issue. I did notice something odd though: Mac OSX finder window reports all my dngs from raw2cdng as being dimensions: 1864 x 1042, whilst all other tools (RawMagic, raw2dng, etc.) report them as 1876 x 1054 (I shot at 1880 x 1058 ad build 4.10.13)

Odd!

Could this be a clue?

Thanks again.

rgds

chmee · « **Reply #408 on:** December 08, 2013, 01:10:16 PM »

but the raw-infos inside raw2cdng are accurate? in dng there s an attribute for padding - i set it to zero. will look on that. thanks.

timbytheriver · « **Reply #409 on:** December 08, 2013, 04:51:39 PM »

@chmee raw2cdng gui reports the dimensions as 1880 x 1058

Malex · « **Reply #410 on:** December 08, 2013, 08:16:46 PM »

Hi, I have just tried 1.4.8 and great! but...
I did have few bugs, I guess it has to do with speedgrade not being stable enough with dng files ? I had this before too. When I use the direct link from premiere to speedgrade, it opens the files fine, OR it opens it with some greenish look... it's random, and I guess it has nothing to do with raw2cdng. It looks good in premiere, even though I think the white balance is off a little bit.

Great job chmee!

jordancolburn · « **Reply #411 on:** December 09, 2013, 02:46:18 AM »

Quote from: chmee on December 07, 2013, 11:18:59 PM

@jordancolburn
appreciate that canon raw have different black and whitelevel for the components (r, g and b) it seems. if you use the partymode, all values lower blacklevel (~1023 or ~2047) will be cutted and all values higher than 15000 will be cutted as well. So, yes, it seems, there is an under- or overflow of values (i already have switches for that, but your picture shows theres another thing i missed)

Thanks for the reply. I'll see if I can recreate it with some more details, but for now here is an example image:

I was trying to do a tough test vs h264 and tried to shoot inside with the lights off and a bright snowy outside, so to get the inside exposed right, the outside was very overexposed, but the highlights came back ok in SG (except the magenta area).

chmee · « **Reply #412 on:** December 09, 2013, 05:03:33 PM »

@jordancolburn

i'm searching/reading for this red/pink highlight thing. It seems, the whitelevel is higher set than needed. in RT there is a bugreport right about that, but they mention lowering the WL (after debiasing?). i'll try some of the things

links:
magenta highlights - https://code.google.com/p/rawtherapee/issues/detail?id=1695
WB-subtraction and RGB to XYZ - http://lclevy.free.fr/cr2/#wb

regards chmee

a1ex · « **Reply #413 on:** December 09, 2013, 05:11:13 PM »

@jordancolburn: may I see your DNG?

(it's offtopic, I'm just experimenting with some highlight recovery algorithms)

jordancolburn · « **Reply #414 on:** December 09, 2013, 06:48:25 PM »

Quote from: a1ex on December 09, 2013, 05:11:13 PM

@jordancolburn: may I see your DNG?

(it's offtopic, I'm just experimenting with some highlight recovery algorithms)

Sure, when I get home today, I'll grab the DNG frame send it to you.

jordancolburn · « **Reply #415 on:** December 09, 2013, 06:52:31 PM »

Quote from: chmee on December 09, 2013, 05:03:33 PM

i'm searching/reading for this red/pink highlight thing. It seems, the whitelevel is higher set than needed. in RT there is a bugreport right about that, but they mention lowering the WL (after debiasing?). i'll try some of the things

Thanks for looking into this! Out side of that bug, raw2cdng really was the final part of the workflow I needed to get into editing and grading. Now I can actually do a real-ish world test comparing h264 and raw with the eos-m.

chmee · « **Reply #416 on:** December 18, 2013, 10:44:28 AM »

https://infogr.am/bmcc-delinearization-curve?src=web
(some brainfood) (now both curves lin>log and log>lin inside)
full csv-data for log>lin on http://dl.phreekz.de/raw2cdng/bmccLog.csv

regards chmee

IliasG · « **Reply #417 on:** December 20, 2013, 06:41:36 PM »

Quote from: chmee on December 18, 2013, 10:44:28 AM

https://infogr.am/bmcc-delinearization-curve?src=web
(some brainfood) (now both curves lin>log and log>lin inside)
full csv-data for log>lin on http://dl.phreekz.de/raw2cdng/bmccLog.csv

regards chmee

Chmee hi,

Do you mean that you (will) use Bmcc curves ?. At which bitdepth ?. If you need something just ask. I think the old gamma curve I provided needs a bit refinement. Of course it was proposed for 14 bit linear to 10 bit log so we will need something different if our intention is to use gamma encoding for 14lin to 12log or 16 to 12.

Regarding magenta highlights, because the clipping White level for each model depends on ISO and apperture, in latest RT there is now an external file (camconst.json) where we can set per ISO White Level and a multiplier depending on apperture. I think a database is the only way to be safe while not loose highlights.

Or you can use exif (maker data) info in CR2 files, latest exiftool decodes more Canon tags and White Levels are among them although they are a bit conservative sometimes ...

Currently, for RT, there is a need of Canon raw samples with at least a "burned" area shot with intermediate ISOs (125, 160, 250, 320 ...) to fill the database ..
5D2 and 5D3 are the only models whith complete data, all others (and 6D) are missing.

glubber · « **Reply #418 on:** December 20, 2013, 09:20:53 PM »

Interesting.... i was stumbling too upon the camconst.json and the researchs of the RT community just today.
But in purpose of finding a solution for the pink highlights in dcraw/UFRAW.
=> http://www.magiclantern.fm/forum/index.php?topic=5197

As i don't have PremierePro, i wasn't aware that the pink cast problem could have the same origins as the dcraw case.
Maybe the new raw engine of Premiere is basing upon dcraw?

Edit: I just remembered seeing your name on the RT issue board.

chmee · « **Reply #419 on:** December 20, 2013, 10:51:17 PM »

i made a second logarithmic convert-mode (12bit with bmcc gamma) - for the shadows and the whole picture grace it does a good job, but it cant eliminate the red tint, because of the different component blacklevel and amplitude - it's time to go deeper - something i wouldnt do, because i recalculate the raw-data more i supposed to. it's a real shame, that the dng-importer in premiere doesnt have a colormatrix-option or reads the colormatrix/cameracalibration-data accordingly. omg.

@IliasG just send me a binary file with ~~16384 int (2byte) entries (14bit->10bit)~~ 16bit->10bit.

@glubber my thoughts, its not really about a red-overflow. if its magenta (red-blue'ish), the green-channel isnt accordingly read, if its green, it was not taking into account, there are two green sensels..

regards chmee

IliasG · « **Reply #420 on:** December 20, 2013, 11:41:14 PM »

Binary ??. I don't know how .. a simple text file with input and output columns is the best I can ..

I'll have to reopen and remember what the refinement is .. hopefully tomorrow

Did you managed to insert the linearization table in DNG exif data ?.

chmee · « **Reply #421 on:** December 21, 2013, 06:14:08 PM »

@iliasG
yeah. do that. i'll convert it here via php. i didnt inserted the linearizationtable by now, because i assume, premiere does not take account of it. For Speedgrade and Resolve it has to exist, so yes, i'll do that today.

edit(!) IliasG, forgot to say: dont cope with the uneven black- (1023/2047) and whitelevel (~15000). take 0 as blacklevel and 65535 as whitelevel for the gammacurve. internally i do now all from 16bit to destination-bitdepth.

rommex · « **Reply #422 on:** December 25, 2013, 03:05:30 PM »

Hello Chmee,

I've been long a fan of your application, but only recently discovered this thread (stupid me).
I read it wholly before I was going to make some points.
First I want to thank you for your efforts and readiness to help.

I'll try to be brief in my points:

1. DaVinci Resolve. As I got from this forum, there was a trend to move to Adobe from Resolve due to a better de-bayering. It was true with Resolve 9 (I hated it) but Resolve 10 for me show far FAR better results than Adobe ACR. I shoot with 50D, and artifacts from Adobe are huge compared to Resolve 10.

Things move quickly, and maybe somebody should give Resolve 10 Lite a try.

2. Roundtrip. So I use R2C for Resolve and Premiere roundtrip. I read the whole discussion about naming and XML problems and while reading really had a desire to jump back into the past ))

2.1. FPS. Now, I am shooting 25 FPS, but when imported to Resolve, all clips are 24FPS. Have to change it manually. When I forget, I have a de-sycnh with audio, and have to re-render. But this is the beginning for surprises ))

2.2. Folder naming. So I have 2 clips from shooting raw video:

1.M24-1803.RAW +2 spanning files
2.M24-1806.RAW +2 spanning files

They were shot in consequence, one after another at 18:03 and 18:06 yesterday, 24 Dec.

After conversion (made today on 25 Dec at about 15:00) I have 2 folders:

131225_1235_M241803K
131224_1235_M241806N

Why 131225_ and 131224_ ? Why _1235_ and _1235_?
All this makes basically impossible to sort these folders by name.

2.3. Resolve clip naming. When imported into Resolve, these clips show as:

M241803K.[000000-003194].dng
M241806N.[000000-004983].dng

That is a step back -- I can sort them now, which is good. But after grading when I render my dailies with "Use source filenames" checked, I have files named:

M241803K.mov
M241806N.mov

2.4. XML issue. After editing these files in Premiere, I export XML file for Resolve. And of course I have that problem where only the first clip shows ok, and others have error. Yes now I know about the solution with reelname suggested by a French-style contributor.

But I also solved this problem in my way: I searched through XML file and changed:

M241803K.mov to M241803K.[000000-003194].mov
M241806N.mov to M241806N.[000000-004983].mov

And now the problem was solved -- Resolve made all linking correctly, without reel name.

Of course you cannot do that for big projects, but I think solution is in file naming -- to make Resolve import clips and render clips with the SAME NAME.

3. 1.4.8 version.

Presently I use 1.3.0 because 1.4.8 is totally unstable for me -- it crashes in the middle of every second conversion. 1.3.0 is as solid as a rock ))

I also tried MLV conversion after placing mlv_dump.exe into the R2C folder, but the program shows only one frame from the clip. Failed to convert.

And by the way, on the screen shot you have 1.4.9. Are you hiding anything?

Just kidding!

Let me say big THANK YOU for your sincere efforts! Happy Christmas!

mediali · « **Reply #423 on:** December 25, 2013, 05:13:54 PM »

same problem other conver is work but raw2cdng have wrong in this :
import in premiere is same problem

2.1. FPS. Now, I am shooting 25 FPS, but when imported to Resolve, all clips are 24FPS. Have to change it manually. When I forget, I have a de-sycnh with audio, and have to re-render. But this is the beginning for surprises ))

chmee · « **Reply #424 on:** December 26, 2013, 11:03:49 AM »

Happy Christmas, mediali & rommex.

Thanks. So, the fps-problem is a thing inside resolve. You have to adjust the basic settings of your project. It seems, resolve is not interested in the fps-metadata.

The problems Wirth the newer versions will come From the multithreaded reading. I'll rewrite that. So stay on 1.3, for resolve it makes no difference.

News:

Author Topic: The CinemaDNG Discussion (raw2cdng) (Read 1056625 times)