Author Topic: The CinemaDNG Discussion (raw2cdng)  (Read 712577 times)

chmee

  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion
« Reply #25 on: May 29, 2013, 11:20:26 PM »
no. cinema dng and "simple" dng differs much - in terms of mandatory tags. i use a cdng-template, i merged myself from looking into another cdng-frames. just to clarify some things:

(*) dng cant be converted into cinema dng with only two or three another tags. but maybe these videoapps understand dng and dont need any cdng-tags?!
(*) main problem is, 14bit bayer is not useable for cinemadng, because no videosoftware can read 14bit frames as sequence. (my app is able to output 14bit cdng - but there is no software able handling it)
(*) dng batch converting is a "half baked" solution, because it ends in 8bit-sequences with loosing all benefits
(*) converting into tiff-rgb 16bit needs demosaicing and color matrix handling -> not my intention (ffmpeg?)
(*) my app (in actual state) does some weird things while converting to 16bit-bayer - and thats the point of "not-usability"

(+) the bayer structure is ok, because cinema dng can handle it, also the apps, that are handling cdng
(+) it must be converted into 10 or 16bit bayer. 10bit would be worse than 16bit, but definitively better than 8bit baked.
 
regards chmee

hjfilmspeed

  • Senior
  • ****
  • Posts: 469
  • 5D III and IV
Re: The CinemaDNG Discussion
« Reply #26 on: June 01, 2013, 07:09:31 PM »
I feel like RAW to Cinema DNG GUI would be amazing!!!! Opening right up in the free version of resolve would be alot of fun!

chmee

  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion
« Reply #27 on: June 02, 2013, 07:33:15 PM »
Just to give you some hope :) The app is working, when i'm writing synthetic data (10, 12 and 16Bit Output) - but i'm stuck understanding the "simple" 14Bit-RAW-Data. Taking the short RAW-Example put on my server (https://github.com/chmee/misc/tree/master/%5Braw%5Dsamples (zipped raw samples 1280x540))

NEED YOUR HELP FOR THE LAST THING!

Example from 1280_540.raw - first 7 Bytes describe the first 4 Sensels ( RGRG)

Code: [Select]
hex - 40 29 42  c7 4c  9c 2a
dec - 64 41 66 199 76 156 42

binary:
| 01000000 | 00101001 | 01000010 | 11000111 | 01001100 | 10011100 | 00101010 |

as far as i understand restructuring to 14 bit goes:

| 01000000001010 | 01010000101100 | 01110100110010 | 01110000101010

dec - 4106 | 5164 | 7474 | 7210
BUT this data dont correspondend with the image i see in 14bit. its a uniform background - greyish on Blacklevel 2046, reddish with blacklevel 0. both pixels Pixel1(RG) and Pixel2(RG) have to be (more or less) same value, what is going on?


regards chmee

chmee

  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion
« Reply #28 on: June 02, 2013, 11:19:31 PM »
ok, here's a v0.99 for beta testing. thanks to reyalp for the hint.
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

all responses, hints are very welcome. regards chmee

olik

  • Freshman
  • **
  • Posts: 81
Re: The CinemaDNG Discussion
« Reply #29 on: June 03, 2013, 01:30:27 AM »
Hello chmee,

this is very exciting. However, after converting a raw file that I created with the 5dmk2 without changing the bitrate, I could not open the dngs with resolve 9. It did not "see" the dngs. So I opened one dng with photoshop where I got presented a very contrasty image, barely recognizable without blue channel (will post dng tomorrow).

arturochu

  • Member
  • ***
  • Posts: 129
  • Filmmaker based in Mexico City
Re: The CinemaDNG Discussion
« Reply #30 on: June 03, 2013, 05:37:03 AM »
same problem as last post, i thought it would make a single cdng file but made a lot of dngs that i cannot open in resolve
Chu

chmee

  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion
« Reply #31 on: June 03, 2013, 08:19:45 AM »
thanks for response. actually i just tested with 5DIII-Files. So it would be fine you're leaving somewhere a short & small raw-file from your bodies. Ah btw. No, it is a filesequence with cinema-dng pictures (their ending is dng as well) - for using in resolve its mandatory changing to 16bit! olik, post a screenshot. (seems i have to delete the 14bit-converting :)

regards chmee

gogauze

  • New to the forum
  • *
  • Posts: 5
Re: The CinemaDNG Discussion
« Reply #32 on: June 03, 2013, 09:36:51 AM »
I did a quick test with my 5D3 and converting into 16bit, works like a charm. I'll be even more happy when one of the Mac folk ports it to OSX. I am sincerely impressed with your skill.

squig

  • Senior
  • ****
  • Posts: 498
  • Crash test dummy MK3
Re: The CinemaDNG Discussion
« Reply #33 on: June 03, 2013, 10:06:19 AM »
What's the increase in file size converting to 16bit?
www.gamma-movie.com post-apocalyptic feature now in pre-production.

gogauze

  • New to the forum
  • *
  • Posts: 5
Re: The CinemaDNG Discussion
« Reply #34 on: June 03, 2013, 11:30:13 AM »
Fairly negligible really. I think it increased by maybe 10% on the quick test that I ran.

IliasG

  • Member
  • ***
  • Posts: 124
Re: The CinemaDNG Discussion
« Reply #35 on: June 03, 2013, 01:21:44 PM »
@ Chmee,

Congratulations, and many thanks.

Regarding file size.
 
A good solution would be to store the data in the exported cine-DNG at 12bit log (after applying a LUT on the 14bit linear raw data) and then inject a inverse LUT (12bit log to 16bit linear) in the corresponding exif tag. It's the same as BlackMagick do and Nikon's "visually lossless" raws also. This would give a 12/16 file size.

What happens if we pass this 16bit cine-DNG through Adobe DNG converter. If it does not destroy the cine-DNG format is would give a respectable size decrease as by default it applies lossless compression which for such files can compress at about half the size.

olik

  • Freshman
  • **
  • Posts: 81
Re: The CinemaDNG Discussion
« Reply #36 on: June 03, 2013, 01:33:35 PM »
Tested again for the 5dmk2, this time I changed the bitrate from 12 to 16bit. This solved my issue, I can now import to resolve. I have a strong green cast though, but it is perfectly correctable. Wonderful job! I love it :-)

Here the C-dng in resolve


vs h264 (same camera settings)


and the first DNG of the sequence
https://picasaweb.google.com/lh/photo/tZ7PtaVRzHpdgX1PKcC_0aHmTw6ITH9KGra-l8uiJak?feat=directlink

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10703
  • 5D Mark Free
Re: The CinemaDNG Discussion
« Reply #37 on: June 03, 2013, 03:28:16 PM »
I have a strong green cast though

The black level is too high; it's probably hardcoded at 2048, which is usually right for 5D3, but too big for 5D2. The one from exif should be a good starting point.

scrax

  • Developer
  • Hero Member
  • *****
  • Posts: 1417
  • Code monkey
Re: The CinemaDNG Discussion
« Reply #38 on: June 03, 2013, 05:29:18 PM »
I can load the app in Osx with mono and load in a file, but how to start it?


EDIT, maybe it did all, I got a lot of dng but with some stranges names due to the win \ for folders I suppose

Also mono gives this error when loading it:
mono[75121] <Error>: kCGErrorInvalidConnection: CGSGetWindowTags: Invalid connection
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus - EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX Lion, Photoshop & Lightroom -no video experience- MLTools

scrax

  • Developer
  • Hero Member
  • *****
  • Posts: 1417
  • Code monkey
Re: The CinemaDNG Discussion
« Reply #39 on: June 03, 2013, 06:07:09 PM »
So far on osx when converting a raw file I got this file structure at the end:

rawfilefolder/M0000001.RAW
rawfilefolder\ML_16/   <--this is not a subfolder of rawfilefolder it's another empty folder
rawfilefolder\ML_16\ML_16_000000.dng
rawfilefolder\ML_16\ML_16_000001.dng
rawfilefolder\ML_16\ML_16_000002.dng
...  and so on with files having the win folder structure in the name.
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus - EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX Lion, Photoshop & Lightroom -no video experience- MLTools

olik

  • Freshman
  • **
  • Posts: 81
Re: The CinemaDNG Discussion
« Reply #40 on: June 03, 2013, 06:10:54 PM »
While we already talk about using this software on different operating systems, I tried using it on Ubuntu Linux with mono, without luck;

Code: [Select]
mono '/raw2cdng.0.99.exe'
Missing method EnableVisualStyles in assembly raw2cdng.0.99.exe, type System.Windows.Forms.Application

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.
File name: 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
[ERROR] FATAL UNHANDLED EXCEPTION: System.IO.FileNotFoundException: Could not load file or assembly 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.

File name: 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

has anyone an idea?


scrax

  • Developer
  • Hero Member
  • *****
  • Posts: 1417
  • Code monkey
Re: The CinemaDNG Discussion
« Reply #41 on: June 03, 2013, 08:19:26 PM »
seems i have to delete the 14bit-converting :)
+1
I'm using ML2.3 for photography with:
EOS 600DML | EOS 400Dplus - EF 100mm f/2.8 USM Macro  - EF-S 17-85mm f4-5.6 IS USM - EF 70-200mm f/4 L USM - 580EXII - OsX Lion, Photoshop & Lightroom -no video experience- MLTools

chmee

  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion
« Reply #42 on: June 03, 2013, 10:02:33 PM »
thanks for the testers.

summarizing for the next versions
* >2GB filehandling
* path-strings uniform and no windows-dumb backslash
* pink fringing
* vertical lines (could be solved with 0xc619 or 0xc61b )
* 10 and 12 bit files (no logarithmic with LUT for now - think, there is a tag for it - linearizationTable 0xc618)
* batchmode?!
* handling other bodies with their metadata (colormatrix, whitelevel and so on)

regards chmee

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 10703
  • 5D Mark Free
Re: The CinemaDNG Discussion
« Reply #43 on: June 03, 2013, 10:27:50 PM »
* vertical lines (could be solved with 0xc619 or 0xc61b )

This is black level for each column, but for the banding issue we need EV correction for each column (mod 8 ). Do you know any tag for that?

chmee

  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion
« Reply #44 on: June 03, 2013, 10:34:05 PM »
ahh. ok. there is no metadata-array for ev correction - as i see that now.

Some Question there have been:

14 to 16Bit -> ~14.2% increase (16/14 = 8/7)
14 to 12Bit -> ~14,2% decrease(12/14 = 6/7)

@IliasG
No, logarithmic transforming is not my intention. for 12Bit i will cut LSB 2 Bit - we'll gonna see, how many data in the shadows is going to be lost - i quite think, its negligible :) Dont forget - the Magiclantern Team, all the coders around it gave you a weapon you never had the money for that - and this "techspec-more is better"- blahblah doesnt make a good film - your movie is not depending on logarithmic 16bit, but on "substance"!

@olik
what version-number of .net-framework is running in mono? it needs 4.0

silwerfedlt

  • New to the forum
  • *
  • Posts: 4
Re: The CinemaDNG Discussion
« Reply #45 on: June 03, 2013, 10:47:28 PM »
This is truly amazing and I´ve tested your tool shortly tonight with my 5D2 and I had to lower the blacklevel alot I think I had it on 1024 (just trying different settings to figure out how it works..). If it was on 2000 I had a very dark picture with alot of green. What would be the right setting for 5D2? Whys does it doffer from 5D3?

I´m new to resolve so I have to learn alot to be able to grade thorougly in the program but the basics are the same as the comp programs I use daily. (The Foundry Nuke and Eyeon Fusion).

Its so nice to be able to skip the Adobe programs - I don´t trust them at all for grading RAW sequences...


Cheers

jtkleine

  • New to the forum
  • *
  • Posts: 4
Re: The CinemaDNG Discussion
« Reply #46 on: June 03, 2013, 10:53:03 PM »
Just wanted to chime in, that I managed to open a file originating from 550D in Resolve. I did run raw2cdng under Parallels in OS X Lion, in 16bit mode of course. The file looked basically OK, but it was just something I shot in a minute. I couldn't do any actual testing, as the clip had horrific noise and no real black or white to begin with. Just wanted to see if it works at all. Sorry, I have not time to do anything more helpful right now.

BTW, I tried also in Speedgrade, just for fun. That didn't work at all (I didn't expect it to). The image was mostly green. I don't think Speedgrade even supports CDNG, maybe only Alexa and R3D.

Chmee you are doing as a big favor. No much point in grading raw video footage anywhere else than in Resolve -- at the moment at least. I tested earlier some clips in AE and graded them wit Color Finesse. That workflow didn't make much sense. I mean, to first bake the raw data into static rgb and then start grading. Very frustrating. So thanks, great stuff.

olik

  • Freshman
  • **
  • Posts: 81
Re: The CinemaDNG Discussion
« Reply #47 on: June 03, 2013, 11:04:35 PM »
Quote
@olik
what version-number of .net-framework is running in mono? it needs 4.0

I have mono-runtime 2.10.8.1 but it seams that 4.0 is not even out, 3.0.10 was just released as a beta (http://www.go-mono.com/mono-downloads/download.html) am I missing sth? :)

chmee

  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion
« Reply #48 on: June 03, 2013, 11:14:04 PM »
@jtkleine - please leave a screenshot, what body you are using? maybe its a 12bit body? is some 12bit-Body able to raw-output? if you want to, just upload a short/small rawfile to analyse, whats happening.

http://dslr-kleinanzeigen.de/Tools/Body_Database/index.php?filter=prodeqcanon|rawbiteq12

@olik dont know much about mono - i just use linux-derivates for xbmc :)

generally, please wait till thursday, for now i'm working all day long - i cant do any changes.

jtkleine

  • New to the forum
  • *
  • Posts: 4
Re: The CinemaDNG Discussion
« Reply #49 on: June 03, 2013, 11:40:31 PM »
OK, I shoot a small clip and upload the raw file a little later. Where should I upload it? Maybe I just put in on a server and link it here. A screen shot too. But I don't have time today to fiddle with the black level settings in raw2cdng, so I just use the default.