Magic Lantern Forum

Using Magic Lantern => Raw Video => Raw Video Postprocessing => Topic started by: chmee on May 23, 2013, 10:46:55 AM

Title: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 23, 2013, 10:46:55 AM
Hey there.. I'm currently trying to realise a CinemaDNG-Converter. First i thought it would just be an existing xmp-packet inside the dng-frame, but it wasn't. After reading the CinemaDNG-Definition i saw, there are about 35 mandatory tags.

DNG Specification 1.4.0.0 - June 2012 (http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf) - CinemaDNG Specification 1.1.0.0 - September 2011 (http://www.cinema-dng.com/wp-content/uploads/2013/06/CinemaDNG_Format_Specification_v1_1.pdf)

The CinemaDNG-File is just a TIFF/PE File (as DNG is also) with these CDNG-Tags. I wrote an App for Win7, building a CDNG-Header with the raw-pictures from the Camera-raw-file. They are 14Bit, isnt it? I figured out, that Resolve and Speedgrade cant handle 14Bit Raw-Frames! Just 8, 10 or 16Bit, maybe 12Bit. So, now i'm sitting on converting these 14bit to 16Bit-Chunks. These CinemaDNG-Sequences are recognised by Resolve and Speedgrade, but they are chaotic because of wrong handling/converting.

If asking, why the simple DNG-Sequences are not usable, it might be that
(+) missing CDNG-Tags
(+) IFD0-Entry is the jpg-Preview in 128x96px 8Bit - thats the one you see in Resolve.
(+) Raw-Data are packed into a subIFD - as in the DNG-Definition described.
(+) they are 14bit.
 
Questions:
(*1) Raw-Data - Subpixel-Chunks, MSB or LSB first? LittleEndian, so LSB first, right?
(*2) is it worth that? Maybe another Fileformat would be more usable? DPX? EXR? TIF?

regards chmee


regarding raw2cdng

i decided to stay on the same name and same tool with integrated mlv-convert (since 1.4.5). Resolve, Speedgrade and all ACR-using-Apps are coping well with the cdng's. With 1.5.0 (coming next) there s a new GUI, and on NAB(07.04.2014) Adobe releases a bigger update promising a better workflow with cdng. 1.5.0 is online. since 1.6.5 audiosync seems stable, colormatrices are refreshed. you find more info on my webblog.

please look in my blog-entry (http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/) to find the newest version!

bitbucket-sourcecode c# (with 1.5.0beta2 i've left it into the free world)
https://bitbucket.org/chmee/raw2cdng

actual release : raw2cdng 1.6.5 for windows
http://www.magiclantern.fm/forum/index.php?topic=5618.msg136306#msg136306

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: a1ex on May 23, 2013, 11:29:06 AM
1) https://bitbucket.org/hudson/magic-lantern/src/tip/src/raw.h . Don't ask me about endianness stuff, I've just scrambled these bits until it started to look good :P

ARM is little-endian.
Title: Re: The CinemaDNG Discussion
Post by: squig on May 23, 2013, 11:37:00 AM
lmao.

thx for coming chmee, save us from Adobe hell.
Title: Re: The CinemaDNG Discussion
Post by: CaptainHook on May 23, 2013, 02:02:08 PM
I figured out, that Resolve and Speedgrade cant handle 14Bit Raw-Frames! Just 8, 10 or 16Bit, maybe 12Bit. So, now i'm sitting on converting these 14bit to 16Bit-Chunks.

BMCC DNG's are 12bit log that unpack as 16bit linear in resolve.
Title: Re: The CinemaDNG Discussion
Post by: scrax on May 23, 2013, 09:40:48 PM
Good topic, since a lot of people seems to miss that compatibility.

My suggestion is (if possible), instead of making a converter for win, will not be better to make ML generate a raw with all what needed and make raw2dng able to convert directly to cinemaDNG? That way we will have a multiplatform solution, also 12 and 10 bit raw are under development so probably the converter should be able to handle different bitrates.
Title: Re: The CinemaDNG Discussion
Post by: chmee on May 23, 2013, 10:25:50 PM
of course. Right now the quest on 10/12Bit is the right way to get cinemaDNG(@10Bit) straight ooCam. finally, there will be a converter computerside - and the integration/rewriting of the cdng-header is the far smallest problem. I'm just sitting to solve the 16bit converting thang, maybe after that we will have a cdng-converter till we implement the idea into raw2dng. its the experience "we" are doing now.

ah, if someone misses the thread 10/12bit -> http://www.magiclantern.fm/forum/index.php?topic=5601.100

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: sicetime on May 23, 2013, 11:06:35 PM
I work in post, and have assimilate's scratch and resolve, I am ready and willing to test any post workflows that come out of this. Thanks for the great work, I think this one of the most exciting ml threads I have read!
Title: Re: The CinemaDNG Discussion
Post by: EOSHD on May 24, 2013, 01:25:50 AM
Here's how the ML DNG metadata differs to Cinema DNG (Blackmagic Cinema Camera)

http://www.eoshd.com/uploads/dng-metadata.png
(Blackmagic Cinema Camera EXIF on the left, Canon raw DNG on the right)

As you can see stills standard DNG EXIF is very different to Cinema DNG.

DaVinci Resolve seems to read the reduced-resolution metadata for 5D Mark III footage.

There seems to be other errors like JPEG compression for the full resolution image and uncompressed for the reduced resolution image?
Title: Re: The CinemaDNG Discussion
Post by: sicetime on May 25, 2013, 05:20:40 AM
Any update on this?
Title: Re: The CinemaDNG Discussion
Post by: EOSHD on May 25, 2013, 05:11:26 PM
My research shows the OP was correct, the problem is -

(+) IFD0-Entry is the jpg-Preview in 128x96px 8Bit - thats the one you see in Resolve.
(+) Raw-Data are packed into a subIFD - as in the DNG-Definition described.

Stills DNG has support for multiple images in one DNG at different resolutions (subfiles) and Cinema DNG only has one 'primary-image'. The raw data in the stills DNG is in a subfile, and seemingly the primary file is a thumbnail. Resolve reads the thumbnail data in IFD0-Entry but cannot access the raw data packed into a subIFD since it lacks support for this file structure.

Could we change Raw2DNG so subIFD becomes ifd0? I.e. pack the raw data into IFD0 instead of SUBIFD? Is that a trivial metadata change or more complex? There is a DNG SDK and Cinema DNG docs, someone just needs to understand the file format and write the correct structure into Raw2DNG's code.

I asked a Blackmagic guy via John Brawley about Resolve supporting stills DNG, and Pete confirmed it doesn't yet support DNG only Cinema DNG. I am sure they are working on it but probably quicker to change Raw2DNG.

This thread at EXIFTool was useful in pinpointing the problem - http://u88.n24.queensu.ca/exiftool/forum/index.php?topic=4927.0
Title: Re: The CinemaDNG Discussion
Post by: chmee on May 25, 2013, 08:40:09 PM
just to give you some food :) Here's the link to the app. (win)

*refresh* to 0.91
---
*lol* sorry, compiling while Championsleague-Finals is a bad idea.
0.92 - http://www.dslr-kleinanzeigen.de/Download/raw2cdng.0.92.zip

*!* NOT FUNCTIONAL*!*
(+) you need installed Microsoft .net 4.0 Runtime
(+) converting limited to 5 frames
(+) just for dev/analysing

problems so far:
(*) not enough time to solve, because workin' hard. pokin' around.. tuesday i'm back free on this thing.
(*) 14bit is functional with working image
(*) 16bit shows converting problems - have no time to finish successfully. (but is workable in resolve)
(*) both are not recognised by photoshop/acr, but viewable with fe irfanview. help appreciated (maybe colormatrix)
(*) for investigating on the converting problem try rawdigger, switch between the Sensel-channels.
(*) it looks like an array not NULLED in loop. rewriting the convert-function back to "simple" bitshifting and masking.

@EOSHD
if the IFDs would be changed, it wouldnt be no more a DNG. My Application throws out a CDNG with 14Bit - but they're not readable in Resolve Speedgrade etc pp.. The 16Bit-Files are readable, but not usable :)

For the devs here. Thats the converting loop: Do you see any Problems in it?
Code: [Select]
// c# code
// source - byte[] - rawstream 14bit
// chunks - int - stripByteCounts
// output -> Dest - byte[]

        public static byte[] to16(byte[] source, int chunks)
        {
            byte[] Dest = new Byte[chunks/14*16];
            UInt32 tt = 0;
           
            // because of bytestream i take 7 Bytes for (odd)RGRG (even)GBGB
            for (var t = 0; t < chunks; t += 7)
            {
                Dest[tt++] = (byte)(source[t] >> 2);
                Dest[tt++] = (byte)((source[t] << 6) | (source[t + 1] >> 2));
 
                Dest[tt++] = (byte)(((source[t + 1] & 0x3) << 4) | (source[t + 2] >> 4));
                Dest[tt++] = (byte)((source[t + 2] << 4) | (source[t + 3] >> 4));
 
                Dest[tt++] = (byte)(((source[t + 3] & 0xF) << 2) | (source[t + 4] >> 6));
                Dest[tt++] = (byte)((source[t + 4] << 2) | (source[t + 5] >> 6));
 
                Dest[tt++] = (byte)(source[t + 5] & 0x3F);
                Dest[tt++] = source[t + 6];
            }
            return Dest;
        }
regards chmee
Title: Re: The CinemaDNG Discussion
Post by: swinxx on May 26, 2013, 10:40:02 AM
i just started playing around with resolve 9 lite - mac version - and have to say that it is hands down the best color grading program i know.
and the best thing is: it is free. the limitation is 1920x1080, but thats ok for me. you can get really great results
the tracking function is so amazing and easy to use.

so i´m really waiting hard for some conversion tools that can convert the raw file from the camera into a raw format that can be used in resolve.

i wanna thank all the developers and enthusiast for all their commitment.
Title: Re: The CinemaDNG Discussion
Post by: g3gg0 on May 26, 2013, 01:23:54 PM
if you had posted the whole code, i would have tried to fix it :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on May 26, 2013, 11:53:40 PM
https://github.com/chmee/misc/tree/master/%5Bcsharp%5Draw2cdng

Sorry, i'm there..
Title: Re: The CinemaDNG Discussion
Post by: notdabod on May 27, 2013, 08:16:21 AM
Does anyone know of a Mac version for a cinema DNG converter?

Thanks
Title: Re: The CinemaDNG Discussion
Post by: squig on May 27, 2013, 08:39:05 AM
Does anyone know of a Mac version for a cinema DNG converter?

Thanks

There is no cinemaDNG converter yet.
Title: Re: The CinemaDNG Discussion
Post by: squig on May 27, 2013, 08:41:58 AM
I tried saving a DNG to a DNG in ACR and opened it in resolve, it was just thumbnail size but it was in colour.
Title: Re: The CinemaDNG Discussion
Post by: IliasG on May 27, 2013, 09:09:12 PM
Since the basic problem for Resolve is that it cannot read the subIFD info in exif and only reads IFD0, I tried to change the exif IFD0 info by hand and it does work but some crucial tags cannot be changed - deleted - moved .. blocked by exiftool's author for security reasons.

If it worked to the end the next step would be a scipt to automate the whole work and could be a interesting feature of the dng2raw + .. GUIs.

But as it is now I think a request to Phill Harvey to deblock changing those tags and even provide an option for exiftool to make easy these changes is the best I am thinking of. Anyone interested (I am not) can post a request ..

http://u88.n24.queensu.ca/exiftool/forum/index.php/board,4.0.html?PHPSESSID=61s2u55elgkbd4g06blqf67nr5

The other option is, the ML team to change the dng2raw code so that it writes the exif in the IFD0 subgroup looks more difficult to me because an expert in exif data is needed.
Title: Re: The CinemaDNG Discussion
Post by: IliasG on May 27, 2013, 09:16:18 PM
I tried saving a DNG to a DNG in ACR and opened it in resolve, it was just thumbnail size but it was in colour.

What you saw is the "preview" jpeg which is embedded in every DNG. It could be full size (DNG converter gives 3 size options .. full - medium - small) but even then you would have in Resolve jpegs not raws. It's because Resolve reads only the IFD0 exif info which in PhotoDNG describes the jpeg preview and the raw data are described - taged in the subIFD group.
Title: Re: The CinemaDNG Discussion
Post by: chmee on May 28, 2013, 09:41:38 AM
@IliasG
To be honest.. Did you built a DNG-File that opens in Resolve with 14Bit Data?
What TIFF-Tags do you want to change, you're not able to?

Edit: I tried to use smaller sizes (720x400px) for generating small sample files. Camera Raw behaves really strange with these data.. https://github.com/chmee/misc/tree/master/%5Braw%5Dsamples (look into the dng-metadata. the colormatrix shows an inf"inite" entry)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: IliasG on May 28, 2013, 02:26:12 PM
Hello chmee,

No I didn't build any DNG file, I stopped after I could not change the needed tags.

Blocked tiff tags. 0x0117 (StripByteCounts) - 0x111 (StripOffsets). As you understand these are the most significant tags ..

I cannot open any of your samples (ACR 5.7, Lightroom 4.4-demo, Dcraw,) nor read - change any exif tag with exiftool !!!... "Error: Not a valid TIFF (looks more like a XMP) - 1280_540.dng"

I think, You have to ask for help from Phil Harvey, he is THE EXPERT in exifs and a very good guy.

 
Title: Re: The CinemaDNG Discussion
Post by: chmee on May 28, 2013, 03:37:23 PM
StripByteCounts and StripOffsets.. thought that. :) It makes no sense to change them in EXIF-Tool. If you want to change them, take a hex-editor. Just not to be misunderstood - by now i dont generate the header, i just use another one and rewrite the important tags. it makes no sense rewriting the whole tiff-header-creation thing, thats already in the magiclantern package - but i m no c(++)-guy.

(*) The samples in the github are dng's converted with raw2dng.exe from 5D3-raws - no changes by me! strange, isnt it? these are raw samples on where we can start talking about - one database and changes in it.
(*) by now i am just interested in the converting from 14 to 16 bit. i m a bit puzzled, that just appending zwo bits to the core data didnt worked, but hey, (michael jackson) "you are not alone" - we're on the way to find a solution..
(*) by the way, irfanview tries his best to show strange data - and in my case it does..

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: IliasG on May 29, 2013, 02:29:47 AM
But if they (the samples) straight from raw2dng why there is no exif ??. Ohh now I see it .. corrupted download ..

Redowloading ..

Believe it or not I had not read this thread from the start :(

You improved alot !!.

I am looking at 704X400
When changing the bitdepth to 16 logic say you have to change StripByteCounts 
704*400*14/8 = 492800 while it is 504000 in exif !!.
704*400*16/8 = 563200

Don't know if StripOffsets need a change to 38400... as is the usual for raw2dng built DNGs

Title: Re: The CinemaDNG Discussion
Post by: chmee on May 29, 2013, 05:07:55 AM
Quote
704*400*14/8 = 492800 while it is 504000 in exif !!.
because its 720x400@14Bit -> StripByteCounts 504000. StripOffsets is a calculated value, where the raw-data starts. Look with a hexeditor into it.. we're talkin 'bout files coming straight from raw2dng.exe(!)

regarcs chmee
Title: Re: The CinemaDNG Discussion
Post by: IliasG on May 29, 2013, 02:08:39 PM
because its 720x400@14Bit -> StripByteCounts 504000. StripOffsets is a calculated value, where the raw-data starts. Look with a hexeditor into it.. we're talkin 'bout files coming straight from raw2dng.exe(!)

regarcs chmee

And where did 704 came from ???!!. This proves that if you are searching Exif tags instead of going to sleep you get blind :)

I have no experience nor the free time to start digging with hex editors ..

You mean that you removed the thumbnail and the subIFD header so the raw data start at 1536 ??.
Title: Re: The CinemaDNG Discussion
Post by: chmee on May 29, 2013, 11:20:26 PM
no. cinema dng and "simple" dng differs much - in terms of mandatory tags. i use a cdng-template, i merged myself from looking into another cdng-frames. just to clarify some things:

(*) dng cant be converted into cinema dng with only two or three another tags. but maybe these videoapps understand dng and dont need any cdng-tags?!
(*) main problem is, 14bit bayer is not useable for cinemadng, because no videosoftware can read 14bit frames as sequence. (my app is able to output 14bit cdng - but there is no software able handling it)
(*) dng batch converting is a "half baked" solution, because it ends in 8bit-sequences with loosing all benefits
(*) converting into tiff-rgb 16bit needs demosaicing and color matrix handling -> not my intention (ffmpeg?)
(*) my app (in actual state) does some weird things while converting to 16bit-bayer - and thats the point of "not-usability"

(+) the bayer structure is ok, because cinema dng can handle it, also the apps, that are handling cdng
(+) it must be converted into 10 or 16bit bayer. 10bit would be worse than 16bit, but definitively better than 8bit baked.
 
regards chmee
Title: Re: The CinemaDNG Discussion
Post by: hjfilmspeed on June 01, 2013, 07:09:31 PM
I feel like RAW to Cinema DNG GUI would be amazing!!!! Opening right up in the free version of resolve would be alot of fun!
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 02, 2013, 07:33:15 PM
Just to give you some hope :) The app is working, when i'm writing synthetic data (10, 12 and 16Bit Output) - but i'm stuck understanding the "simple" 14Bit-RAW-Data. Taking the short RAW-Example put on my server (https://github.com/chmee/misc/tree/master/%5Braw%5Dsamples (zipped raw samples 1280x540))

NEED YOUR HELP FOR THE LAST THING!

Example from 1280_540.raw - first 7 Bytes describe the first 4 Sensels ( RGRG)

Code: [Select]
hex - 40 29 42  c7 4c  9c 2a
dec - 64 41 66 199 76 156 42

binary:
| 01000000 | 00101001 | 01000010 | 11000111 | 01001100 | 10011100 | 00101010 |

as far as i understand restructuring to 14 bit goes:

| 01000000001010 | 01010000101100 | 01110100110010 | 01110000101010

dec - 4106 | 5164 | 7474 | 7210
BUT this data dont correspondend with the image i see in 14bit. its a uniform background - greyish on Blacklevel 2046, reddish with blacklevel 0. both pixels Pixel1(RG) and Pixel2(RG) have to be (more or less) same value, what is going on?


regards chmee
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 02, 2013, 11:19:31 PM
ok, here's a v0.99 for beta testing. thanks to reyalp for the hint.
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

all responses, hints are very welcome. regards chmee
Title: Re: The CinemaDNG Discussion
Post by: olik on June 03, 2013, 01:30:27 AM
Hello chmee,

this is very exciting. However, after converting a raw file that I created with the 5dmk2 without changing the bitrate, I could not open the dngs with resolve 9. It did not "see" the dngs. So I opened one dng with photoshop where I got presented a very contrasty image, barely recognizable without blue channel (will post dng tomorrow).
Title: Re: The CinemaDNG Discussion
Post by: arturochu on June 03, 2013, 05:37:03 AM
same problem as last post, i thought it would make a single cdng file but made a lot of dngs that i cannot open in resolve
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 03, 2013, 08:19:45 AM
thanks for response. actually i just tested with 5DIII-Files. So it would be fine you're leaving somewhere a short & small raw-file from your bodies. Ah btw. No, it is a filesequence with cinema-dng pictures (their ending is dng as well) - for using in resolve its mandatory changing to 16bit! olik, post a screenshot. (seems i have to delete the 14bit-converting :)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: gogauze on June 03, 2013, 09:36:51 AM
I did a quick test with my 5D3 and converting into 16bit, works like a charm. I'll be even more happy when one of the Mac folk ports it to OSX. I am sincerely impressed with your skill.
Title: Re: The CinemaDNG Discussion
Post by: squig on June 03, 2013, 10:06:19 AM
What's the increase in file size converting to 16bit?
Title: Re: The CinemaDNG Discussion
Post by: gogauze on June 03, 2013, 11:30:13 AM
Fairly negligible really. I think it increased by maybe 10% on the quick test that I ran.
Title: Re: The CinemaDNG Discussion
Post by: IliasG on June 03, 2013, 01:21:44 PM
@ Chmee,

Congratulations, and many thanks.

Regarding file size.
 
A good solution would be to store the data in the exported cine-DNG at 12bit log (after applying a LUT on the 14bit linear raw data) and then inject a inverse LUT (12bit log to 16bit linear) in the corresponding exif tag. It's the same as BlackMagick do and Nikon's "visually lossless" raws also. This would give a 12/16 file size.

What happens if we pass this 16bit cine-DNG through Adobe DNG converter. If it does not destroy the cine-DNG format is would give a respectable size decrease as by default it applies lossless compression which for such files can compress at about half the size.
Title: Re: The CinemaDNG Discussion
Post by: olik on June 03, 2013, 01:33:35 PM
Tested again for the 5dmk2, this time I changed the bitrate from 12 to 16bit. This solved my issue, I can now import to resolve. I have a strong green cast though, but it is perfectly correctable. Wonderful job! I love it :-)

Here the C-dng in resolve
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Flh5.googleusercontent.com%2F-4_0OEZ2yHis%2FUayBKvOC7NI%2FAAAAAAAANcY%2FkxVO0LMC8Ik%2Fs640%2FScreen%252520Shot%2525202013-06-03%252520at%2525201.32.46%252520PM.png&hash=d6aae600cffc777b6e9c879355caaa56)

vs h264 (same camera settings)
(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Flh4.googleusercontent.com%2F-DxK0xs3_MJU%2FUayBxqKFOTI%2FAAAAAAAANdA%2FX-Sp4nqpc-Q%2Fs640%2FScreen%252520Shot%2525202013-06-03%252520at%2525201.33.26%252520PM.png&hash=3e5d3f31fcfdfb53917d1878721a24b3)

and the first DNG of the sequence
https://picasaweb.google.com/lh/photo/tZ7PtaVRzHpdgX1PKcC_0aHmTw6ITH9KGra-l8uiJak?feat=directlink (https://picasaweb.google.com/lh/photo/tZ7PtaVRzHpdgX1PKcC_0aHmTw6ITH9KGra-l8uiJak?feat=directlink)
Title: Re: The CinemaDNG Discussion
Post by: a1ex on June 03, 2013, 03:28:16 PM
I have a strong green cast though

The black level is too high; it's probably hardcoded at 2048, which is usually right for 5D3, but too big for 5D2. The one from exif should be a good starting point.
Title: Re: The CinemaDNG Discussion
Post by: scrax on June 03, 2013, 05:29:18 PM
I can load the app in Osx with mono and load in a file, but how to start it?
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FC7Hscex.png&hash=85d5a38e7668469a705b979388d0b6dc)

EDIT, maybe it did all, I got a lot of dng but with some stranges names due to the win \ for folders I suppose

Also mono gives this error when loading it:
mono[75121] <Error>: kCGErrorInvalidConnection: CGSGetWindowTags: Invalid connection
Title: Re: The CinemaDNG Discussion
Post by: scrax on June 03, 2013, 06:07:09 PM
So far on osx when converting a raw file I got this file structure at the end:

rawfilefolder/M0000001.RAW
rawfilefolder\ML_16/   <--this is not a subfolder of rawfilefolder it's another empty folder
rawfilefolder\ML_16\ML_16_000000.dng
rawfilefolder\ML_16\ML_16_000001.dng
rawfilefolder\ML_16\ML_16_000002.dng
...  and so on with files having the win folder structure in the name.
Title: Re: The CinemaDNG Discussion
Post by: olik on June 03, 2013, 06:10:54 PM
While we already talk about using this software on different operating systems, I tried using it on Ubuntu Linux with mono, without luck;

Code: [Select]
mono '/raw2cdng.0.99.exe'
Missing method EnableVisualStyles in assembly raw2cdng.0.99.exe, type System.Windows.Forms.Application

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.
File name: 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
[ERROR] FATAL UNHANDLED EXCEPTION: System.IO.FileNotFoundException: Could not load file or assembly 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.

File name: 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

has anyone an idea?

Title: Re: The CinemaDNG Discussion
Post by: scrax on June 03, 2013, 08:19:26 PM
seems i have to delete the 14bit-converting :)
+1
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 03, 2013, 10:02:33 PM
thanks for the testers.

summarizing for the next versions
* >2GB filehandling
* path-strings uniform and no windows-dumb backslash
* pink fringing
* vertical lines (could be solved with 0xc619 or 0xc61b )
* 10 and 12 bit files (no logarithmic with LUT for now - think, there is a tag for it - linearizationTable 0xc618)
* batchmode?!
* handling other bodies with their metadata (colormatrix, whitelevel and so on)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: a1ex on June 03, 2013, 10:27:50 PM
* vertical lines (could be solved with 0xc619 or 0xc61b )

This is black level for each column, but for the banding issue we need EV correction for each column (mod 8 ). Do you know any tag for that?
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 03, 2013, 10:34:05 PM
ahh. ok. there is no metadata-array for ev correction - as i see that now.

Some Question there have been:

14 to 16Bit -> ~14.2% increase (16/14 = 8/7)
14 to 12Bit -> ~14,2% decrease(12/14 = 6/7)

@IliasG
No, logarithmic transforming is not my intention. for 12Bit i will cut LSB 2 Bit - we'll gonna see, how many data in the shadows is going to be lost - i quite think, its negligible :) Dont forget - the Magiclantern Team, all the coders around it gave you a weapon you never had the money for that - and this "techspec-more is better"- blahblah doesnt make a good film - your movie is not depending on logarithmic 16bit, but on "substance"!

@olik
what version-number of .net-framework is running in mono? it needs 4.0
Title: Re: The CinemaDNG Discussion
Post by: silwerfedlt on June 03, 2013, 10:47:28 PM
This is truly amazing and I´ve tested your tool shortly tonight with my 5D2 and I had to lower the blacklevel alot I think I had it on 1024 (just trying different settings to figure out how it works..). If it was on 2000 I had a very dark picture with alot of green. What would be the right setting for 5D2? Whys does it doffer from 5D3?

I´m new to resolve so I have to learn alot to be able to grade thorougly in the program but the basics are the same as the comp programs I use daily. (The Foundry Nuke and Eyeon Fusion).

Its so nice to be able to skip the Adobe programs - I don´t trust them at all for grading RAW sequences...


Cheers
Title: Re: The CinemaDNG Discussion
Post by: jtkleine on June 03, 2013, 10:53:03 PM
Just wanted to chime in, that I managed to open a file originating from 550D in Resolve. I did run raw2cdng under Parallels in OS X Lion, in 16bit mode of course. The file looked basically OK, but it was just something I shot in a minute. I couldn't do any actual testing, as the clip had horrific noise and no real black or white to begin with. Just wanted to see if it works at all. Sorry, I have not time to do anything more helpful right now.

BTW, I tried also in Speedgrade, just for fun. That didn't work at all (I didn't expect it to). The image was mostly green. I don't think Speedgrade even supports CDNG, maybe only Alexa and R3D.

Chmee you are doing as a big favor. No much point in grading raw video footage anywhere else than in Resolve -- at the moment at least. I tested earlier some clips in AE and graded them wit Color Finesse. That workflow didn't make much sense. I mean, to first bake the raw data into static rgb and then start grading. Very frustrating. So thanks, great stuff.
Title: Re: The CinemaDNG Discussion
Post by: olik on June 03, 2013, 11:04:35 PM
Quote
@olik
what version-number of .net-framework is running in mono? it needs 4.0

I have mono-runtime 2.10.8.1 but it seams that 4.0 is not even out, 3.0.10 was just released as a beta (http://www.go-mono.com/mono-downloads/download.html) am I missing sth? :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 03, 2013, 11:14:04 PM
@jtkleine - please leave a screenshot, what body you are using? maybe its a 12bit body? is some 12bit-Body able to raw-output? if you want to, just upload a short/small rawfile to analyse, whats happening.

http://dslr-kleinanzeigen.de/Tools/Body_Database/index.php?filter=prodeqcanon|rawbiteq12

@olik dont know much about mono - i just use linux-derivates for xbmc :)

generally, please wait till thursday, for now i'm working all day long - i cant do any changes.
Title: Re: The CinemaDNG Discussion
Post by: jtkleine on June 03, 2013, 11:40:31 PM
OK, I shoot a small clip and upload the raw file a little later. Where should I upload it? Maybe I just put in on a server and link it here. A screen shot too. But I don't have time today to fiddle with the black level settings in raw2cdng, so I just use the default.
Title: Re: The CinemaDNG Discussion
Post by: JulianH on June 03, 2013, 11:44:43 PM
How can I determinate the correct Blacklevel? (for the 50D). Default was at 2037 but that completely crushes the file. I just tried random numbers, around 1800 gives about the same exposure as Raw2DNG, but the colors are different when opened in ACR.

Anyway, it's workable, going to find out if my computer can keep up editing cinemaDNG in resolve... Thanks for the development!
Title: Re: The CinemaDNG Discussion
Post by: a1ex on June 03, 2013, 11:48:18 PM
By averaging the black areas from the DNG I've asked you to post a couple of times...

http://www.magiclantern.fm/forum/index.php?topic=5586.msg44299#msg44299
Title: Re: The CinemaDNG Discussion
Post by: noisyboy on June 04, 2013, 12:04:16 AM
What a cool day! Magic Lantern community making breakthroughs all day long like an absolute BOSS  8) Amazing work!

Thanks for your hard work on this man, I can't wait to finally play with my footage in Resolve... Awesome :)
Title: Re: The CinemaDNG Discussion
Post by: mrwolf on June 04, 2013, 12:30:27 AM
Thanks for the tool chmee! My footage looked good in resolve.

One question do you set the black level and white level from the raw file or is this a default setting? Should I change this value to suit the 550D? I assume this value is in the EXIF somewhere does it depend on the footage or is it always the same?

In other tools such as RAWanizer, using the tiff files, there was a purple cast and it clipped the top of the whites. While using your tool it cam out fine and I was able to access the camera raw tools in resolve which I couldn't do before.
Title: Re: The CinemaDNG Discussion
Post by: JulianH on June 04, 2013, 12:42:47 AM
By averaging the black areas from the DNG I've asked you to post a couple of times...

http://www.magiclantern.fm/forum/index.php?topic=5586.msg44299#msg44299
Sorry a1ex, there is no Silent DNG option on all of the 50D builds I've been using.
I'd gladly help out, but in this case, I don't know how.
Title: Re: The CinemaDNG Discussion
Post by: a1ex on June 04, 2013, 12:53:08 AM
Ask the guys to enable it in features.h.
Title: Re: The CinemaDNG Discussion
Post by: jtkleine on June 04, 2013, 01:39:42 AM
@chmee, I'm shooting with 550D. I suppose it's a 14bit body.

Here's a link to a raw file:
http://kleine.me/clients/mlraw/M04-0102.RAW (http://kleine.me/clients/mlraw/M04-0102.RAW)

Screenshots from Resove & Speedgrade:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fkleine.me%2Fclients%2Fmlraw%2Fresolve_screenshot.png&hash=61fd109980cbb107b147381d72e92420)
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fkleine.me%2Fclients%2Fmlraw%2Fspeedgrade_screenshot.png&hash=37b04a02434fcba69bae96411b5fd4cd)
Title: Re: The CinemaDNG Discussion
Post by: silwerfedlt on June 04, 2013, 08:35:54 AM
By averaging the black areas from the DNG I've asked you to post a couple of times...

http://www.magiclantern.fm/forum/index.php?topic=5586.msg44299#msg44299

So I should do the same for my 5D2? Since the black and white levels differs from 5D3..
Or are there any precise nr for the 5D2 I should use?

Title: Re: The CinemaDNG Discussion
Post by: a1ex on June 04, 2013, 09:22:33 AM
For 5D2, the black value from raw footer should be OK. I don't know if this app uses it or not.

If not, convert the raw with raw2dng and look in the exif of one DNG file.
Title: Re: The CinemaDNG Discussion
Post by: IliasG on June 04, 2013, 02:18:11 PM
....
@IliasG
No, logarithmic transforming is not my intention. for 12Bit i will cut LSB 2 Bit - we'll gonna see, how many data in the shadows is going to be lost - i quite think, its negligible :) Dont forget - the Magiclantern Team, all the coders around it gave you a weapon you never had the money for that - and this "techspec-more is better"- blahblah doesnt make a good film - your movie is not depending on logarithmic 16bit, but on "substance"!
...

I wouldn't have any objection for 14to12bit linear by truncating ..but on the single sample that I have seen with this method there is posterization (middle bottom on the windows frame).
http://www.magiclantern.fm/forum/index.php?topic=5601.msg39482#msg39482

Will you also truncate the "Black Level" value in exif ?.

Don't get me wrong, I am pushing this Log conversion in raw2dng as a first test to try it's effectiveness and if for example 14to12 or 10bit log will be proved of good quality ML team could use this strategy in-camera to improve the recording module for longer and/or faster fps shots ..

I can calculate and upload (within 24h) what I think are optimum LUTs (compress and linearize) if needed.   
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 04, 2013, 09:02:50 PM
(*) IliasG, if you're interested in implementing logarithmic scale in every kind of transcoding-process, you're welcome. (the reason why ACR didnt open the file was, the whitelevel was higher than the possible maximum output - if there was for example whitelevel 15.000 - its 12 bit overflow error). and of course the black- and whitelevel have to be corrected. so it means for me, recalculating with lin16bit->log12bit-table and embedding log->lin-LUT into cdng.

(*) i am not so sure, if recalculating in body is possible - as i understand, the possibility to record the sensoroutput in realtime is reasoned by shipping data stright to the dma, without digic and other computing-helper..

(*) jtkleine - i've seen this in my testversion of speedgrade as well - for now i have no clue, why its handling different.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: roughstudios on June 04, 2013, 09:13:13 PM
Hi everyone, fantastic work so far, big up.

I've had some time to experiment with the firmware, trying to find a workflow that works for FCPX and Resolve. raw2cdng is great but will be amazing when there's a batch version and hopefully mac also :D

Anyway, for me workflow is the thing that will decide if I'll be able to use the firmware or not, and now raw2cdng is getting close! Was trying all kinds of ways to get a cdng before but it was painful.

So now when I used raw2cdng all was good getting into Resolve, but then the issues started. First take a look at the video:


So a couple of things.

- There's a lot of what I would say is chromatic aberation. Pinkish especially in the trees. This could be a white balance thing combined with the sunset highligt, bit my best guess is that something either happened in the RAW recording or the raw2cdng conversion. Anyone with the same issues?

- Then the workflow. One word: painful. I think the timecode is breaking the connection with resolve, because there wasn't a way to get it to automatically connect to the old files. It has done in my previous tests but they were much smaller and without raw2cdng. So maybe something has gone missing in the raw2cdng conversion? All is ok until the XML is going back to Resolve from FCPX.

Any thoughts on the issues?

And keep up the fantastic work!
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 04, 2013, 10:10:50 PM
(*) pink fringes - it seems, its a canon problem in a strange way. other apps are handling that. it may be, there are some highcontrast-points higher than whitelevel point, and these are badly interpreted - so i have to look into another solutions, whats the best. hard limiting sensel values over whitelevel-value? and so on. (its on the todolist)

(*) timecode. hmm. could you post a xml-file in pastebin? just to see, what strange data there are? (i am win-user, so i have no fcpx)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: yobarry on June 04, 2013, 11:42:43 PM
I see a lot of people saying they want to be able to grade in Resolve with CinemaDNG, and I understand it'd be more convenient that way, but are you losing any data when you grade with converted TIFF files from ACR?
Title: Re: The CinemaDNG Discussion
Post by: roughstudios on June 04, 2013, 11:56:01 PM
I see a lot of people saying they want to be able to grade in Resolve with CinemaDNG, and I understand it'd be more convenient that way, but are you losing any data when you grade with converted TIFF files from ACR?

I wouldn't say you loose that much data, it's negligible, you would loose all the benefits of not having processed the image though. Everything from white balance, noise reduction, sharpening would bring out more artefacts etc. But it would probably very subtle.

The biggest reason though is workflow, TIFF is at least 3 times as much space on your hard drive from a RAW file. If in 16 bit probably even more.

There's no way I would think that's a sustainable solution, even though at the moment TIFF is the safest workflow since it's reliable in conversion from the RAW file. Which the CinemaDNG isn't yet, but it will be!

(*) timecode. hmm. could you post a xml-file in pastebin? just to see, what strange data there are? (i am win-user, so i have no fcpx)
I'll try to post both a xml with video that works with timecode and one of the broken ones tomorrow. Thanks for the great development!
Title: Re: The CinemaDNG Discussion
Post by: yobarry on June 05, 2013, 12:13:44 AM
Thanks for the info

I wouldn't say you loose that much data, it's negligible, you would loose all the benefits of not having processed the image though. Everything from white balance, noise reduction, sharpening would bring out more artefacts etc. But it would probably very subtle.
I'm a little unsure of the difference between raw and TIFF, I thought TIFF was just a container, so it just 'held' all the raw data, changing nothing?

For instance if you set a white balance in ACR and convert to a TIFF, is that white balance still 'raw' or is it baked in? And if you adjusted nothing and put sharpening/NR to zero, does the TIFF file still contain the extra data (exposure leeway) that you could have adjusted in ACR?
Title: Re: The CinemaDNG Discussion
Post by: CaptainHook on June 05, 2013, 02:34:15 AM
- There's a lot of what I would say is chromatic aberation. Pinkish especially in the trees. This could be a white balance thing combined with the sunset highligt, bit my best guess is that something either happened in the RAW recording or the raw2cdng conversion. Anyone with the same issues?

Resolve wont do any chromatic aberration fixing to cDNG's like ACR etc will. I see CA with BMCC DNG's as well. You need to fix it yourself in resolve.
Title: Re: The CinemaDNG Discussion
Post by: roughstudios on June 05, 2013, 04:54:05 PM
Resolve wont do any chromatic aberration fixing to cDNG's like ACR etc will. I see CA with BMCC DNG's as well. You need to fix it yourself in resolve.
I know, that's why it needs to be fixed in the conversion :)

Personally I would choose a BMCC any day over the 5D if the workflow means going into manual RAW processing, then converting to CDNG to get into resolve and then edit then back for grading. It just isn't worth the time spent fixing. It might work for commercial stuff, but for documentary there's just too much material to process.
Title: Re: The CinemaDNG Discussion
Post by: Irri on June 05, 2013, 05:44:09 PM
I'm also getting the pink fringes. It really shows up when adding sharpening in resolve, and goes green when using blur. It seems to be in high contrast areas. I thought it could be related to not down sampling from the raw Bayer data, but a down sample didn't seem to help.

Title: Re: The CinemaDNG Discussion
Post by: shawnwytch on June 05, 2013, 05:51:21 PM
I was watching a tutorial yesterday it was covering how u can change how resolve see the footage. So it looks how u shot it, I'm at a editing job now when I get home I'll post link
Title: Re: The CinemaDNG Discussion
Post by: EOSHD on June 05, 2013, 11:50:41 PM
Guys is the pink fringing the same as what I reported to chmee? Here's how I see it...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.eoshd.com%2Fuploads%2Fpink-dots-and-vertical-lines.jpg&hash=6a5e1072e707410495acab6f03a44f34)

The standard Raw2DNG frames don't have this. What is that code doing differently?
Title: Re: The CinemaDNG Discussion
Post by: sergiocamara93 on June 06, 2013, 12:14:52 AM
Guys is the pink fringing the same as what I reported to chmee? Here's how I see it...
I think it is. It can seem a "fringe" around contrasty areas of the image. I've tried changing to BMC Film interpretation in the Camera Raw tab and it "solves" it. But I don't really like how it re-interpret it as BMC.

Thanks a lot for this tool chmee!!! I'd love to have batch enconding and renaming but it's already great to been able to do direct RAW->CDNG processing.
Title: Re: The CinemaDNG Discussion
Post by: Irri on June 06, 2013, 12:31:53 AM
Here's what I get...

No sharpening (This actually looked fine until ImageShack added compression:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fimageshack.us%2Fscaled%2Fthumb%2F842%2Fscreenshot20130605at231.png&hash=adc86c832c21a388d788e56666d89c00)

+5 sharpening:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fimageshack.us%2Fscaled%2Fthumb%2F46%2Fscreenshot20130605at231.png&hash=e74592515216769a5cac14d63d34733b)

-5 sharpening:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fimageshack.us%2Fscaled%2Fthumb%2F801%2Fscreenshot20130605at231.png&hash=5ceb874064489eb7515e5de468293f30)
Title: Re: The CinemaDNG Discussion
Post by: DaveAbbott on June 06, 2013, 12:32:48 AM
Hey guys,

I've converted a few clips to cinema dng but they're not showing up in DaVinci file browser. Any ideas?

If I use the normal raw2dng app I can see the sequence in DaVinci, but it looks like balls. If I use raw2cdng the sequence doesn't appear at all and the folder presents as though it's empty.

Thanks in advance!
Title: Re: The CinemaDNG Discussion
Post by: Irri on June 06, 2013, 12:42:06 AM
Dave, have you tried setting it at 16 bit instead of 14 bit (in raw2cdng)? That worked for me.
Title: Re: The CinemaDNG Discussion
Post by: DaveAbbott on June 06, 2013, 01:51:55 AM
Amazing! Thanks Irri!

Now all I need to do is to solve my black point issue and I can delete 800GB of .TIFs and give Lightroom a breather.
Title: Re: The CinemaDNG Discussion
Post by: Irri on June 06, 2013, 01:56:41 AM
Don't thank me; there are people doing fantastic work here, and I'm not one of them :)

Glad you got it working.
Title: Re: The CinemaDNG Discussion
Post by: Oedipax on June 06, 2013, 04:09:28 AM
I can't get raw2cdng running under Mono on my Mac, either. I'm running the latest version of Mono (just installed from Macports). No luck in CrossOver, either.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 06, 2013, 10:46:15 AM
@IliasG
after some thoughts and discussions i think, it makes sense to lower the bitamount without losing any informations - so i would give the logarithmic idea a try. The final challenge would be to get the rawdata in a 10bit@log space and compressing it - because we can :)

(small patchversion is on the way - just got a job today, so i think the "bigger" patches will be on weekend)
Title: Re: The CinemaDNG Discussion
Post by: squig on June 06, 2013, 11:53:40 AM
I can't get raw2cdng running under Mono on my Mac, either. I'm running the latest version of Mono (just installed from Macports). No luck in CrossOver, either.

We won't have to wait much longer http://www.magiclantern.fm/forum/index.php?topic=6218.msg46920;topicseen#msg46920
Title: Re: The CinemaDNG Discussion
Post by: scrax on June 06, 2013, 12:48:26 PM
I can't get raw2cdng running under Mono on my Mac, either. I'm running the latest version of Mono (just installed from Macports). No luck in CrossOver, either.
Don't install with macport, use the precompiled binary from mono site, that worked for me. I've posted the link in raw2dng.app topic
Title: Re: The CinemaDNG Discussion
Post by: IliasG on June 06, 2013, 03:09:11 PM
@IliasG
after some thoughts and discussions i think, it makes sense to lower the bitamount without losing any informations - so i would give the logarithmic idea a try. The final challenge would be to get the rawdata in a 10bit@log space and compressing it - because we can :)

(small patchversion is on the way - just got a job today, so i think the "bigger" patches will be on weekend)

I was away from my PC last week (and I'll be away again) so I could not give any try on this but today I'll be at home ...

Just a summary of my thoughts for the log curve.

We can clip most of the unused "below black point" levels. Now it is around 2048/14bit, we can keep 64 or 32 of them (or less) if we want a footroom just in case something goes wrong with Black Point. and say 64 or 32 over the Black Point.

It's desirable to keep linearity at the darks.
So for 14to12bit the lowest (say 32 below black + 32 ) could be mapped 1:1 giving a 4X slope at the linear part and then start the log part.

For 14to10bit it is a bit more complicated as a 16X slope would needed for 1:1 mapping but this way we will have heavy compression at highlights. I am thinking of an intermediate solution, slope less than 8X ( 8X gives levels density equivalent to 13bits) and maybe use smaller linear part and less (or zero)"below black" levels.

Anyway we can try for a start with rec.709 curve (linear with slope 4.5 and then gamma=2.22)
http://www.poynton.com/notes/colour_and_gamma/GammaFAQ.html#gamma_correction
Title: Re: The CinemaDNG Discussion
Post by: fatpig on June 06, 2013, 10:08:45 PM
@chmee:

As soon as you implement Commandline, I will be happy to include it in BATCHelor. :)
(it's sticky'd if you don't know it.)
Is it planned?

Still need a 5DII .RAW File? I have one.
Title: Re: The CinemaDNG Discussion
Post by: danielcreed on June 06, 2013, 10:41:38 PM
don't know if this will help but it states here that black level is 1023 and white level is 15600

http://lclevy.free.fr/cr2/

as far as the 5d Mark 2 goes

Title: Re: The CinemaDNG Discussion
Post by: a1ex on June 06, 2013, 11:12:53 PM
The values change depending on video mode; 1023 is just a rough starting point. On the same link, it says that black point is found by averaging the black borders from the image (which ML does).
Title: Re: The CinemaDNG Discussion
Post by: IliasG on June 07, 2013, 09:26:20 AM
...

Anyway we can try for a start with rec.709 curve (linear with slope 4.5 and then gamma=2.22)
http://www.poynton.com/notes/colour_and_gamma/GammaFAQ.html#gamma_correction

Quick and dirty calculations in Excel spreadsheet for rec.709 curves.

Input is a typical Canon raw, 14bit linear. Clipped at -31 below the Black Level, mapped to 10bit gamma rec.709.

The inverse mapping (linearization table) which should be injected in exif tag is from 10bit gamma to 16bit linear with Black Level at 141/16bit.

http://rapidshare.com/files/3859944133/curves-14bit_to_10bit_gamma-rec709.xlsx

I changed slightly the offset to fight a discontinuity at the crosspoint.

Gives a good balance between the linear part and highlight compression. The levels's density at the darks is a bit better than 12bit linear while keeping highlight compression robust enough to withstand heavy grading (300 levels for the last stop).
Title: Re: The CinemaDNG Discussion
Post by: hjfilmspeed on June 08, 2013, 06:45:35 AM
I know it would be an extra step but are there any dng to cinema dng converters?
Title: Re: The CinemaDNG Discussion
Post by: zach915m on June 09, 2013, 07:38:19 PM
I was able to successfully convert a couple DNG files to 16 bit DNG and now it only does 14 bit files and then when trying to upconvert to 16 bit I get this error message:

"index was outside the bounds of the array"

Any idea how to fix this? 

Thanks for any help.

Zach
Title: Re: The CinemaDNG Discussion
Post by: budafilms on June 10, 2013, 09:03:06 AM
RAWMAGIC VS RAW2DNG

Same shot: Build June 6 1920x720 25p no skipped frame. 5D MARK III. Scandisk 60 mb/s

- RAWMAGIC (beta4) open in DaVinci Resolve setting withe balance tungsten - exported as pro res HQ
- RAWTODNG (0.11) open in ACR setting tungten exported from compressor as pro res HQ

Nothing more applied, any corrections, all filters desactivated.

Imported both to FINAL CUT X and exported as h264 1920x1080

Facts:
- the more saturation and black levels in RESOLVE (appears to be sharpness).
- Magenta dots in RESOLVE (See the hair)
- blurry in ACR - maybe less black level

I really don´t know if is RESOLVE whocreates de magenta o RAWMAGIC

The video to download:
https://www.dropbox.com/s/vjrfigp0xi4t2pi/prueba%20RAW%20MAGIC%20vs%20RAW2DNG.mov
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 10, 2013, 01:01:01 PM
just a short sign of life of me. just got the flu (in summer, hate it).
brain apparatus is on 30%-can and -10%-want-strength :)

(*) batchmode is coded
(*) most code is rewritten
(*) no more 14bit-output, because not needed
(*) blacklevel and whitelevel read from raw-data
(*) colormatrix read from raw-data

(+) sitting on 12bit linear mode - after finishing that, exe will be published

by now
(-) no adjustments to new stripesize-changes (extremly urgent on list)
(-) no 10bit logarithmic (urgent on todolist) (thx iliasg, appreciate your work) (think, we have to hide the 14bit in 16bit-words - we know, resolve cant handle 14bit)
(-) banding issue for 5diii-sources (urgent) (thx to thomas worth)
(-) pink fringes (investigating) (its a canon-kind problem - adobe is handling that manually so to say)
(-) commandline-mode (in one of the next versions)
(-) compression (will come after 10bit logarithmic)

@zach - two days ago, a1ex changed the bytestripelength - did you used some "abnormal" picture-sizes? have to update my 5diii with the new code to investigate..

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: zach915m on June 10, 2013, 09:56:17 PM
Hey Chmee,

I believe it must have something to do with what Alex did to the bytestripelength as I tried this further today on two other systems and they both came up with the same error.  My only change in workflow was using the most recent version of Magic Lantern on my 5D Mark III instead of the late May version I was using before.  All footage was shot at 1920 by 1080.

Sorry to hear about you getting the flu that is bad news!  Feel better!
Title: Re: The CinemaDNG Discussion
Post by: tin2tin on June 11, 2013, 08:51:01 AM

(*) batchmode is coded
(*) most code is rewritten
(*) no more 14bit-output, because not needed
(*) blacklevel and whitelevel read from raw-data
(*) colormatrix read from raw-data

(+) sitting on 12bit linear mode - after finishing that, exe will be published
Great to hear you're making progress. Thank you for investing your time in this much needed tool. Now you have succeeded with the 14 to 12 bit conversion, I can't help my curiosity on the question of this process is too heavy to implement in camera or not(as discussed in the 12/10 bit conversion thread), but I guess it is premature to ask those questions.  ;)
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 11, 2013, 12:12:40 PM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2F_raw2cdng_10.jpg&hash=1c30d7bf35beff5d448f2cc80bd2f486)

version 1.0 is out - you know. its beta - and dont be too much optimistic :) the 12bit mode is built because of the easy way and a hope, that cutting the lsb 2 bits are not that visible :) the result is still far better than the "old" canon output.

@tin2tin:
the conversion from 14 to 12 bit is not that big work - but it needs some memory and some bitshifting and masking. my assumings are, if between sensor and dma-controller is no "processing-unit" able to handle that. hmm. the first approach was, to drive all the sensordata thru the digic - and this was the bottleneck.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: a1ex on June 11, 2013, 12:27:57 PM
Quote
lower whitelevel by 15%

On what cameras is this needed? If raw2dng has pink highlights too, it's a bug in ML and you should report it.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 11, 2013, 12:42:34 PM
so far i understand the pink highlights problem - its no problem of data handling but of managing the "beyond whitelevel sensels". the checkbutton is a fast built in tester if its changing something. if you search after "canon raw pink highlights" you (we) see, its a problem since years - manually managed by the raw-converters - and the dcraw-community is the only one trying to solve that by explaining the problem.

example links:

may 2012 - http://sourceforge.net/p/ufraw/discussion/434060/thread/b43582e3/
march 2009 - http://www.luminous-landscape.com/forum/index.php?topic=32942.0
feb 2010 - http://www.dpreview.com/forums/thread/2748822
oct 2011 - http://code.google.com/p/rawtherapee/issues/detail?id=1037
dec 2009 - http://www.flickr.com/groups/photomatix/discuss/72157623070526860/

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: a1ex on June 11, 2013, 12:57:15 PM
In photo mode (CR2): white level depends on digital ISO, e.g. at 160, 320 and so on it's 1/3-stop lower than at full-stop ISO. Oddly enough, at ISO 250 it's not higher than at ISO 200.

In movie LiveView: digital ISO does not affect raw data, so the white level is always the same. The level from exif should be correct.

In photo LiveView, ML alters the white level to adjust the histogram so it matches the CR2 one. That's one of the reasons why you should not record raw video in photo mode.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 11, 2013, 01:05:47 PM
.In photo LiveView, ML alters the white level to adjust the histogram so it matches the CR2 one. That's one of the reasons why you should not record raw video in photo mode.
So. meaning, in m and videomode it shouldnt happen?
Title: Re: The CinemaDNG Discussion
Post by: a1ex on June 11, 2013, 01:08:26 PM
M is photo mode, movie mode is movie mode. Don't mix them.
Title: Re: The CinemaDNG Discussion
Post by: tin2tin on June 11, 2013, 01:18:51 PM
A quick test with a file from a 60D in Movie mode, a couple of ML builds back: http://we.tl/4Mv47vaiRS

There seems to be a bug in 12 bit + 15% unchecked which makes the screen black with green monocrome highlights.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 11, 2013, 01:23:41 PM
the moviemode reacts on the mode-dial.. so, the only 100% functional way is to set on moviemode AND mode-dial on m.
now properly said?

@tin2tin investigating - made a dumb mistake (last minutes) lowering the blacklevel by 4 not by 2 should be the right way. on the way. thx for the raw.
Title: Re: The CinemaDNG Discussion
Post by: a1ex on June 11, 2013, 01:26:28 PM
Yes, on 5D3 you need to set both.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 11, 2013, 02:07:21 PM
fixed. v1.1.1 is out. http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

just now playing with the newest build (http://www.magiclantern.fm/forum/index.php?topic=6362.0) to examine the stripByteCounts-Thing. fixed in 1.1.2

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: Jakobmen on June 11, 2013, 09:30:16 PM
fixed. v1.1.1 is out. http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

just now playing with the newest build (http://www.magiclantern.fm/forum/index.php?topic=6362.0) to examine the stripByteCounts-Thing. fixed in 1.1.2

regards chmee

THANKS! Chmee

to bad these files CinemaDNG don't open in Avid media composer
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 11, 2013, 10:10:46 PM
@jakobmen  :o >:( is there any log or error-message? its my concern to see that playing in media composer as well - did you try the mac-version from thomas worth? is his cdng working?

handling bmcc cinema dng
Quote
Currently, you cannot import the camera raw files directly into Avid Media Composer, Apple Color or Adobe Premiere Pro.
it cant handle bmcc-cdng-files? is that still so?

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: Jakobmen on June 11, 2013, 10:16:57 PM
@jakobmen  :o >:( is there any log or error-message? its my concern to see that playing in media composer as well - did you try the mac-version from thomas worth? is his cdng working?

regards chmee

My bad bro im dumb! avid does not open CinemaDNG.. eheh
Title: Re: The CinemaDNG Discussion
Post by: IliasG on June 12, 2013, 12:47:47 AM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2F_raw2cdng_10.jpg&hash=1c30d7bf35beff5d448f2cc80bd2f486)

version 1.0 is out - you know. its beta - and dont be too much optimistic :) the 12bit mode is built because of the easy way and a hope, that cutting the lsb 2 bits are not that visible :) the result is still far better than the "old" canon output.

@tin2tin:
the conversion from 14 to 12 bit is not that big work - but it needs some memory and some bitshifting and masking. my assumings are, if between sensor and dma-controller is no "processing-unit" able to handle that. hmm. the first approach was, to drive all the sensordata thru the digic - and this was the bottleneck.

regards chmee

chmee, thanks for the new version, glad you recovered fast. :)

12bit linear should be OK since there is more than 2bits noise as metered on Black Frames (>6 14bit raw levels). Theoretically, only ISO160 could suffer from this in a case of heavy processing.
But this truncation makes me nervous .. how did you managed to make it better than "old"Canon output ??. Waiting for tests ..
Any way, after you implement 10bit log we can expand using LUTs for 12 bit log ..
Title: Re: The CinemaDNG Discussion
Post by: IliasG on June 12, 2013, 01:18:58 AM
so far i understand the pink highlights problem - its no problem of data handling but of managing the "beyond whitelevel sensels". the checkbutton is a fast built in tester if its changing something. if you search after "canon raw pink highlights" you (we) see, its a problem since years - manually managed by the raw-converters - and the dcraw-community is the only one trying to solve that by explaining the problem.

example links:

may 2012 - http://sourceforge.net/p/ufraw/discussion/434060/thread/b43582e3/
march 2009 - http://www.luminous-landscape.com/forum/index.php?topic=32942.0
feb 2010 - http://www.dpreview.com/forums/thread/2748822
oct 2011 - http://code.google.com/p/rawtherapee/issues/detail?id=1037
dec 2009 - http://www.flickr.com/groups/photomatix/discuss/72157623070526860/

regards chmee

In fact the problem comes from Dcraw which uses a single (many times wrong) value for White Level while WL floats depending on ISO settings (different for "integer" ISOs and "intermediate" ISOs) and on len's f-stop (there is digital amplification for larger than f/2.8 settings pushing the WP higher).

With DNGs the solution could be easy if ISO & f-stop was reported because there is an exif tag for WL, raw2dng/raw2cdng can take care of it.

For native raws (*.CR2) there is a need from the decoders side for a database as will be implemented by RT team .. http://code.google.com/p/rawtherapee/issues/detail?id=1752
I am sure Adobe uses a database also with rather conservative settings and as far as I know in DNG conversions they do not increase WL for open f-stops, not a big issue .. just a slight loss of highlights.

As it is now, ML uses the conservative settings that we see in Adobe's DNG conversions and I think they are safe, but we need a decoder to cooperate and maybe it's Resolve's error interpreting the exif tags wrongly .. like RT did a while ago https://code.google.com/p/rawtherapee/issues/detail?id=1695
Title: Re: The CinemaDNG Discussion
Post by: Jakobmen on June 12, 2013, 02:18:38 AM
@jakobmen  :o >:( is there any log or error-message? its my concern to see that playing in media composer as well - did you try the mac-version from thomas worth? is his cdng working?

handling bmcc cinema dngit cant handle bmcc-cdng-files? is that still so?

regards chmee


Chmee im trying to do this

raw2cdng to Resolve works great..

resolve to Avid media composer works great..

BUT when coming from Avid back to resolve its not linking right, its only linking the first clip in the edit and replace the other clips with that first clip..

could it be a file name problem or something in your code when making the CDNG files ?

here's what im trying to do
https://www.youtube.com/watch?feature=player_embedded&v=Kpg0k3aD088

english is not my first so forgive me..
Title: Re: The CinemaDNG Discussion
Post by: iunknown on June 12, 2013, 06:09:24 AM
chmee, incredible work. Jakobmen, I don't think its a file naming issue.  I get a "does not support image sequence" error exporting an xml out of resolve.  It doesn't do this with cinema dng's from the black magic camera.

I guess resolve 10 will be out soon, and looks like we'll have enough tools to cut in resolve, rather then round-tripping to premier.  So raw2cdng might be the easiest workflow in the long run.
Title: Re: The CinemaDNG Discussion
Post by: zach915m on June 12, 2013, 07:27:59 AM
Same thing happened to me with the first clip linking to all in the resolve/avid workflow.  Just found this which seems like a much more proper workflow then using proxies:


Worth a watch and I believe this will be better.  Trying it tomorrow.  Unless this method is what you already did?
Title: Re: The CinemaDNG Discussion
Post by: zach915m on June 12, 2013, 09:17:12 AM
Well I tried the above method I just posted numerous times as well and am having the same problem.  All of the separate clips in the AAF only link to the first DNG file instead of the separate ones that were assigned.  It works if I use the proxies but I can't get back to the DNG's.  Anyone with a fix or is this in the code?
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 12, 2013, 09:25:28 AM
@iliasG "how did you managed to make it better than "old"Canon output ??"
umph.. maybe wrong said - old means the original-video-output via h.264-mov.

xml-problems - there are timecode and framerate tags for cdng - i do not use them - but maybe its worth to activate them :)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: tin2tin on June 12, 2013, 11:02:10 AM
If there is a reel name tag for cdng too, I would suggest using the embedded file creation time as time code and the date as reel name as defaults(most softwares only read first 8 characters of reel name tags).
Title: Re: The CinemaDNG Discussion
Post by: Jakobmen on June 12, 2013, 02:42:44 PM
@iliasG "how did you managed to make it better than "old"Canon output ??"
umph.. maybe wrong said - old means the original-video-output via h.264-mov.

xml-problems - there are timecode and framerate tags for cdng - i do not use them - but maybe its worth to activate them :)

regards chmee


thats what i was thinking.. Thank you again for your work you do..
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 13, 2013, 10:51:02 AM
@IliasG
Just to be on the right track.. Your ConversionLUT 10<->14bit you linked.
on left side is the lin->log, on the right side the inversion log->lin?!
Title: Re: The CinemaDNG Discussion
Post by: IliasG on June 13, 2013, 01:04:40 PM
@IliasG
Just to be on the right track.. Your ConversionLUT 10<->14bit you linked.
on left side is the lin->log, on the right side the inversion log->lin?!

Exactly.
14bit linear (A) to 10bit log (H) is on the left.
 Just fill in columnA the clipped values (<2017) all mapped to zero in columnH.

As you can see I forgot to adapt columnA to a variable Black Level .. just used the typical for 5DIII.
Normally I should start from the full 14bit range, clip 31 values lower than Black Level (which is variable) for columnB ...

10bit log (J) to 16bit linear (Q) is on the right. With red color (141) is the 16bit black Level.

It was very dirty calculation sorry.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 13, 2013, 07:59:51 PM
when the children are sleeping i will try to realise the 10bit|log :)
(Reelname-Tag is built in as well as changed sequencenames- we'll test, if this lasts for solving the xml/edl-problems)

by the way - do you all know the hunterLUT for resolve? as seen in the vimeo-example (https://vimeo.com/67970827), its doin' a good job. (downloadlink is found in the vimeo-description)

regards chmee

Title: Re: The CinemaDNG Discussion
Post by: Jakobmen on June 13, 2013, 08:04:03 PM
when the children are sleeping i will try to realise the 10bit|log :)
(Reelname-Tag is built in as well as changed sequencenames- we'll test, if this lasts for solving the xml/edl-problems)

by the way - do you all know the hunterLUT for resolve? as seen in the vimeo-example (https://vimeo.com/67970827), its doin' a good job. (downloadlink is found in the vimeo-description)

regards chmee

I don't think is matters but I use AAF

Thanks!
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 14, 2013, 01:56:00 PM
v1.1.6 is out - its the first attempt to fix the edl/xml-Problem - changing filename-structure and inserting Reelname-Tag into header.
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: artyg on June 14, 2013, 02:50:54 PM
v1.1.6 is out - its the first attempt to fix the edl/xml-Problem - changing filename-structure and inserting Reelname-Tag into header.
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

regards chmee

Hm.... I try with edl, xml and aaf, but have same result... no changes. What names should have proxy files? In my case:
Proxy - M0000001.mov
CDNG - M0000001.000000.dng
Title: Re: The CinemaDNG Discussion
Post by: Jakobmen on June 14, 2013, 03:10:32 PM
same problem for me
only relinks first clip in the SEQ from Avid to resolve.



thanks
Title: Re: The CinemaDNG Discussion
Post by: tin2tin on June 14, 2013, 03:30:16 PM
not working with AAF
My guess is that it might be helpful if you added how you test it.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 14, 2013, 04:24:43 PM
please just link an edl or xml-example. right now i have no time to test myself -
but could it be, your workflow with proxies is not right? are you sure, you're doing it right?

why should resolve know, that m00001.mov picturesequence has to be replaced with m0001.[].dng?
is it described in the edl/xml?

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: artyg on June 14, 2013, 05:37:08 PM
please just link an edl or xml-example. right now i have no time to test myself -
but could it be, your workflow with proxies is not right? are you sure, you're doing it right?

why should resolve know, that m00001.mov picturesequence has to be replaced with m0001.[].dng?
is it described in the edl/xml?

regards chmee

EDL https://docs.google.com/file/d/0BxDyTWKlBKcWd1k5ekNCOWJtTzA/edit?usp=sharing
XML https://docs.google.com/file/d/0BxDyTWKlBKcWcmlRbjYtOTA3M0E/edit?usp=sharing

So I wanted to ask - how proxy should be named, that they are properly conform in Resolve? Because proxy is a single video file (.mov or mp4) and original is dng sequence... I understand that Resolve don't now about it, but how do this?
Title: Re: The CinemaDNG Discussion
Post by: keyth on June 14, 2013, 07:34:43 PM
Hi all,i have try last version of raw2cdng.1.1.6 and when import sequence in resolve i see hot pixel why? i have canon60d with tecnicolor cinestyle..in camera raw hot pixel not are present in davinci resolve yes...what is wrong? HERE A PIC


 ----> (https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi40.tinypic.com%2F2qisigz.jpg&hash=4c365dfcec09956ae4445db3ec5b16cb) HELP PLEASE,THANK U ALL!
Title: Re: The CinemaDNG Discussion
Post by: Jakobmen on June 14, 2013, 08:30:25 PM

why should resolve know, that m00001.mov picturesequence has to be replaced with m0001.[].dng?
is it described in the edl/xml?

regards chmee

yes edl/aaf/xml..

i will send my AAF file once i get home
Title: Re: The CinemaDNG Discussion
Post by: iunknown on June 14, 2013, 08:42:14 PM
That works Chmee!!  This is awesome!  Here is the workflow:

http://www.hurlbutvisuals.com/blog/2013/04/blackmagic-camera-post-workflow/
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 14, 2013, 08:56:49 PM
@jakobmen
i looked into the linked xml/edl's. i do not find any hints of a picturesequence. masterclip and proxyclip have the same name. So, sorry i play naive - in this problem-case i have to believe in your thoughts - likewise i try to minimise my work on that. :)

@iunknown
is this checkbutton the solution?
Quote
while re importing edl/xml deselect “Automatically import source clips into media pool.”

@keyth
its a bad idea to compare with adobe-products. they have just a little bit too much knowledge managing and fixing "problems". hotpixels are tagged and marked in CR2 and DNG - adobes RAW-Engine is able to detect hotpixels and masks them out. are these hotpixels static? i think so.. (but yes of course, it could be another problem - look into the cr2/dng/cdng with rawjuggler or dcraw)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: artyg on June 15, 2013, 05:56:25 AM
That works Chmee!!  This is awesome!  Here is the workflow:

http://www.hurlbutvisuals.com/blog/2013/04/blackmagic-camera-post-workflow/

HOW??? You work with RAW from Canon Mark? I try this, but at last step Resolve conform xml with wrong content. Cuts right. Please, tell, how?? Maybe some screenshots and xml? I'm trying to solve this problem for several days...
Title: Re: The CinemaDNG Discussion
Post by: zach915m on June 15, 2013, 08:59:21 PM
I also cannot seem to get the relinking to work.  Many hours trying different methods but no success.  I wonder if this is a case where something which worked in Final Cut does not work in AVID which I am using?    Confused!
Title: Re: The CinemaDNG Discussion
Post by: JulianH on June 16, 2013, 12:10:21 AM
I've got a bunch of DNG files I want to proces in Resolve, but I already deleted the .raw files... Any chance I can put those DNG's back into a .raw or convert them straight to cinema DNG? tips?
Title: Re: The CinemaDNG Discussion
Post by: iunknown on June 16, 2013, 01:16:27 AM
artyg/zach915m,
Where exactly does it get tripped up?

workflow:
import all the files into resolve
export proxies
create timeline in premier, edit and export xml
import xml into resolve
deselecting “Automatically import source clips into media pool.”

Make sure you save the original project in resolve so that all the original clips are in the media pool.
Title: Re: The CinemaDNG Discussion
Post by: zach915m on June 16, 2013, 05:58:55 AM
Here's the workflows I've tried both of them getting tripped up at the same point:

Both of these are CDNG to Resolve to AVID back to Resolve

1.  Use CDNG to make image sequences
2.  Import into resolve
3.  Export using either Avid AAF round trip or just using a regular export to DNX 36 8bit files.
4.  Edit in Avid
5.  Export AAF
6.  Open AAF in Resolve, un-clicking "auto import source clips" I have also tried clicking "ignore file extensions" and using EDL to go back to resolve and the same thing happens every time which is that it links to the first DNG in my "master" bin and uses that clip with all the cut points that are given.

The issue is not that it doesn't recognize the cut points, it's that it doesn't link back to the correct DNG files and nothing seems to remedy this including "reconform from bins."  I do hope this is my error and not an issue with the CDNG!  Sorry for making a ruckus!  It also seems that if others are having success with Premiere and FCP that it could be an avid issue.

-Zach
Title: Re: The CinemaDNG Discussion
Post by: Jakobmen on June 16, 2013, 06:54:51 AM
Here's the workflows I've tried both of them getting tripped up at the same point:

Both of these are CDNG to Resolve to AVID back to Resolve

1.  Use CDNG to make image sequences
2.  Import into resolve
3.  Export using either Avid AAF round trip or just using a regular export to DNX 36 8bit files.
4.  Edit in Avid
5.  Export AAF
6.  Open AAF in Resolve, un-clicking "auto import source clips" I have also tried clicking "ignore file extensions" and using EDL to go back to resolve and the same thing happens every time which is that it links to the first DNG in my "master" bin and uses that clip with all the cut points that are given.

The issue is not that it doesn't recognize the cut points, it's that it doesn't link back to the correct DNG files and nothing seems to remedy this including "reconform from bins."  I do hope this is my error and not an issue with the CDNG!  Sorry for making a ruckus!  It also seems that if others are having success with Premiere and FCP that it could be an avid issue.

-Zach

This is the same for me...
Title: Re: The CinemaDNG Discussion
Post by: marekk on June 16, 2013, 03:20:40 PM
DNG header need a badpixel_opcode field like files generated by raw2dng. ACR fixes bad/hot pixels automatically with badpixel_opcode. I don't know how it works in Davinci Resolve ..

https://bitbucket.org/hudson/magic-lantern/src/b93d6ae2b30dc6581efeb27d2e4571c6fd92ab6e/src/chdk-dng.c?at=unified

Hi all,i have try last version of raw2cdng.1.1.6 and when import sequence in resolve i see hot pixel why? i have canon60d with tecnicolor cinestyle..in camera raw hot pixel not are present in davinci resolve yes...what is wrong? HERE A PIC

Title: Re: The CinemaDNG Discussion
Post by: artyg on June 16, 2013, 05:48:14 PM
Here's the workflows I've tried both of them getting tripped up at the same point:

Both of these are CDNG to Resolve to AVID back to Resolve

1.  Use CDNG to make image sequences
2.  Import into resolve
3.  Export using either Avid AAF round trip or just using a regular export to DNX 36 8bit files.
4.  Edit in Avid
5.  Export AAF
6.  Open AAF in Resolve, un-clicking "auto import source clips" I have also tried clicking "ignore file extensions" and using EDL to go back to resolve and the same thing happens every time which is that it links to the first DNG in my "master" bin and uses that clip with all the cut points that are given.

The issue is not that it doesn't recognize the cut points, it's that it doesn't link back to the correct DNG files and nothing seems to remedy this including "reconform from bins."  I do hope this is my error and not an issue with the CDNG!  Sorry for making a ruckus!  It also seems that if others are having success with Premiere and FCP that it could be an avid issue.

-Zach

No, it's not avid problem - i work with Premiere Pro CS6 and export xml and have the same problem.

iunknown,
artyg/zach915m,
Where exactly does it get tripped up?

workflow:
import all the files into resolve
export proxies
create timeline in premier, edit and export xml
import xml into resolve
deselecting “Automatically import source clips into media pool.”

Make sure you save the original project in resolve so that all the original clips are in the media pool.

Can you show your screenshots of folders with original dng files and proxy files. And send your xml file? Becouse we do the same as you, but... strange things happen =(

Here my workflow screenshots

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.pixs.ru%2Fthumbs%2F5%2F6%2F7%2FSnimok1PNG_2014616_8223567.jpg&hash=f8166f489a94c63503754f79b6e5b28e) (http://pixs.ru/showimage/Snimok1PNG_2014616_8223567.png)  (https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi5.pixs.ru%2Fthumbs%2F5%2F7%2F6%2FSnimok2PNG_9422966_8223576.jpg&hash=39700962a45f1b2371d11405ea15adcf) (http://pixs.ru/showimage/Snimok2PNG_9422966_8223576.png)  (https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.pixs.ru%2Fthumbs%2F5%2F8%2F1%2FSnimok3PNG_7362325_8223581.jpg&hash=65e302f1069e2b482bc3013a5f41bce2) (http://pixs.ru/showimage/Snimok3PNG_7362325_8223581.png)  (https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi5.pixs.ru%2Fthumbs%2F5%2F8%2F5%2FSnimok4PNG_3585798_8223585.jpg&hash=e3441b35ad9fa0227ec75130912a98a8) (http://pixs.ru/showimage/Snimok4PNG_3585798_8223585.png)  (https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi5.pixs.ru%2Fthumbs%2F5%2F9%2F5%2FSnimok5PNG_9067860_8223595.jpg&hash=bd945d428076b2d0ada37964369ea82a) (http://pixs.ru/showimage/Snimok5PNG_9067860_8223595.png)  (https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi5.pixs.ru%2Fthumbs%2F5%2F9%2F8%2FSnimok6PNG_8969078_8223598.jpg&hash=7d302f006fe9663cbfdc8924a9b68a97) (http://pixs.ru/showimage/Snimok6PNG_8969078_8223598.png) 
Title: Re: The CinemaDNG Discussion
Post by: fatpig on June 17, 2013, 12:20:32 AM
Hello chmee.
Thank you for your great work!

I am developing BATCHelor and awaiting eagerly your implementation of commandline. :)
I would like to know if it is possible to not have to join .RAW and .R00 , .R01,... split files beforehand, as this is tedious.

Can you implement the extraction of CDNG from split files natively?  :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 17, 2013, 01:24:43 PM
@marekk  since when is the badpixel thing implemented? i have to look to newest raw-files.. :) thx.

@keyth maybe it helps if you use the inbuilt hotpixel-remapping/cleaning?!

@fatpig the command line is on the todo-list - please be patient. and yes, of course the split-files will be handled as one in my converter - so you have only to give the .raw-file and it will find the rest, if there..

Roundtrip Ideas - im not at home (for next 6 days) so i cant try that - are you able to rename the footage in resolve? as with the last version i tried to differentiate the name from the framecount with the dot - maybe it was wrong.

original - M3115.[00001-00250].dng (it looks like resolve thinks, "M3115." is the name, but we need "M3115")
proxy - M3115.mov

the next attempt will be of course creating timecode into the dng-header.


bytheway - do i see that right? raw2dng is now converting to 16bit? are you able to use these files in resolve? if yes, i would turn my application into a bitrate-converter - because all benefits (badpixels, verticalbanding) are built in..


v1.1.7 is downloadable here (http://212.227.97.228/DSLR_KLEINANZEIGEN/Download/raw2cdng.1.1.7.zip). its a testversion for figuring out about the resolve-roundtripping.
(*) deleted this dot in filename between name and framecount
(*) fixed filename to 8 chars, more will be truncated, less will be filled up with random chars.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: peoplemerge on June 17, 2013, 04:40:12 PM
No, it's not avid problem - i work with Premiere Pro CS6 and export xml and have the same problem.
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi5.pixs.ru%2Fthumbs%2F5%2F9%2F8%2FSnimok6PNG_8969078_8223598.jpg&hash=7d302f006fe9663cbfdc8924a9b68a97) (http://pixs.ru/showimage/Snimok6PNG_8969078_8223598.png)

Me too, I'm having the same issue with Premiere Pro CS6.  When importing the XML back into Resolve, it has the same source clip repeated.  I suspect a Resolve bug. Without being an expert, I looked at the Premiere XML output and it looked fine. 

What I don't get about the process is how Resolve knows how to link the proxies back to the original source media.  Could it be stored in it's project somehow?
Title: Re: The CinemaDNG Discussion
Post by: iunknown on June 17, 2013, 07:23:50 PM
peoplemerge, artyg,
Yeah, your right, its not relinking the files correctly when importing the xml back into resolve.

Chmee,
Just tried it with version 1.1.7, and same thing.  I have no idea how resolve knows to point back to the original raw files with the black magic footage (when selecting "Automatically import source clips into media pool.” )

Here is the naming for BMCC footage raw folders:
Greg Fiske_1_2013-02-15_1422_C0000
and the default name resolve gives to proxies:
V1-0001_Greg Fiske_1_2013-02-15_1422_C0000_1241904

cinemadng raw folder:
CDNG16_M091158U
and default name resolve gives to proxies:
V1-0002_M091158U000000


Chmee,
What is the variable "_1241904" added to the end of the proxie names on the bmcc files?  It is the one thing that is different with the two xml files.  The cinemadng comes up as zero, but the cinemadng xml has this:

<frame>1241904</frame>


Here are the xml files:

bmcc xml
https://docs.google.com/file/d/0B-cS4Mdd09ShYkd0YXg5c2dTT0k/edit?usp=sharing

cinemadng xml
https://docs.google.com/file/d/0B-cS4Mdd09ShR3JTTzVJMFNvVXc/edit?usp=sharing
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 17, 2013, 08:02:14 PM
<string>14:22:26:00</string>
<frame>1241904</frame>

seems to be sort of timestamp - with frames as smallest dividor.

1241904 / 24 / 60 /60 = 14 ..... [hour]

modulo (1241904-1209600) = 32304
32304 /24/60 = 22.... [minute]

modulo (32304-31680) = 624
624/24 = 26 [second]

So, it seems, i have to set the timecodes and fps-tag. i hope, this is the final and functional key.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: JulianH on June 18, 2013, 12:01:31 AM
I'm getting weird results in Resolve when loading CinemaDNG's converted with raw2cdng.1.1.6. I'm shooting with the Canon 50D.
If I open the frames in ACR, they look fine. In Resolve there is a faint blue/orange coloured line inside of the frame.

See screenshot here: http://www.magiclantern.fm/forum/index.php?topic=5586.msg52177#msg52177

Could this be caused by raw2cdng? I don't think so since the DNG's look fine in ACR. Could it be a 50D problem? Anyone seen this and knows a fix for Resolve?
Title: Re: The CinemaDNG Discussion
Post by: fatpig on June 18, 2013, 11:16:48 AM
maybe you can make blacklevel editable somehow, because some of my 5DII Videos, which should have 1024 - have a wrong blacklevel of 2037 set inside the .RAW and look really bad.
I would like to set it to 1024 again.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 18, 2013, 07:25:31 PM
gave you an answer in another thread - im not sure if an override-values field is a good idea
..finally it is (as bonus), but by now the important thing is the roundtrip-functionality with other editors.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: peoplemerge on June 19, 2013, 01:54:05 AM
+1 on the roundtrip!  I have a fun story to tell.  Go chmee!
Title: Re: The CinemaDNG Discussion
Post by: noisyboy on June 19, 2013, 02:31:57 AM
Dudes - what raw space should we be working in inside of Resolve with ML RAW? As in what RAW settings are best to use within Resolve? Is it best to decode with CinemaDNG default or with Clip and Rec.709 (or is does this default to the same anyway?).

Is it best to Decode with "Clip" and About to have a play! Sooooooo Excited! (again - can't thank you enough for this!)
Title: Re: The CinemaDNG Discussion
Post by: nightvision04 on June 19, 2013, 02:46:48 AM
Dudes - what raw space should we be working in inside of Resolve with ML RAW? As in what RAW settings are best to use within Resolve? Is it best to decode with CinemaDNG default or with Clip and Rec.709 (or is does this default to the same anyway?).

Is it best to Decode with "Clip" and About to have a play! Sooooooo Excited! (again - can't thank you enough for this!)

I find it's best to decode with Clip and film log. Then you can interpret things on the fly with an awesome LUT like one of these:
http://juanmelara.com.au/print-film-emulation-luts-for-download/
Title: Re: The CinemaDNG Discussion
Post by: peoplemerge on June 19, 2013, 03:01:50 AM
nightvision04 is absolutely right.  Set it to "Clip" "As Shot" "BMD Film".  I'm grading an hour of 5D2 raw footage without LUTs learning resolve lite as I go and it looks very filmic.   rec. 709 will show pink fringing and even more moire.  The BMD film setting is a vast improvement.
Title: Re: The CinemaDNG Discussion
Post by: noisyboy on June 19, 2013, 03:40:02 AM
The BMD film setting is a vast improvement.

Thanks dudes!  8)
Title: Re: The CinemaDNG Discussion
Post by: nightvision04 on June 19, 2013, 10:18:19 AM
Check out those LUTS! Big films are known to use Print LUTs to emulate film colors, and Juan is a best kept secret on the internet for high quality LUTS. They are free, and he has a great tutorial to help you install them.
Title: Re: The CinemaDNG Discussion
Post by: marekk on June 19, 2013, 10:46:32 AM
@marekk  since when is the badpixel thing implemented? i have to look to newest raw-files.. :) thx.

It's not in raw file. There is a special badpixel_opcode in DNG tags. Look at chdk-dng.c file. With this tag dead/hot pixels can be removed automatically for example by Adobe Camera Raw.
Title: Re: The CinemaDNG Discussion
Post by: artyg on June 19, 2013, 12:21:09 PM
nightvision04 is absolutely right.  Set it to "Clip" "As Shot" "BMD Film".  I'm grading an hour of 5D2 raw footage without LUTs learning resolve lite as I go and it looks very filmic.   rec. 709 will show pink fringing and even more moire.  The BMD film setting is a vast improvement.

Where this settings must be set?
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 19, 2013, 09:00:27 PM
@marekk it seems its just a flag for allowing repairing "bad" pixels - described with a number 0-4 - hmm. ok. will put it on toDo-list. but we have to check, if resolve/speedgrade is able to repair badpixels - so inbody masking is imho the better solution)

here is a version 1.1.8 with reelname, timecode and framerate in header - maybe it helps. the framerate is static on 24fps - because we're in testing mode - if it works i will change to variable framerates and timecode (if needed, starttime of filegenerate-datetime)

Download v1.1.8 here (http://dslr-kleinanzeigen.de/Download/raw2cdng.1.1.8.zip)

regards chmee

ToDo-List 20.06.2013
(*) Roundtrip must work
(*) IliasG's idea of 10bit log (state 90%)
(*) strippedFiles
(*) Repair-Flag für badpixels? (simpler workaround: demasking in body)
(*) commandline

Title: Re: The CinemaDNG Discussion
Post by: peoplemerge on June 19, 2013, 10:51:16 PM
Where this settings must be set?

Color pane, bottom left panel there's 4 icons, a circle, bar chart, 3 circles in a triangle, and a movie cam.  The last one, the movie cam, is for "Camera Raw".  below that is Master Settings pane.  Set the Decode Using to "Clip" etc as discussed above
Title: Re: The CinemaDNG Discussion
Post by: tin2tin on June 19, 2013, 11:09:44 PM
@chmee Great to see you're still going strong! I haven't got resolve on this computer, so I can't test atm. Let's hope the mediainfo guys will add info subraction of cdng files.
Title: Re: The CinemaDNG Discussion
Post by: mediali on June 20, 2013, 12:14:00 AM
@marekk it seems its just a flag for allowing repairing "bad" pixels - described with a number 0-4 - hmm. ok. will put it on toDo-list. but we have to check, if resolve/speedgrade is able to repair badpixels - so inbody masking is imho the better solution)

here is a version 1.1.8 with reelname, timecode and framerate in header - maybe it helps. the framerate is static on 24fps - because we're in testing mode - if it works i will change to variable framerates and timecode (if needed, starttime of filegenerate-datetime)

Download v1.1.8 here (http://dslr-kleinanzeigen.de/Download/raw2cdng.1.1.8.zip)

regards chmee

ToDo-List 20.06.2013
(*) Roundtrip must work
(*) IliasG's idea of 10bit log (state 90%)
(*) strippedFiles
(*) Repair-Flag für badpixels? (simpler workaround: demasking in body)
(*) commandline
still not work  about timecode  problem

here is the xml file please have a look

http://www.istorm2010.com/temp/A-Resolve-out.xml (http://www.istorm2010.com/temp/A-Resolve-out.xml)

http://www.istorm2010.com/temp/B-premiere out(Importing A).xml (http://www.istorm2010.com/temp/B-premiere out(Importing A).xml)

http://www.istorm2010.com/temp/C-Resolve out again (impotrting B).xml (http://www.istorm2010.com/temp/C-Resolve out again (impotrting B).xml)
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.istorm2010.com%2Ftemp%2FC-Resolve+out+again.jpg&hash=7c5a94cdae539037ca53c6ecce2e6c2f)
Title: Re: The CinemaDNG Discussion
Post by: iunknown on June 20, 2013, 01:03:30 AM
Yeah, tested, same issue, duplicate clips on the xml import into resolve.  Thanks for the effort chmee.
Title: Re: The CinemaDNG Discussion
Post by: nowhereman86 on June 20, 2013, 03:10:32 AM
I was wondering if anyone has had luck roundtriping from Resolve with the "AAF Roundtrip" function and using AVID to edit. I used RAWMagic to decode the RAW to CDNG, then followed the steps in this video: (
This worked perfectly for me! I edited a one minute short with multiple clips and it went back and forth between the programs without a hitch. Anyone else had luck trying this method?

Title: Re: The CinemaDNG Discussion
Post by: zach915m on June 20, 2013, 04:25:55 AM
I'm on a PC so I can't test RawMagic.  But I did use the AAF roundtrip to test this and it didnt work...=(
Title: Re: The CinemaDNG Discussion
Post by: marekk on June 20, 2013, 12:28:21 PM
OpcodeLists are allowed in CinemaDNG file format so maybe Davinci Resolve support this feature..

@marekk it seems its just a flag for allowing repairing "bad" pixels - described with a number 0-4 - hmm. ok. will put it on toDo-list. but we have to check, if resolve/speedgrade is able to repair badpixels - so inbody masking is imho the better solution)

Title: Re: The CinemaDNG Discussion
Post by: chmee on June 20, 2013, 12:37:46 PM
"maybe" is the right description. As of version 1.1.0.0 of the CDNG-specification opcodes are optional by decoders, not mandatory. i put it on toDo-list, but there are some fixes before that..

by now i'm out for the next three/four days - because i cant solve the roundtrip-problem here on my netbook - when i'm back home i will analyse differences between (functional) bmcc-roundtrips and our stuff.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: peoplemerge on June 20, 2013, 05:11:12 PM
@nowhereman86 this is a substantially different workflow from the final cut XML.  Interesting!  My resolve with fcp dailies crashes trying to export AAF with some of my existing projects but I'll try with smaller ones and the Avid MXF dailies.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 20, 2013, 09:16:54 PM
right before implementing the 10Bit-log-Curve i'm in doubt, if some bodies have a blacklevel lower than 2017..?!

Someone any info for me?
Title: Re: The CinemaDNG Discussion
Post by: a1ex on June 20, 2013, 09:37:47 PM
Yes, 1024, 1500...

It's computed on the fly; I have no idea what the absolute range is.

FYI: http://lclevy.free.fr/cr2/#wb
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 20, 2013, 10:41:01 PM
so it seems i have to compute the rec709-log on the fly also. ok. thanks.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: peoplemerge on June 20, 2013, 11:50:34 PM
@nowhereman86 Same problem with importing/exporting Premiere Pro, which doesn't work with MXF files.  Just noticed Adobe has updates for all products, Premiere CC, which does support DNxHD MXF files!  However with both versions, the relinking still fails using both aaf and xml exports.  You might not notice if you only have 1-2 clips.  Can you confirm this works in Avid on a project with 7-10 clips?  Are you able to export xml from avid, and if so does the workflow still work?

Title: Re: The CinemaDNG Discussion
Post by: IliasG on June 21, 2013, 11:02:01 AM
right before implementing the 10Bit-log-Curve i'm in doubt, if some bodies have a blacklevel lower than 2017..?!

Someone any info for me?

Yes, models older than  7D (5DII, 50D, 40D) have BL at around 1024. This in photo-raws, video-raws can be a bit different.

In any case for a correct calculation we need the full range limits

Except BL which is computed in camera by ML and accessible in metadata, there is the White Level which is set at a predefined value depending on model and based on photo-raw WL as used in CR2 to DNG conversions. This is not accurate enough. For example metadata for 5DIII say WL=15000 while histograms show 16382 ..

I think that samples for every model with burned areas are necessary to declare the WL based on real video-raw data ..   

Yes, 1024, 1500...

It's computed on the fly; I have no idea what the absolute range is.

FYI: http://lclevy.free.fr/cr2/#wb

I didn't knew a case of BL at 1500 !!.

Alex, as I saw on your 5DIII black frames the BL calculated by ML is truncated to the next lower integer so for values 2047.8-2047.9 it becomes 2047. Not a serious error for 14bit DNG, but after truncating to 12bit (as raw2cdng does for the moment) it can be a bit harmful ..



Title: Re: The CinemaDNG Discussion
Post by: IliasG on June 21, 2013, 11:23:16 AM
so it seems i have to compute the rec709-log on the fly also. ok. thanks.

regards chmee

Or use predefined "caseA", caseB" ..

A small detail which might help if searching for "perfection" ..
While I started with BL-32, I used BL-31 to keep the BL=2048 at the center when mapping to 10bit (2047, 2048, 2049 all mapped to 10) so as to have the smallest error. With radically different range (WL-BL) this could change.   
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 21, 2013, 11:26:46 AM
@IliasG as the log-curve goes to full 14bit (16384) and the highlevel end is quite flat, i dont think a whitelevel-difference f.e. 15.000 or 13.500 is any significant problem. now thinking about two log-lists (one with ~2017 start and another one with ~1010) or really computing on the fly..

(like the caseA/caseB-thing much more :) )

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: marekk on June 21, 2013, 01:52:52 PM
I don't know how to test it now because Davinci Resolve doesn't open 14bit DNGs.. So I need a 12 or 16bit DNG with hot pixels and badpixel_opcode..

"maybe" is the right description. As of version 1.1.0.0 of the CDNG-specification opcodes are optional by decoders, not mandatory. i put it on toDo-list, but there are some fixes before that..

Title: Re: The CinemaDNG Discussion
Post by: revast on June 21, 2013, 05:20:09 PM
hello, have tried your program in wine, under 1.5.31 with mono, it starts up perfectly, but the drag&drop seems to be disfuct. Really looking forward to the commandline version then.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 22, 2013, 09:57:29 AM
@revast minimum requirement is .net4 compatibility.
http://appdb.winehq.org/objectManager.php?sClass=version&iId=17886

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: nowhereman86 on June 22, 2013, 03:04:30 PM
@peoplemere Yeah, this workflow seemed to move along just fine with 10 clips in a thing I put together. I didn't use XMl though I used exclusively the AAF Roundtrip function and followed the steps in this video. I think the order in which you execute these steps is very critical since. If you have Avid and resolve, try it out and see if it works okay.

Title: Re: The CinemaDNG Discussion
Post by: zach915m on June 23, 2013, 02:13:09 AM
Yeah that's the video I've been using too but the roundtrip doesn't work for me... :'(
Title: Re: The CinemaDNG Discussion
Post by: fatpig on June 23, 2013, 11:30:36 AM
are you aware that the progress bars lock after processing about 1/4th of the frames?
it still processes all of them, but the GUI greys out until its done if I click inside.
Title: Re: The CinemaDNG Discussion
Post by: JulianH on June 23, 2013, 07:01:14 PM
are you aware that the progress bars lock after processing about 1/4th of the frames?
it still processes all of them, but the GUI greys out until its done if I click inside.
Same here (windows 7).
Title: Re: The CinemaDNG Discussion
Post by: peoplemerge on June 24, 2013, 05:56:44 AM
OK I figured out the round trip for CinemaDNG raw, Premiere CC, Da Vinci Resolve Light!  The trick is to get the reel names consistent, I'm using the source clip path, by jumping through some hoops to rename the directories.  Details here:
Title: Re: The CinemaDNG Discussion
Post by: Andy600 on June 24, 2013, 07:45:31 AM
OK I figured out the round trip for CinemaDNG raw, Premiere CC, Da Vinci Resolve Light!  The trick is to get the real names consistent, I'm using the source clip path, by jumping through some hoops to rename the directories.  Details here:

Excellent! Very useful tutorial. Thanks :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 24, 2013, 10:12:05 AM
@peoplemerge great. nice finding. will try that. YOURE THE STAR OF THE DAY! workin! the important thing is the creation of the ReelNo in Resolve.

@fatpig thx. seems, i have to refresh the gui from time to time.
Title: Re: The CinemaDNG Discussion
Post by: iunknown on June 24, 2013, 06:28:11 PM

Great find peoplemerge.  Thanks so much for all the hard work you guys!

For everyone else, discription of the problem (reelno variable missing) at timestamp 7:11.
Title: Re: The CinemaDNG Discussion
Post by: Jakobmen on June 24, 2013, 11:24:32 PM
OK I figured out the round trip for CinemaDNG raw, Premiere CC, Da Vinci Resolve Light!  The trick is to get the reel names consistent, I'm using the source clip path, by jumping through some hoops to rename the directories.  Details here:

anyone try this on windows yet ?

trying to get home to try it lol  ;D
Title: Re: The CinemaDNG Discussion
Post by: swaneon on June 25, 2013, 12:59:56 AM
I'm on windows 8 and I can't use drag/drop at all it seems like
does anyone have a solution?
Title: Re: The CinemaDNG Discussion
Post by: iunknown on June 25, 2013, 01:20:47 AM
Works on windows 7.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 25, 2013, 07:47:45 AM
@swaneon is drag and drop a general problem on your computer/win8? maybe thats a solution?
http://www.c-sharpcorner.com/UploadFile/nipuntomar/drag-drop-not-working-in-windows-8/
Title: Re: The CinemaDNG Discussion
Post by: fatpig on June 25, 2013, 06:14:47 PM
@chmee:
is the roundtrip something that can be solved in future versions of raw2cdng or do we have to follow these steps every time from now on?
(at work, cant watch right now)
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 25, 2013, 06:26:43 PM
hmm. i wrote in another thread, i tried a simple xml-roundtrip with blackmagic-CC-Files - it failed the same way as with canon-cdng's. the bmcc-files also had no ReelNo - so, my assumption is, if filename/path/reelName isn't 100% in right order (for resolve) it fails.. i built it in reelname, framerate and timecode and it didnt helped. by now the oneclick in the settings for the reelNo-generation is a no-brainer. for myself i will leave this roundtrip-salvation and make it a sidejob, because its too easy to solve..

todolist 25.06.2013
(*) 10bit log (nearly ready, maybe tonight for first testing)
(*) splitted files support (try to solve it today as well)
(*) vertical banding
(*) compression
(*) command line
(*) threading for multi-cpu-support

(-) dragndrop problems in win8 (or leaving till command line is working)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: peoplemerge on June 25, 2013, 07:00:15 PM
the bmcc-files also had no ReelNo - so, my assumption is, if filename/path/reelName isn't 100% in right order (for resolve) it fails.. i built it in reelname, framerate and timecode and it didnt helped. by now the oneclick in the settings for the reelNo-generation is a no-brainer. for myself i will leave this roundtrip-salvation and make it a sidejob, because its too easy to solve..

Is it true that you can add the reel number into cdng metadata?  Otherwise creating distinct folders would take out half of the pain of this process.

Just throwing out ideas, but you might leave it up to the user to multi select [and append the name]:
[ ] use the raw file sequence (eg M0000000) - though this is not sufficient since we can expect the user to want to use clips from multiple cards and they reset to M0000000 when you power down ML.
[ ] use some string
[ ] use the file creation time of the rawfile
[ ] use the current time

(*) vertical banding
(*) compression
(*) command line
(*) threading for multi-cpu-support

Vertical banding ++ !!!
What do you have in mind for compression?
Title: Re: The CinemaDNG Discussion
Post by: sarotaz on June 25, 2013, 07:11:13 PM
On 60D still not working 100%  :( I hope you can include bad pixel opticode...in my case cdng files after conversion have black and hot pixel  :(
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 25, 2013, 08:02:33 PM
@peoplemerge since v1.1.8 there is reelname, framerate and timecode written in the cdng-header - but reelname is not shown in resolve, like the bmcc-files as well. In consistency with edl CMX3600 the reelname is only 8 uppercase-chars or numbers long - with v1.2.0 the path will be named with a timedate (dont know if timestamp or humanreadable ymdhm) before the reelname. the file will be reelname.sequencenumber.dng - simply. i decided this way, because the roundtrip-solution is so simple, i care more of readable file-structure than of (possible) reelno-understanding by resolve - as i said the bmcc werent recognised as well (link to the thread (http://www.magiclantern.fm/forum/index.php?topic=6271.msg53796#msg53796))

btw - m00000.raw is no filenaming-convention anymore - with the newer versions of rec_raw its written
mXX-YYYY.raw - f.e. M35-6148.raw

@sarotaz there is no guarantee, its working, but i will give it a try. badpixel-operations and recognition are no must of cdng-decoders.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: swaneon on June 26, 2013, 12:21:23 AM
@swaneon is drag and drop a general problem on your computer/win8? maybe thats a solution?
http://www.c-sharpcorner.com/UploadFile/nipuntomar/drag-drop-not-working-in-windows-8/
I tried that and the interesting thing is that every other program is fine with Drag + Drop—only raw2cdng does not work
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 26, 2013, 12:36:36 AM
@swaneon thx for info - will figure that out - but have no win8 here ;)
tried to run the app as administrator? some coder write, its about UAC..

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: fatpig on June 26, 2013, 06:49:45 PM
great todo-list! Exactly what I hoped for. :)

As for the drag n drop,
when commandline is implemented, people can use different GUIs like RAWanizer and BATCHelor.

I am looking forward to input and output path parameters and compression.
I can imagine that we would use our CF card as input on set and just save the (compressed?) DNGs to the laptops HDD as output, saving time and space. :)
Title: Re: The CinemaDNG Discussion
Post by: swaneon on June 27, 2013, 01:28:33 AM
got drag and drop to work

downloaded and tried 1.1.2 and it worked
then tried 1.1.6 and that worked
then retried 1.1.8 and it works!

Title: Re: The CinemaDNG Discussion
Post by: GregoryOfManhattan on June 27, 2013, 06:33:24 AM

btw - m00000.raw is no filenaming-convention anymore - with the newer versions of rec_raw its written
mXX-YYYY.raw - f.e. M35-6148.raw


the naming convention for raw files currently is:
M for magic lantern
dd for 2 digit day of the month
hhmm for 2 digit hour and 2 digit minute when recording began.
(the last two digits get incremented if you've already recorded videos in the same minute up to 99 (if anyone could click that fast))
so every file name is unique.

so your example would never be generated by magic lantern.
M01-M31 first part range
0000 - 2399 second part range (though seldom will there be values greater than 2359)

it may be helpful to preserve these dates and times in yours and other post-processing tools.
(i was looking at this in case there was a need to insert a timecode into the cinemaDNG exif data)
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 27, 2013, 08:28:19 AM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

raw2cdng v1.1.9 is online
- 10bit log is in (but experimental) - no linearizationtable (thx IliasG)
- naming conventions slightly changed
- fixed freezing gui

(!) 10bit log only works for bodies with blacklevel higher 2000 (!) next patch will have a lut for blacklevels around 1024 (!)

@IliasG
is your excel-calc that stable to calculate this lut by changing start values?

(lol. short test in resolve - 10bit/log is not looking that bad :) and hotpixels are gone in this one sample i have :D - if the red tint is too much, lower red gain to ~0.86 - but now its time to test the ability to grade and work with them - we squeezed all information into the 10bit - as a side advantage/drawback theres nearly no more data in blownout or dark parts of the picture - i dont really know if 10bit/log is a good grading base - possibly good for directly working in premiere/fcp/avid, after transcoding into a useable codec)

@gregoryOfManhattan
ah ok. so, my example was wrong - basically it was just as a information about the changed naming of raw-files. finally i leave this numbers in the [reelname] - additionally i put timedate of creation into the pathname.

@swaneon
good news :)

urgent on todoList for next days
- support of splitted files
- 10bit|log for blacklevel ~1024-bodies.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: iaremrsir on June 28, 2013, 12:36:41 AM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

raw2cdng v1.1.9 is online
- 10bit log is in (but experimental) - no linearizationtable (thx IliasG)
- naming conventions slightly changed
- fixed freezing gui

(!) 10bit log only works for bodies with blacklevel higher 2000 (!) next patch will have a lut for blacklevels around 1024 (!)

@IliasG
is your excel-calc that stable to calculate this lut by changing start values?

(lol. short test in resolve - 10bit/log is not looking that bad :) and hotpixels are gone in this one sample i have :D - if the red tint is too much, lower red gain to ~0.86 - but now its time to test the ability to grade and work with them - we squeezed all information into the 10bit - as a side advantage/drawback theres nearly no more data in blownout or dark parts of the picture - i dont really know if 10bit/log is a good grading base - possibly good for directly working in premiere/fcp/avid, after transcoding into a useable codec)

@gregoryOfManhattan
ah ok. so, my example was wrong - basically it was just as a information about the changed naming of raw-files. finally i leave this numbers in the [reelname] - additionally i put timedate of creation into the pathname.

@swaneon
good news :)

urgent on todoList for next days
- support of splitted files
- 10bit|log for blacklevel ~1024-bodies.

regards chmee

This version killed drag and drop for me. It still works with previous versions though. It's only this program.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 28, 2013, 08:27:42 AM
@iaremrsir
under what OS? are some other users with the same problem?
did you rename your files? is it showing the dragndrop-icon (arrow with plus) on drag-action?

regards chmee 
Title: Re: The CinemaDNG Discussion
Post by: tin2tin on June 28, 2013, 10:46:15 AM
First time I ran the 1.1.9 version on Win 8 d'n d didn't work. Second time it did work.
Title: Re: The CinemaDNG Discussion
Post by: araucaria on June 28, 2013, 10:23:07 PM
I just tried your converter (1.1.9) for the first time, and it's working very slow for me (like one frame every two or more seconds) but I have 0% CPU usage, am I doing something wrong? Maybe I have an old verstion of raw2dng, I don't know.

Ok I was doing it wrong, just drag and drop the RAW file, not the dng sequence, thanks for the converter!
Title: Re: The CinemaDNG Discussion
Post by: iaremrsir on June 29, 2013, 11:03:40 PM
@iaremrsir
under what OS? are some other users with the same problem?
did you rename your files? is it showing the dragndrop-icon (arrow with plus) on drag-action?

regards chmee

Windows 7 x64. I dragged the files straight from the card without altering them. It shows the + sign cursor, but after dropping nothing happens. The clips don't show in the queue.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 30, 2013, 07:00:24 AM
@iaremrsir the decoded dngs will be saved on the location, where the raw-files are - so its no good idea to drag them straight from the card into the app. did you tested with a copy from a hdd?

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: iaremrsir on June 30, 2013, 07:22:37 AM
@iaremrsir the decoded dngs will be saved on the location, where the raw-files are - so its no good idea to drag them straight from the card into the app. did you tested with a copy from a hdd?

regards chmee

I tried both ways, neither worked. I also tried turning off my security software to see if it might have been blocking functionality of the program because its happened before. But even then it still didn't work.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 30, 2013, 07:29:24 AM
what camera, what version (date) of your raw-rec-modul? could you upload a short (and small) example file (~100mb), so that i can try it on my computer? it seems, something's different in your files..

!FOUND! something with the filenames/analyzing is wrong - if you rename them, they will be imported, but they dont work - bug, solved in the coming version.

v1.2.0 is out - splitted Files support
(*) you just have to drag the first (.RAW) file - app searches for split-files..
(*) i had to rewrite the bytestream-reader - so please tell, if there are bugs in "normal" non-splitted dng-rewrite-mode..
(*) filename bug fixed

http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
(please test and give me some info if workin & bugs)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: Yaros525 on June 30, 2013, 06:43:11 PM
Windows 7 Pro x64 / NET 4 standalone directly from Microsoft installed.
Drag & Drop doesn't work for me. It just refuses to eat  my 5D2 raw. Is there other way to supply the files? Simple [Open] maybe?

Thnx for your work.

P.S. WinXP SP3 works just fine.
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 30, 2013, 06:58:04 PM
you've got 1.2.0?
is the drag-icon showing up? (throwing into listview!)
starting as administrator?

thread about dragndrop-problems in win7 -> http://answers.microsoft.com/en-us/windows/forum/windows_7-performance/drag-and-drop-not-working-in-windows-7/f6dcfa11-8c51-e011-8dfc-68b599b31bf5
Title: Re: The CinemaDNG Discussion
Post by: Yaros525 on June 30, 2013, 07:56:58 PM
Sure, 1.2.0
Got it. its lovely windows UAC. Once disabled, Drag & Drop works. But  ... I dont want it to be off permanently )
Title: Re: The CinemaDNG Discussion
Post by: blackroom on June 30, 2013, 08:04:52 PM
thanks for this great utility!
sorry i'm lost with this: how do i get the 5D2 specific raw2dng (5D2_WIN_raw2dng) info to work with raw2cdng?
raw2cdng is showing 5D2 in the batchlist but the DNG reads as Canon EOS PHR in ACR with embedded profile.
is only meant to work with Resolve?
thanks again!
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 30, 2013, 08:07:25 PM
@Yaros is it working, if you start it as administrator (without disabling UAC)? - you can change your rights on this application as its starting every time with admin-rights - rightclick -> properties -> compatibility - check "run this program as an administrator"

@blackroom i read it the first time, there is a specific 5d2-version of raw2dng? if i know, what changes there are, i can give you an answer ;) but finally, if you're generally working with adobe-workflow (acr), there's no real benefit using raw2cdng. (ok, 12bit and 10bit|log :D but thats no benefit, if youre saving in 16bit tif..)

:)
Quote
It adds Camera Model: Canon EOS 5D Mark II to the DNG
Is that all? Is there any benefit in ACR, if the Camera Model is set accordingly? "Canon EOS PHR" is just a footprint of me - PHR is short for PHReekz.de, my website :)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: chmee on June 30, 2013, 10:17:45 PM
Please someone upload somewhere a short and small RAW-File with hot/deadpixels and link it here. I just 've got an idea :)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: blackroom on July 01, 2013, 01:48:08 AM
aha cool, its your name! thanks for the reply! well one thing the camera profile can be loaded in ACR with the proper exif to set how it was shot , Neutral or Standard etc. so quicker to get to a place for FilmConvert profiles in AE, for instance.
i'm kinda between ACR / AE and resolve, just getting into da Davinci:)
Title: Re: The CinemaDNG Discussion
Post by: iaremrsir on July 01, 2013, 06:47:25 AM
1.2.0 fixed drag and drop for me
Title: Re: The CinemaDNG Discussion
Post by: DTSET123 on July 01, 2013, 09:08:46 AM
Is there a fix for Speedgrade?
Title: Re: The CinemaDNG Discussion
Post by: marekk on July 01, 2013, 10:51:54 AM
I will upload sample RAW file with hot pixels from 60D in the evening.

Please someone upload somewhere a short and small RAW-File with hot/deadpixels and link it here. I just 've got an idea :)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: chmee on July 01, 2013, 11:09:16 AM
@DTSET123 You mean the wrong whitebalance? so, first, try to set it manually. and then :) we're discussing saving whitebalance-point to the raws. http://www.magiclantern.fm/forum/index.php?topic=6779.0

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: fatpig on July 01, 2013, 11:51:10 AM
hey chmee, how about making an output folder option until commandline is finished?

I would love to just set my CF card as source and output to my project folder.
regards
Title: Re: The CinemaDNG Discussion
Post by: chmee on July 01, 2013, 11:52:22 AM
@fatpig :) just started yesterday to implement this.. on the way.
Title: Re: The CinemaDNG Discussion
Post by: IliasG on July 01, 2013, 05:38:05 PM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

raw2cdng v1.1.9 is online
- 10bit log is in (but experimental) - no linearizationtable (thx IliasG)
- naming conventions slightly changed
- fixed freezing gui

(!) 10bit log only works for bodies with blacklevel higher 2000 (!) next patch will have a lut for blacklevels around 1024 (!)

@IliasG
is your excel-calc that stable to calculate this lut by changing start values?

(lol. short test in resolve - 10bit/log is not looking that bad :) and hotpixels are gone in this one sample i have :D - if the red tint is too much, lower red gain to ~0.86 - but now its time to test the ability to grade and work with them - we squeezed all information into the 10bit - as a side advantage/drawback theres nearly no more data in blownout or dark parts of the picture - i dont really know if 10bit/log is a good grading base - possibly good for directly working in premiere/fcp/avid, after transcoding into a useable codec)

urgent on todoList for next days
- support of splitted files
- 10bit|log for blacklevel ~1024-bodies.

regards chmee

chmee, thanks for your work !!.

Sorry for the late responce, I was offline for a week.

Excel-calc should be stable (not 100% sure about this) for calculating on different black/white levels, only needs to pay attention on the details to have the smallest possible error at Black Level. It's easy (because the rec.709 gamma curve is linear there) by just increasing/decreasing the clipping point by 1-2 steps. I can give a hand on this as I have plenty of free time these days ..

About wrong colors .. don't bother with dirty corrections. Both WB and color matrices need to be applied on linear data or else we take strong color shifts. So we have to wait for the linearization table to be injected in exif data before we can make valuable tests. If this "injection" is a hard work there is a possibility we can surpass this need by a dedicated icc profile and use a converter which can make use of icc profiles (RawTherapee).

Talking about tests .. can you upload some RAW files which include dark and blown areas, made with the latest ML code ?. 

Title: Re: The CinemaDNG Discussion
Post by: 1% on July 01, 2013, 05:46:02 PM
The 6D white level shows at 15K... it should be 13K. Maybe there is a problem in raw.c code and the undef didn't work.

The 12 bit has been nice... lop off the least bits and it saves some space. Didn't see much impact in quality.. now if it zip compressed the dngs it would be all winner.
Title: Re: The CinemaDNG Discussion
Post by: chmee on July 01, 2013, 06:58:18 PM
@iliasG i'd be happy if you would do the calculation for the 1024 blacklevel - the linearizationtable isnt a problem, it's prepared :)

@1% thx for the tests..

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: chmee on July 01, 2013, 10:13:44 PM
v1.2.1 is out - selectable Folder - setting will be stored, so it stays in app, until you abort while selecting folder. check-box for short disabling static folder and it will save the dng-folder as previous, in the same folder as the raw file is..
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: fatpig on July 02, 2013, 11:02:47 AM
thanks for implementing this so fast! :) great! :)
Title: Re: The CinemaDNG Discussion
Post by: fatpig on July 02, 2013, 11:11:11 PM
1% mentioned zip compression. Is there a possibility for lossless compression like in Adobes DNG Converter? Remains directly editable.
Title: Re: The CinemaDNG Discussion
Post by: chmee on July 03, 2013, 12:42:39 AM
if i finish 10bit|log, compression will be next. i have to read the dng-papers to look, whats possible.

( im a bit longer AFK, holiday-roundtrip to spain for 14 days)
Do your work, test, check and make some good videos :)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: hjfilmspeed on July 04, 2013, 07:22:32 PM
1.2.1 and 1.2.0 are crashing on me. i converted a clip to test then i deleted the CinemaDNG the converted the same clip again and it  started crashing. Seems to be running slower to now. im not sure what happened. 1.1.9 still works for me. 1.2.1 and 1.2.0 crash after one clip finishes

I love your software though its amazing how you are able to this thank you!!!
Title: Re: The CinemaDNG Discussion
Post by: Joachim Buambeki on July 04, 2013, 10:15:57 PM
Latest version is crashing on my Win8 64bit machine because of insufficient disk space, I haven't checked older versions but I suspect them to behave in the same way.
Maybe you could add an error message and/or a check if the needed space is available on the volume?

Keep up the good work chmee!
Title: Re: The CinemaDNG Discussion
Post by: vikado on July 09, 2013, 11:09:07 PM
my workflow is as follow.
RAW>raw2cdng>CDNG sequence>Resolve>premiere>Resolve for final render


anyone know a way to properly white balance in ACR?
the method i saw on Resolve is rubbish.
Title: Re: The CinemaDNG Discussion
Post by: sarotaz on July 09, 2013, 11:46:45 PM
my workflow is as follow.
RAW>raw2cdng>CDNG sequence>Resolve>premiere>Resolve for final render

you can use adobe cinema dng importer (for adobe premiere cs 5.5 but it works with cs6 too) for import your cdng sequences in premiere. After you edit your movie, you can import xml and conform your sequence in Resolve then grading and finally render.
Title: Re: The CinemaDNG Discussion
Post by: mageye on July 10, 2013, 01:21:45 AM
my workflow is as follow.
RAW>raw2cdng>CDNG sequence>Resolve>premiere>Resolve for final render


anyone know a way to properly white balance in ACR?
the method i saw on Resolve is rubbish.

I don't know how you are white balancing in ACR?, but if there is a reliable reference white in the frame then you can use the picker tool at the top.

My white balance is a more involved process. I try to take an image of a dedicated white card/grey card. I am only using one of the cheaper ones, but they seem to be as accurate as I need them to be.

I will usually take a normal shot of the white card (as a photo)in the light conditions that I am in and then use that for the following videos in that light by saving the white balance collected from that image (as a preset).

Also I try to get a white balance for every single shot but that can just be too time consuming and frankly it's not something you can do for every single shot, every time.
Title: Re: The CinemaDNG Discussion
Post by: hjfilmspeed on July 10, 2013, 01:26:10 AM
The 10bit log mode is a lot of fun!!! has a real nice filmic look to it.
Title: Re: The CinemaDNG Discussion
Post by: vikado on July 10, 2013, 03:02:32 AM
you can use adobe cinema dng importer (for adobe premiere cs 5.5 but it works with cs6 too) for import your cdng sequences in premiere. After you edit your movie, you can import xml and conform your sequence in Resolve then grading and finally render.
where do i download this importer?
so this workflow is RAW>>ACDI>>premiere>>resolve?
there's no use of proxy files?
this seems like an easier way as i dont have to do no goofy roundtripping.

I don't know how you are white balancing in ACR?, but if there is a reliable reference white in the frame then you can use the picker tool at the top.

My white balance is a more involved process. I try to take an image of a dedicated white card/grey card. I am only using one of the cheaper ones, but they seem to be as accurate as I need them to be.

I will usually take a normal shot of the white card (as a photo)in the light conditions that I am in and then use that for the following videos in that light by saving the white balance collected from that image (as a preset).


in ACR, i change "As shot" to "Auto".
it does a fantastic job of white balancing.
unless there's a more "pro" manual route.

Also I try to get a white balance for every single shot but that can just be too time consuming and frankly it's not something you can do for every single shot, every time.
in my last shoot, we used a calibration target in conjunction with the slate for every take in a scene.
the calibration target was used to denoise footage in post. (i used neatvideo to denoise the footage).
obviously using the target for every take would be a pain in the ass, but we soon learned that using the target only on the first set of takes in a new scene was sufficient.
we only used the target in a new scene. not necessarily all takes of the same scene.
hope i explained it clearly.

edit:in new scene, i meant not new camera angle, i mean new physical scene (i.e., outdoors to indoor, or other room)
Title: Re: The CinemaDNG Discussion
Post by: vikado on July 10, 2013, 03:32:42 AM
wait, does premiere convert these files to 8-bit or does it retain the 12-bit color space when i process the raw files in raw2CDNG?
Title: Re: The CinemaDNG Discussion
Post by: sarotaz on July 10, 2013, 09:18:29 AM
where do i download this importer?
so this workflow is RAW>>ACDI>>premiere>>resolve?
there's no use of proxy files?
this seems like an easier way as i dont have to do no goofy roundtripping.

https://dl.dropboxusercontent.com/u/26702029/cinemadng_p4_cs5-5importer_060211.zip (https://dl.dropboxusercontent.com/u/26702029/cinemadng_p4_cs5-5importer_060211.zip)

I've tested it with premiere cs6 and a short example sequence from blackmagic cinema camera and works perfect.


Follow this tutorial. You can adjust white-balance, black point, white point ecc. like ACR.  :)

Hope this helps.
Title: Re: The CinemaDNG Discussion
Post by: vikado on July 10, 2013, 08:26:08 PM
https://dl.dropboxusercontent.com/u/26702029/cinemadng_p4_cs5-5importer_060211.zip (https://dl.dropboxusercontent.com/u/26702029/cinemadng_p4_cs5-5importer_060211.zip)

I've tested it with premiere cs6 and a short example sequence from blackmagic cinema camera and works perfect.


Follow this tutorial. You can adjust white-balance, black point, white point ecc. like ACR.  :)

Hope this helps.

thanks!
just tried this myself.
everything is fine except for realtime playback. it chugs along. resolution at 1880x800 @ 12bit.
the same sequence plays fine on resolve but its having a difficult time in premiere.
i've tried cs5.5 and CC. and its the same.
i have a modest i7 2600k OC at 4.9ghz, 32GB 1600 memory.
i've edited R3D 5k footage before as a 1080p sequence on premiere and my computer has had no issues crunching those files.

i just wanted access to ACR in my workflow for proper balance and noise reduction. :/
plus premiere converts these to 8-bit color space, this wont be acceptable in the silver screen.
bummer.
Title: Re: The CinemaDNG Discussion
Post by: DTSET123 on July 10, 2013, 08:33:04 PM
I thought that was a 8 bit importer  :o
I tried to import 12 bit footage and it could't do so- I wonder what I did wrong?
Title: Re: The CinemaDNG Discussion
Post by: pascal on July 10, 2013, 10:13:00 PM
Hi, thank you very much for this fine software.
Which bit depth do I need to choose for ML raw on 5D3? Thank you.
Title: Re: The CinemaDNG Discussion
Post by: scarluuk on July 10, 2013, 11:21:18 PM
Just found a way so that Davinci Resolve Recognizes CinemaDNG sequences while they still remain in the folder where all the files are.
This makes it a lot easier sorting and browsing trough your video's without having to open an extra folder before you can see it.
I don't know if i'm terribly late with this discovery or one of the first but just wanted to share it with you guys.
You can see what i mean in this picture:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fimg849.imageshack.us%2Fimg849%2F2374%2F65ql.png&hash=7f2470ee18a78fbe04ce6a17ee9ec162)

This saves a lot of time and frustration, now i don't have to put all the CinemaDNG files from different video's in one folder so i could browse through them.
What i did was take the name of the folder of a shoot i did with the Blackmagic Cinema Camera and just renamed one of my 5D RAW map names to this new one.

Edit: Just scraped a lot of my findings since i just found out that all you have to do is add a C befor the last four numbers.
Example, if your file is called "M16-2227" all you have to do is change it to "M16-C2227" and Davinci Recognizes it as a movie instead of a folder
Title: Re: The CinemaDNG Discussion
Post by: sarotaz on July 11, 2013, 08:48:09 AM
@chmee

any news for bad pixel opcode tag in your converter?
Title: Re: The CinemaDNG Discussion
Post by: fatpig on July 11, 2013, 11:05:03 AM
@scarluuk:
I dont have to change anything in the names, and can use the CDNG folders in resolve rightaway.
Title: Re: The CinemaDNG Discussion
Post by: scarluuk on July 12, 2013, 06:57:49 PM
@scarluuk:
I dont have to change anything in the names, and can use the CDNG folders in resolve rightaway.

It's a bit hard to explain for me, English isn't my native language.
Adding the C makes Resolve interpret the folder as a clip instead of a folder.
Normally you have to open the folder the see the clip, now you can see the folder itself as a clip so it's easyer to browse through your clips.
Title: Re: The CinemaDNG Discussion
Post by: Midphase on July 12, 2013, 09:42:47 PM
everything is fine except for realtime playback. it chugs along. resolution at 1880x800 @ 12bit.
the same sequence plays fine on resolve but its having a difficult time in premiere.
i've tried cs5.5 and CC. and its the same.
i have a modest i7 2600k OC at 4.9ghz, 32GB 1600 memory.
i've edited R3D 5k footage before as a 1080p sequence on premiere and my computer has had no issues crunching those files.

It appears as though the CinemaDNG import "experimental" plugin is no longer available from Adobe. They cite poor real-time performance as the primary reason for pulling the plugin from the Labs section.

I too have had a great deal of positive experience working with .r3d files within Premiere, even on an aging 1st gen Mac Pro with no Mercury acceleration. It seems that if Resolve is able to provide realtime or near-realtime playback of these files, Premiere Pro should too.

Please consider sending Adobe a feature request, the more they hear from our community, the more likely they are to revisit the functionality.
Title: Re: The CinemaDNG Discussion
Post by: Midphase on July 12, 2013, 11:00:54 PM
Adding the C makes Resolve interpret the folder as a clip instead of a folder.

You sure about this? If I add a C to the name, I can't get Resolve to see anything. Is this a Mac/PC thing?

I can't  get it to work, all I know is that the way it works without the C seems pretty efficient and fast.
Title: Re: The CinemaDNG Discussion
Post by: Anisotropic on July 13, 2013, 07:36:08 AM
Try to new Resolve 9.1.5, in changelog they wrote "Improved DNG format support". maybe this was fixed.
Title: Re: The CinemaDNG Discussion
Post by: sarotaz on July 13, 2013, 08:59:50 AM
Try to new Resolve 9.1.5, in changelog they wrote "Improved DNG format support". maybe this was fixed.

THANKS!! New version of resolve read dng sequence from raw2dng!
Title: Re: The CinemaDNG Discussion
Post by: pascal on July 14, 2013, 12:59:13 PM
I experienced crashes with raw2cdng when coverting some raw movies from 5D3.
Title: Re: The CinemaDNG Discussion
Post by: chmee on July 14, 2013, 05:08:40 PM
Hey there, back from holiday (picture? (http://www.flickr.com/photos/chmee/9269206557/)):) So.. where to start. With the important thing: With resolve 9.15 support of 14bit dng's i dont see any need paying more attention to my app. (i'm a bit puzzled, blackmagic did that - thought, they have some more important things to code and to distant their own camera - but finally they get more users with this decision.) if there's no real pro argument, v1.2.1 will stay the last one :)

@pascal - no infos, no help.

@all - thanks to all, especially them used my tool.
Title: Re: The CinemaDNG Discussion
Post by: Andy600 on July 14, 2013, 05:18:52 PM
Hey there, back from holiday :) So.. where to start. With the important thing: With resolve 9.15 support of 14bit dng's i dont see any need paying more attention to my app. (i'm a bit puzzled, blackmagic solved that - thought, they have some more important things to code and to distant their own camera - but finally they get more users with this decision.) if there's no real pro argument, v1.2.1 will stay the last one :)

@pascal - no infos, no help.

@all - thanks to all, especially them used my tool.

Hi Chmee - Welcome back! :)

Even with the Resolve update there are still benefits to using your app. It's simple to use for a start. The 12bit compression is useful for saving space and I like being able to convert from the card to a HD folder without copying the raw files. I hope you continue developing it.
Title: Re: The CinemaDNG Discussion
Post by: IliasG on July 14, 2013, 09:57:29 PM
Hey there, back from holiday (picture? (http://www.flickr.com/photos/chmee/9269206557/)):) So.. where to start. With the important thing: With resolve 9.15 support of 14bit dng's i dont see any need paying more attention to my app. (i'm a bit puzzled, blackmagic did that - thought, they have some more important things to code and to distant their own camera - but finally they get more users with this decision.) if there's no real pro argument, v1.2.1 will stay the last one :)

@pascal - no infos, no help.

@all - thanks to all, especially them used my tool.

Well come back.

I was expecting you back on July 17 so I didn't upload a pack of LUTs (I added 12bit log also) which are ready. I just want to do some refining and maybe include linearization to 14bit now that Resolve supports it.  If nothing goes wrong I 'll upload in the next 12 hours.

Do the new Resolve support losslessly compressed DNGs as those we take after running through Adobe DNG converter ?.

I think 10bit log (and 12bit) losslesly compressed is a great space saver. Everyone needs it.

Title: Re: The CinemaDNG Discussion
Post by: pascal on July 14, 2013, 10:36:49 PM
@pascal - no infos, no help.
Hi, I wasn't sure if someone was interested in details. Thanks for your interest.
I uploaded a RAW file that crashes the program upon processing.

http://www.speedyshare.com/XQHVT/M13-1629.RAW
(click on Download: M13-1629.RAW on top)

PS: there is no option in raw2cdng for 14 bit DNG? 10, 12, 16, what does ML Raw need?
Title: Re: The CinemaDNG Discussion
Post by: chmee on July 14, 2013, 11:14:30 PM
@andy600 thx appreciate that :)

@iliasG :D ok, so "our" benefit is the space saving :) beside the bugpatching the next bigger step will be finishing 10bit/log and 12bit/log with compression. fine fine :)

@pascal i removed 14bit, because it was not needed (it was in the first versions) - resolve couldnt handle that and apps like in the adobeworld dont need cdng (ok, speedgrade, but this one needs 10 12 or 16bit) - tomorrow i'll download your file and try to figure out, whats happening

@fatpig you're not forgotten.. but the cards now are reshuffled (is it a known english phrase? :)) - the users can use the official raw2dng to get into resolve - we try to optimize the filesizes without losing quality. to be honest, i have to learn/figureout about commandline how to implement it into an existing winform-app. ;)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: pascal on July 14, 2013, 11:18:30 PM
i removed 14bit, because it was not needed (it was in the first versions) - resolve couldnt handle that and apps like in the adobeworld dont need cdng (ok, speedgrade, but this one needs 10 12 or 16bit) - tomorrow i'll download your file and try to figure out, whats happening
Hmm I use DNG with Adobe. Why would Adobe users not need DNG?
Title: Re: The CinemaDNG Discussion
Post by: chmee on July 14, 2013, 11:54:36 PM
for them raw2dng is the right converter.
Title: Re: The CinemaDNG Discussion
Post by: pascal on July 15, 2013, 12:17:39 AM
Is there a way to batch convert conveniently with raw2dng.exe?
Title: Re: The CinemaDNG Discussion
Post by: Redrocks on July 15, 2013, 12:59:15 AM
Edit: Just scraped a lot of my findings since i just found out that all you have to do is add a C befor the last four numbers.
Example, if your file is called "M16-2227" all you have to do is change it to "M16-C2227" and Davinci Recognizes it as a movie instead of a folder

Works for me on Windows, although it introduces the same problem AE does in that all my clips come in as
"DNG[000000-000670000million].dng". I'll find a way round that though. Great find, Scarluuk.

Welcome back, Chmee. Your app was vital till BM upgraded Resolve, as was Fatpig's till I needed to use Resolve, so thank you both for that. What I like about R2C is its ease of use, whilst BATCHelor is great for organising projects. RAWanizer has its uses though - the file information it provides and it opening the last folder you selected upon restart. I'd love a raw converter that would combine all of these things and maybe use a Resolve style GUI that exchanged Media, Conform, Colour... with Basic, On set, Power user, Info... Either way, the work done by you guys on the raw converters doesn't go unappreciated at all.
Title: Re: The CinemaDNG Discussion
Post by: fatpig on July 24, 2013, 09:48:46 AM
@pascal:
http://www.magiclantern.fm/forum/index.php?topic=5645.125

@chmee:
ok, i get it.
still: raw2dng.exe does not support split files.
Title: Re: The CinemaDNG Discussion
Post by: chmee on August 12, 2013, 12:14:02 AM
Hey there, i'm much workin.. so, finally i dont really will continue the raw2cdng-tool - all changes and small patches (if..!) are only tests for the forthcoming mlv2cdng-tool - MLV is in every form the better approach to stabilize the raw-revolution - and mlv2cdng will be coded from scratch as a commandline-tool with an external GUI. please be patient.

But back to raw2cdng: i thought about integrating exif-data from a cr2-file into the metadata of the dng. by now i only read the RGGB_Levels_measured from the cr2-makernotes and convert them into the dng-AsShotNeutral-Values. If someone's up to test that, feel free to write, if it helps/works in ACR or Resolve.

The Approach is easy - right before recording movie, do a photo. put both into the same path and give the CR2-file the same name.

Example:
M02-2221.RAW
M02-2221.CR2

raw2cdng will recognise the file automatically (it will be shown up with +CR2 in the importlist, same as splitted files..). It's tested by now only with a 5DIII. By the Way, if you're recording a whole set with the same whitebalance, you can name this one photofile ALL.CR2 and all files in the importlist will use this data - except the files having their own CR2.

Example:
M02-2201.RAW (will use ALL.CR2)
M02-2221.RAW (will use M02-2221.CR2)
M02-2221.CR2
M02-2312.RAW (will use ALL.CR2)
ALL.CR2

raw2cdng v1.3.0 - with CR2-whitebalance (http://dl.phreekz.de/raw2cdng/raw2cdng.1.3.0.zip)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: vikado on August 12, 2013, 01:09:06 AM
oh wow, that's a fantastic addition chmee!
i did a quick test myself. color balance looks spot on.
that's one less step on resolve for color balancing!
awesome job, bro!  ;D
Title: Re: The CinemaDNG Discussion
Post by: Canon eos m on August 13, 2013, 05:29:33 PM
Hey there, i'm much workin.. so, finally i dont really will continue the raw2cdng-tool - all changes and small patches (if..!) are only tests for the forthcoming mlv2cdng-tool - MLV is in every form the better approach to stabilize the raw-revolution - and mlv2cdng will be coded from scratch as a commandline-tool with an external GUI. please be patient.

But back to raw2cdng: i thought about integrating exif-data from a cr2-file into the metadata of the dng. by now i only read the RGGB_Levels_measured from the cr2-makernotes and convert them into the dng-AsShotNeutral-Values. If someone's up to test that, feel free to write, if it helps/works in ACR or Resolve.

The Approach is easy - right before recording movie, do a photo. put both into the same path and give the CR2-file the same name.

Example:
M02-2221.RAW
M02-2221.CR2

raw2cdng will recognise the file automatically (it will be shown up with +CR2 in the importlist, same as splitted files..). It's tested by now only with a 5DIII. By the Way, if you're recording a whole set with the same whitebalance, you can name this one photofile ALL.CR2 and all files in the importlist will use this data - except the files having their own CR2.

Example:
M02-2201.RAW (will use ALL.CR2)
M02-2221.RAW (will use M02-2221.CR2)
M02-2221.CR2
M02-2312.RAW (will use ALL.CR2)
ALL.CR2

raw2cdng v1.3.0 - with CR2-whitebalance (http://dl.phreekz.de/raw2cdng/raw2cdng.1.3.0.zip)

regards chmee

can't wait for the mlv2cdng  . . fingers crossed
Title: Re: The CinemaDNG Discussion
Post by: araucaria on August 13, 2013, 05:55:35 PM
@chmee

Maybe this has already been answered but, if I got it right the cdng format is just a tiff with some metadata (or whatever) this means it's already debayered, right? But when I put these clips into Resolve (coming from a 50d) I get false colores and artifacts from debayering, although they show up correctly in ACR. I don't get this  ???
Title: Re: The CinemaDNG Discussion
Post by: chmee on August 13, 2013, 06:54:54 PM
..if I got it right the cdng format is just a tiff with some metadata (or whatever) this means it's already debayered, right..
* tiff is a containerformat.
* tiff can contain pattern-oriented picturedata as well, here bayer-pattern from a sensor.
* dng is in general a tiff with some mandatory tags in the metadata
* btw CR2 is as well a tiff with another file-ending ;)

saying this is very important for me :)
* cinemadng is quite the same as dng, with some videorelevant tags (f.i. framerate, reelname, timecode)

shortly saying, no, the dngs (cdngs too) are not debayered - and by now i think, the debayering-algorithm in resolve is not the best - far away from the adobe-world, which is able to de-vignette or remove badpixels (automatically or/and with help of metadata) and so on..

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: araucaria on August 13, 2013, 10:39:58 PM
Thank you :D
Title: Re: The CinemaDNG Discussion
Post by: Canon eos m on August 14, 2013, 04:22:01 PM
can't wait for the mlv2cdng  . . fingers crossed

Hi,

I am missing something here. I copied a *.CR2 file and name it the same as the RAW video file and dragged both on the raw2cdng convertor. The interface opens. What next?
Title: Re: The CinemaDNG Discussion
Post by: chmee on August 14, 2013, 08:44:01 PM
? You just drag/drop the RAW-File into the list inside the exe - NOT on the exe-icon! The Converter recognises itself, if the file is splitted and if a cr2-file exists. if it finds splitted files, it will be shown in the importlist with (+[Number]). if it finds a corresponding cr2-file, it will be shown as (+CR2). Thats it. Then convert via button.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: vikado on August 14, 2013, 08:45:20 PM
Hi,

I am missing something here. I copied a *.CR2 file and name it the same as the RAW video file and dragged both on the raw2cdng convertor. The interface opens. What next?
open the utility first and then drag and drop the .cr2 file and the raw. you'll see a (+cr2) after the raw file.
plus, you dont have to rename the .cr2 file, and if you do, just rename it "ALL.cr2" like he mentioned and use that for all your conversions.

btw, chmee, i noticed a noticeable difference of white balance between your utility and adobe camera raw(7.0).
adobe camera raw white balance was set to "auto". temperature at  "2850". and Tint set to "+24"
the light source was a typical tungsten lightbulb on a lamp, where the shade of the lamp gave off a warmer diffusion of the light.
i felt the proper white balance was more true on adobe camera raw vs your utility.
even though yours looked right, it was a bit off based on the light source.
hope this was insightful :)
link below of comparison.

http://i.imgur.com/cFDv5B2.jpg
Title: Re: The CinemaDNG Discussion
Post by: chmee on August 14, 2013, 09:02:23 PM
@vikado because i m out of home im thankful to see some picture-examples and im excited, my idea works :) So.. facing this example i'm not sure, if my whitebalance-correction is needed.. Did you try (asShot) in ACR? My idea was indeed a neutral whitebalance, not a pleasing (subjective!) one - the question is, what is easier to handle? changing a neutral picture into a warm one or correcting a tint into neutral :)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: vikado on August 14, 2013, 10:08:48 PM
@vikado because i m out of home im thankful to see some picture-examples and im excited, my idea works :) So.. facing this example i'm not sure, if my whitebalance-correction is needed.. Did you try (asShot) in ACR? My idea was indeed a neutral whitebalance, not a pleasing (subjective!) one - the question is, what is easier to handle? changing a neutral picture into a warm one or correcting a tint into neutral :)

regards chmee
here is the breakdown between the two files. http://i.imgur.com/JRuSweJ.jpg
left side is the dng with 1.3.0 and the right is the 1.2.1.
both having "as shot" and "auto" for comparison.


based on what the light source is (single tungsten), you'll see that ACR set to auto is the most accurate, in my humble opinion.

Quote
the question is, what is easier to handle? changing a neutral picture into a warm one or correcting a tint into neutral :)
changing the kelvin values and not touching the tint is best, (i think)
Title: Re: The CinemaDNG Discussion
Post by: nico on August 14, 2013, 10:30:19 PM
Hi, I am new in the magic lantern world, just got a brand new 5d m3, I was wondering, what kind of a computer do I need to process all this data? Thanks!
Title: Re: The CinemaDNG Discussion
Post by: vikado on August 14, 2013, 10:41:08 PM
Hi, I am new in the magic lantern world, just got a brand new 5d m3, I was wondering, what kind of a computer do I need to process all this data? Thanks!
just to process raw to dng sequence or dnxhd444/prores444?
you dont need a lot of cpu power to crunch through raw files.
the difficulty is playing back those DNGs in realtime on resolve or your NLE.
i have two computers running an overclocked 2600k set to 4.9ghz, 32gb 1600mhz RAM, one has a gtx 570 and the other a gtx 580.
playback in realtime is fine. i use proxies on premiere so the real resource hog is in resolve, where i'll get dropped frames during playback when i start to use noise reduction on a node.
so my recommendation is to get a gtx 680 or above to be on the safe side or maybe a 670 if you're short on funds.
as for myself, im going to buy a gtx780 before we start on our feature.
Title: Re: The CinemaDNG Discussion
Post by: chmee on August 14, 2013, 11:02:58 PM
@vikado yeah, if i see the comparison, the auto-definition made by adobe is a pleasing one - by "mathematical" view i'd say, the AsShot-measured RGB-Values are ok, because neutral-gray :) have to sleep and sit back home on my desktop on that :)
Title: Re: The CinemaDNG Discussion
Post by: Mei Lewis on August 16, 2013, 11:16:39 AM
The Resolve readme for the latest versions says:

"What's new in DaVinci Resolve 9.1.5
...• Improved DNG format support
"

Is it really DNG they mean or cinemaDNG?

Can't get ordinary photo DNG recognised.
Thanks

Title: Re: The CinemaDNG Discussion
Post by: araucaria on August 16, 2013, 07:05:03 PM
Since Davinci Resolve won't work fine (even after blurring the chroma,etc...) I wanted to try out speedgrade. It plays the cdngs at realtime (with a shitty 560ti, resolve wouldn't do more than 12fps with 1080p footage) but everything has a green cast to it. Is there already a cure? I'm a total n00b to speedgrade, so any help is welcome!
Title: Re: The CinemaDNG Discussion
Post by: chmee on August 16, 2013, 07:48:40 PM
The Adobeforum's got some really good guys - there are already some threads about it. bmcc-user have this problem as well.
the weakest (with good links in it ) :) http://forums.adobe.com/message/5579981

regards chmee

edit: please do me a favour: open one (ML)dng with ACR - save it as 16bit-DNG-uncompressed and import it in speedgrade. Did it changed its behaviour?
Title: Re: The CinemaDNG Discussion
Post by: araucaria on August 17, 2013, 01:35:26 PM
It won't show, it only accepts footage in cdng. I took a small cdng secuences made with your converter and opened it in PS as 16bit then clicked save as, DNG, uncompressed, and then tried to open in Speedgrade without luck (it won't render it "Failed to decode frame).

I'll try to read through the Adobe forums, thanks a lot!
Title: Re: The CinemaDNG Discussion
Post by: chmee on September 09, 2013, 03:06:56 PM
marvelous News - Premiere CC Update in October will give us native CDNG-Support.
http://blogs.adobe.com/premierepro/2013/09/adobe-premiere-pro-cc-october-2013-release.html

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: Davidos on September 11, 2013, 09:16:22 PM
... and Cinema DNG can be debayered on a supported GPU (in Premiere CC October release) for even better performance.  :) ;) :'(
Title: Re: The CinemaDNG Discussion
Post by: hookah on October 08, 2013, 02:39:46 PM
i got the chance to play with some cdng sequences in the new adobe CC (converted via this tool) - works okay - but the picture turns out very pinkish (maybe some metadata are missing ?? or maybe premiere dosent read the black/white levels correct).

But can be played in realtime in fullres. :-D
 - a shame that ARC is not available as in AE.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fs9.postimg.org%2Fx6y3g2y7z%2Fwoods.jpg&hash=b71a8f79b1083781aa015421a7d51e4b)
Title: Re: The CinemaDNG Discussion
Post by: mityazabuben on October 10, 2013, 01:21:31 PM
Hi. When I try to open a folder, where imported by raw2cdng files are located, in Resolve it hangs up and closed with error. What can be wrong? Im using 16bit linear and 15% white unchecked when importing. AE opens without problem.
Some example of DNGs:
https://drive.google.com/folderview?id=0B2NWwDUimG9cOGEyOGEwMjUtZDYyNC00ZmUyLWIyZDgtMzhmNjE4ZWU5NTMy&usp=sharing
Title: Re: The CinemaDNG Discussion
Post by: bnvm on October 10, 2013, 04:46:31 PM
I have seen green shifting with cdng's as well. I believe it is from incorrect color matrix data in the meta data. I tried a sequence in another app and got a green shift, setting the app to not use the color matrix fixed the issue. I guess it is possible that the magenta shifting is from the same issue.
Title: Re: The CinemaDNG Discussion
Post by: vikado on October 12, 2013, 11:33:52 AM
im getting green fringing with the oct.4 build for my mk2.
but only on the 10-bit, but 12bit and 16bit are fine.
this is for the 1.3.0 version
Title: Re: The CinemaDNG Discussion
Post by: mityazabuben on October 14, 2013, 12:04:37 PM
Hi. When I try to open a folder, where imported by raw2cdng files are located, in Resolve it hangs up and closed with error. What can be wrong? Im using 16bit linear and 15% white unchecked when importing. AE opens without problem.
Some example of DNGs:
https://drive.google.com/folderview?id=0B2NWwDUimG9cOGEyOGEwMjUtZDYyNC00ZmUyLWIyZDgtMzhmNjE4ZWU5NTMy&usp=sharing
Resolve 10.b3 works fine.
Title: Re: The CinemaDNG Discussion
Post by: chmee on October 16, 2013, 11:45:00 AM
after investing time and brain for work and other projects i'll spend next time back on raw2cdng and mlv2cdng. next days i'll analyze the color shifts. btw. 10bit log output is ONLY for bodies with blacklevel ~2048 - i think, in raw2cdng i'll code a selector depending on body if 10bit-output is enabled or not. (multithreaded transcoding almost done, will be in v1.3.5)

@bnvm
interesting opinion, i'll look if its true.

The crashes in beta 2 are confirmed by other users (in rawmagic-thread) - solved with beta 3 as said by mityazabuben.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: vikado on October 20, 2013, 02:53:06 AM
Hi. When I try to open a folder, where imported by raw2cdng files are located, in Resolve it hangs up and closed with error. What can be wrong? Im using 16bit linear and 15% white unchecked when importing. AE opens without problem.
Some example of DNGs:
https://drive.google.com/folderview?id=0B2NWwDUimG9cOGEyOGEwMjUtZDYyNC00ZmUyLWIyZDgtMzhmNjE4ZWU5NTMy&usp=sharing
i had the same issue.
do you have a graphics card installed?
if you do, turn off your integrated graphics chip in your bios.
i turned mines off and it solved the problem.
something about how resolve crashes if there's a gpu and integrated gpu both on.
Title: Re: The CinemaDNG Discussion
Post by: DTSET123 on October 31, 2013, 11:09:02 PM
Quote
after investing time and brain for work and other projects i'll spend next time back on raw2cdng and mlv2cdng. next days i'll analyze the color shifts. btw. 10bit log output is ONLY for bodies with blacklevel ~2048 - i think, in raw2cdng i'll code a selector depending on body if 10bit-output is enabled or not. (multithreaded transcoding almost done, will be in v1.3.5)

Thank you Chmee! Your RAW2CDNG is a great solution, if you could fix this problem it would be greatly appreciated!
Title: Re: The CinemaDNG Discussion
Post by: painya on November 01, 2013, 03:37:09 AM
Hello. with the October update of SpeedGrade the program doesn't want to handle them. Odd right? Premier does well with them, but SG just crashes. Any ideas?
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 01, 2013, 12:07:07 PM
because i'm no subscriber of Adobe, i've got no "new" adobe-apps. simply i can't check.
(but im thinkin' about it after reading some workflow-experiences)
Title: Re: The CinemaDNG Discussion
Post by: eoshq on November 01, 2013, 06:04:38 PM
because i'm no subscriber of Adobe, i've got no "new" adobe-apps. simply i can't check.
(but im thinkin' about it after reading some workflow-experiences)

chmee, As of right now, we have no way to use 5D3 DNG or CinemaDNG in the new Premiere Pro CC 7.1

DNG (with newest raw2dng) will not import because it says there is no metadata, stops dead on error without importing.

CDNG (with newest raw2cdng) imports but has a terrible pink cast which cannot be removed.

If there was some way around this via a new raw2cdng or raw2dng we would be very grateful for the chance to try this new workflow.
Title: Re: The CinemaDNG Discussion
Post by: hookah on November 01, 2013, 06:16:00 PM
DNG (with newest raw2dng) will not import because it says there is no metadata, stops dead on error without importing.
Think thats because its 14bit.

CDNG (with newest raw2cdng) imports but has a terrible pink cast which cannot be removed.
Read more here about it: http://www.magiclantern.fm/forum/index.php?topic=9131.msg86286;topicseen#msg86286

Title: Re: The CinemaDNG Discussion
Post by: dude on November 01, 2013, 06:46:41 PM
As far as i can say from premiere cs6, it does not import any 14 bit files, except via dynamik link
Title: Re: The CinemaDNG Discussion
Post by: a1ex on November 01, 2013, 06:48:04 PM
Does it import dual ISO files? (DNGs saved by cr2hdr - they are 16 bit)
Title: Re: The CinemaDNG Discussion
Post by: dude on November 01, 2013, 06:52:20 PM
If you re talking about premiere cs6, i can try it if someone has a file for me
Title: Re: The CinemaDNG Discussion
Post by: hookah on November 01, 2013, 07:32:54 PM
Does it import dual ISO files? (DNGs saved by cr2hdr - they are 16 bit)

nope, same error as when processing with raw2dng.. (would not take them at all - says "The file appears to have no media data")
So maybe its more a header (DNG vs CDNG) problem - than a 14bit vs 16bit thing (or could be a mix of both?)..
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 01, 2013, 07:43:47 PM
In raw2cdng 1.3 there is the chance to import the whitebalance-data from a taken cr2-photo. did you tried that? did it changed the colors? description is in blog entry. http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: hookah on November 01, 2013, 08:10:05 PM
In raw2cdng 1.3 there is the chance to import the whitebalance-data from a taken cr2-photo. did you tried that? did it changed the colors? description is in blog entry. http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

regards chmee

Yep, tried that - did not remove the pinky stuff :-)

But if i change the bit depth to 10bit CDNG via your tool it looks better (easier to fix)  8)
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fs15.postimg.org%2F734ptwux7%2Fwoods10bit.jpg&hash=675843a0dff7e7d66d7457c5e488e402)
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 01, 2013, 08:28:34 PM
:D nevermind, decided to jump on the cc-train. so will check that. please be patient 8)
Title: Re: The CinemaDNG Discussion
Post by: eoshq on November 01, 2013, 08:38:30 PM
:D nevermind, decided to jump on the cc-train. so will check that. please be patient 8)

Great news, Thank you 1000 times for trying to help us.

I read today on the Adobe forum that they never even tried other cameras besides BMCC, but they did not tell anyone that until today. Adobe are the worst at deceptive marketing, they knew since the September 8th announcement that CinemaDNG support was only going to work with BMCC files. Despicable.
Title: Re: The CinemaDNG Discussion
Post by: vikado on November 01, 2013, 08:51:06 PM
:D nevermind, decided to jump on the cc-train. so will check that. please be patient 8)
awesome.
this is what you call dedication.
Title: Re: The CinemaDNG Discussion
Post by: a1ex on November 01, 2013, 09:26:12 PM
In raw2cdng 1.3 there is the chance to import the whitebalance-data from a taken cr2-photo. did you tried that?

It's not a WB issue, but a black level one. The Adobe editor reads the black level most likely as 0, instead of 1024 or 2048 or whatever the right value is.

My hypothesis: the metadata in the CDNG files is corrupted (http://www.magiclantern.fm/forum/index.php?topic=9131.msg86286;topicseen#msg86286) (e.g. having a wrong type for a tag can cause read errors on all subsequent tags in an IFD), and some parsers happen to get over it and others not, depending on how they choose to handle this error. If this is true, the problem is in the DNG converter, not on Adobe's side.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 01, 2013, 10:05:54 PM
ahh, thanks. i used the validator long time ago.. generally i saw the problem on my side, of course.
Title: Re: The CinemaDNG Discussion
Post by: jrumans on November 02, 2013, 05:28:42 AM
- a shame that ARC is not available as in AE.

Have you tried right clicking on the dng and clicking "Edit Original"?  It opens ACR for me.


(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fs14.postimg.org%2Fppin3moo1%2FScreen_Shot_2013_11_01_at_11_19_24_PM.png&hash=435949889356e37e7c12d92632f6c0f4)
Title: Re: The CinemaDNG Discussion
Post by: eoshq on November 02, 2013, 07:18:11 AM
Have you tried right clicking on the dng and clicking "Edit Original"?  It opens ACR for me.

And what does this accomplish? Absolutely nothing! Sure it will open the first cdng file of the sequence in Photoshop hosted camera raw, but it doesn't do anything to help this pink cast situation. What is the point of your posting this? Do you not try things before you post about them on a forum?
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on November 02, 2013, 07:29:59 AM
but it doesn't do anything to help this pink cast situation.
I found a work around , with AE link below
http://www.magiclantern.fm/forum/index.php?topic=8177.msg86395#msg86395
Title: Re: The CinemaDNG Discussion
Post by: eoshq on November 02, 2013, 07:57:37 AM
I found a work around , with AE link below
http://www.magiclantern.fm/forum/index.php?topic=8177.msg86395#msg86395

This really gains us nothing because you could have been doing this all along, and all the way back to at least CS5, maybe further. There has been a Dynamic Link between After Effects and  Premiere pro for a long time. So, yes this does get your cdng (or dng, BTW) footage into Premiere Pro without the pink cast, but I would not call it a workaround. It is really apples and oranges, two totally different processes. Let me explain further. This way negates any advantages of the new cdng functionality of PP 7.1. By using Dynamic Link back to After Effects it is massively slower and using tons of extra CPU because it is literally using After Effects to render the footage in the background. Check the Process viewer while doing this and you will see that an instance of After Effects is running along side Premiere Pro.

Oh and there is no reason to save a copy of your After Effects project as a CS6 copy when you do this Dynamic Link method as described in your video, not sure where that idea came from but it does nothing to benefit this process.
Title: Re: The CinemaDNG Discussion
Post by: dude on November 02, 2013, 12:36:06 PM
Sorry, but that s wrong.
At the moment, importing dngs in AE, making proxys and open this AE Sequence (not like in the video safe as premiere) in premiere, you have the best workflow.
no exporting, no xml, no metadata and no comlicated workflows with davinci.
unless Premiere is able to open dng s via acr, this is the best non destructiv workflow.
At the moment, Premiere only applys some Camera metadata to the dngs when opening them, no way to work with that.
Title: Re: The CinemaDNG Discussion
Post by: hookah on November 02, 2013, 12:59:45 PM
Sorry, but that s wrong.
At the moment, importing dngs in AE, making proxys and open this AE Sequence (not like in the video safe as premiere) in premiere, you have the best workflow.
no exporting, no xml, no metadata and no comlicated workflows with davinci.
unless Premiere is able to open dng s via acr, this is the best non destructiv workflow.
At the moment, Premiere only applys some Camera metadata to the dngs when opening them, no way to work with that.

This workflow was possible all the time - but its almost impossible to work that way. (even on my workstation 24core xeon, 24Gb ram, quadro 5000)
Its okay for small sequences/projects.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 02, 2013, 01:29:21 PM
..in terms of efficiency its wrong calling it workflow - its tea- and coffeetime with some moments of interacting  ;D
Title: Re: The CinemaDNG Discussion
Post by: dude on November 02, 2013, 02:32:30 PM
I just did a 200 GB project with this workflow,
it s the only one which holds up the full quality during the post process.
I am editing videos for about 15 years now, and with my new i7, i can play for and back in the timeline as i want it. i remeber some time when even viewing h264 footage was not really able with premiere.
Feel free to show a more efficient workflow...
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 02, 2013, 03:29:25 PM
discussion has been done in other threads - the resolve-roundtrip-workflow "was" an efficient one imho. the simple things to change, core-values like exposure or whitebalance means, re-rendering raw-files - a no-go (!) in terms of efficency(!). NOW we have a better resolve10 and hopefully a doped adobe-world. Did you try roundtrippin/dynamiclinking with speedgrade now? It's to me/to us to optimizing now the converter-tools. (davinci-xml-roundtrip was not complicated and gave you all the time control over these core-values. acr is the better debayer-tool, but fast? never.)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: mrnv45 on November 02, 2013, 08:59:37 PM
ive been doing this workflow and its been perfect since i cant run resolve with my dngs.. they force close my resolve
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 03, 2013, 02:06:03 AM
by now i sat at the pink or green cast in the raws - its not scientific, but a first help if you're using speedgrade with dynamic link. actually i downloaded speedgrade CC as a trial. give it a try, because: you're able to twist the wrong color matrix into a useable one. i bought for this reason a colorchecker passport, just to see how good the raws are.

(1) in "clip" you'll find the "color space", the third tab. to give you an idea of the direction i linked a picture with my settings on a greenish picture (dont know why its not magenta). you can make a first justage on greycards or some other black/white things. look on the histogram on the left to match the upper and lower boundaries, being same value.
http://dl.phreekz.de/raw2cdng/magiclantern_correctMLI.jpg
(2) after that i pushed the contrasts, working good with the colormatch-layer..
http://dl.phreekz.de/raw2cdng/magiclantern_correctMLII.jpg
(3) this example just took 10minutes to understand speedgrade and to solve..

here's one picture from the file
http://dl.phreekz.de/raw2cdng/colorchecker_example_16bit.dng
if you rebuild my example, you'll see, only with the green value for blacklevel you can barely eliminate green or magenta fill-light

dont hesitate, try this video to understand speedgrade.
http://tv.adobe.com/watch/no-stupid-questions-with-colin-smith/introduction-to-speedgrade/

but, i'm sitting on that to figure out, how to solve that without this rework. i assume, till now the apps didnt look into the colormatrices, speedgrade is the first one. (!) please do not forget, adobe will work with 10, 12 or 16bitfiles. not with 14bit.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: a1ex on November 03, 2013, 07:26:32 AM
you're able to twist the wrong color matrix into a useable one

A related trick: if you have problems with highlights, try turning off the color matrix. I've fixed this bug in ufraw (http://www.magiclantern.fm/forum/index.php?topic=5197) (basically I'm shifting the matrix towards eye(3) as you approach the clipping point), but judging from a recent sample from Audionut (http://www.magiclantern.fm/forum/index.php?topic=7139.msg85905#msg85905), Lightroom has the same problem (and probably all other Adobe apps).
Title: Re: The CinemaDNG Discussion
Post by: Malex on November 03, 2013, 01:16:28 PM
Hi Alex, what editing software are you using?
Do you think a modification in the DNG export softwares could fix that pink color in Premiere/speedgrade, I saw this question asked somewhere but with no answer, is it possible to export cdng from raw so that it would be like the blackmagic cdng ?
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 03, 2013, 01:57:55 PM
@malex we're workin' on that. (i wrote that 1 entry before!)
Title: Re: The CinemaDNG Discussion
Post by: eoshq on November 04, 2013, 07:37:46 AM
Guys we have been sold on a false idea when it comes to editing Cinema DNGs in the new Premiere Pro 7.1

Even when it works correctly, as with Black Magic Cinema Camera (BMCC) files, it is pretty much worthless as you lose the main reason for shooting raw in the first place, raw adjustments.

Over on the Creative Cow forums, one user has put it like this:

" It appears that Premiere Pro uses only part of the latitude of the RAW format, there's apparently no way to change it, and it's apparently suddenly impossible to access the full tonal range of RAW.

As an example, I have some footage with burnt out clouds. If I open a DNG image in Photoshop or After Effects, I can edit the exposure upon import, and bring back all the latitude on the clouds. That's the whole point of using RAW.

In Premiere Pro, CinemaDNG seems cropped to video levels, and I can't bring back any information in the highlights. I simply turn down blotchy over-exposed highlights."

You can read the whole thread here http://forums.creativecow.net/thread/3/947862

Sadly, as it stands now I see no point in editing cdng natively in Premiere Pro.
Title: Re: The CinemaDNG Discussion
Post by: hookah on November 04, 2013, 07:49:48 AM
Sadly, as it stands now I see no point in editing cdng natively in Premiere Pro.

Damm We was so close :-)

But adobe said:
We understand your frustration and are continuing to work hard on this. Getting correctly debaryed GPU accelerated CinemaDNG was quite a success for us. Adding the options is now possible.

So we must wait a little more..

Title: Re: The CinemaDNG Discussion
Post by: gary2013 on November 04, 2013, 10:00:03 AM
Guys we have been sold on a false idea when it comes to editing Cinema DNGs in the new Premiere Pro 7.1

I was upset after the latest PPro update. But, I have been happy using Ginger HDR workflow. It's pretty easy to wrap a DNG seq to a gnr file and import that gnr into PPro CC.

Gary
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 04, 2013, 10:42:24 AM
:) I'm a little bit sneering :) Adobe only said, Premiere will have cdng-support - they never said it will support full bitdepth-range including all effects. but it works good, if you're working with speedgrade as "controling-unit" via dynamic link. no coffeetime-pre-, re- or backgroundrendering (as with AE), and fulltime-control over the important values, in a significantly better and faster GUI than ae. Please, think about it - if you have been fooled by adobe or you have fooled yourself because of high-hopes - now you can put your cndg-sequences directly into premiere, then linking from there into the better tools to refine the footage - non-destructive.

But, i hope as well, they put at least a "simple" cdng-properties-window into premiere to change some core-values as wb/temperature and exposure.


my 2cents.  chmee
Title: Re: The CinemaDNG Discussion
Post by: gary2013 on November 04, 2013, 11:24:33 AM
I was avle to import ML DNG seqs before the 7.1 update. So, nothing new there. And I was able to get/use ACR on an import of ML DNG seqa and now that is gone with the latest AE CC update?? The least I wanted/expected was ACR on import of DNGs to PPro and to maintain access to the 14 bit raw for grading in PPro. If, if I needed more tools for grading, trehn a trip to SG. I was hoping to recall ACR in PPro like Photoshop has always done. Maybe ML can hack Photoshop into being a good raw video editor/color app?  :)

Last Sept when Adobe announced this update, I was talking in email to an Adobe rep and he did lead me to believe what I just said. I told him I was using Canon with ML and I wanted to import raw DNG seqs and edit them "natively" with all the beauty of the original raw 14 bit. He said that was coming in the Oct " 15th" 7.1 update. And now it's a different story. We need a new editor app from a small hip upstart company. They can be Black Lantern.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 04, 2013, 11:30:53 AM
..by now you've got Speedgrade as your Realtime ACR, with Premiere as the main Holder of your timeline - neither with proxies nor with baken 8bit-sequences. Thats my thoughts..
Title: Re: The CinemaDNG Discussion
Post by: gary2013 on November 04, 2013, 11:32:29 AM
Premiere, AE and SG are changing the DNG files. Didn't you read the thread we had in the COW forum? Per kivked it off wit-
Per Holmes on Oct 31, 2013 at 5:11:17 pm

Hi,

I just got the new Premiere Pro 7.1, and excitedly started testing the CinemaDNG workflow, and I've hit a massive wall. It appears that Premiere Pro uses only part of the latitude of the RAW format, there's apparently no way to change it, and it's apparently suddenly impossible to access the full tonal range of RAW, even with Speedgrade or After Effects.

As an example, I have some footage with burnt out clouds. If I open a DNG image in Photoshop or After Effects, I can edit the exposure upon import, and bring back all the latitude on the clouds. That's the whole point of using RAW.

In Premiere Pro, CinemaDNG seems cropped to video levels, and I can't bring back any information in the highlights. I simply turn down blotchy over-exposed highlights.

If I take the sequence over into SpeedGrade, it's the same. I'm not able to pull any information out of the clouds, only reduce the brightness of already blotchy over-exposed clouds.

If this really is designed this way, what the heck is the point? The ONLY, I feel I can safely repeat, ONLY reason to shoot RAW is to be able to access the full tonal range of the image. But Premiere Pro seems to crop the latitude range to a video range.

But then there's no point in shooting CinemaDNG, and not simply shoot ProRes.

What am I getting wrong? Or what in the world are the designers thinking here, killing the one and the only reason anyone would use RAW in the first place?

Further, it appears that it's not even possible to "upgrade" to full CinemaDNG. If you've imported CinemaDNG in Premiere Pro, the manual says that the clip will be locked to that tonal range, even as you bring it over into SpeedGrade, which I've confirmed.

I can't figure it out. If this is true, CinemaDNG playback is an completely pointless non-feature.

What the heck?

Per
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 04, 2013, 11:39:40 AM
sorry, didnt read the cow-thread - ok. thats bad. so, i'd take speedgrade as footageholder and import them via dynamiclink into premiere - just be patient with my opinions, i'll upgrade to CC-subscription end of this week.
Title: Re: The CinemaDNG Discussion
Post by: gary2013 on November 04, 2013, 11:46:17 AM
I am sorry. I forgot the link. check out all the posts in that thread.
http://forums.creativecow.net/readpost/3/947862
Title: Re: The CinemaDNG Discussion
Post by: gary2013 on November 04, 2013, 11:47:41 AM
so, i'd take speedgrade as footageholder and import them via dynamiclink into premiere
I will have to try that out. but, on the other hand, that can be a drag having a ton of footage sitting there in SG that does not make the final edit.seems backwards.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 04, 2013, 01:57:59 PM
actually i'm playing around with the dng-tags from a 5DIII-Frame. while irfanview shows massive changes (changing values for colormatrix, analogbalance,asShotNeutral and so on) and photoshop cant even import them (colormatrices doesnt map equal camera values to XYZ D50) -> Speedgrade isnt interested in these changes, it looks all the same. tried that with bmcc-files as well (colormatrix to 100 010 001) - nothings changing.

It seems, Speedgrade is reading only the raw/bayer-data while not been interested in correctiondata. i dont like the idea of dividing green sensels by 2 to get "working" pictures, because thus i destroy the philosophy of raw-data. so it must be done in spedgrade.

please some do me a favour, give me a dng-file with a magentacast (mine have clearly greencast, because of the two green sensels (double the value) and raw output without colormatrix.. the only working way would be to build a LUT. The DCP-Files (4750°K) from X-Rite doesnt work, but i found a Tool (http://dcptool.sourceforge.net/Installation.html) to convert them into xml, so here's the pastebin-link (http://pastebin.com/8JUa7Xs4). its not a lut, but a colormatrix.

btw. put the RGBtoXYZ-"Correction" on the Footage, (its the +Sign in the layer-window) does it help?

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: hookah on November 04, 2013, 02:40:15 PM
uhh will play with it later :-)

Download a single DNG frame with pinkcast here.
https://docs.google.com/file/d/0BxHiL_qPbYP0VURRcUdISWdoSDA/edit?usp=sharing
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 04, 2013, 03:30:57 PM
i just played a lot more :)

i wrote a colormatrix (as template/example) you just can put on the whole timeline. Take this XML-Data (http://pastebin.com/1uMLpJrV), save it into a .look-File (f.i 5DIII_Greencast.look). Save this File into

C:\Program Files\Adobe\Adobe SpeedGrade CC\settings\looks\CinemaDNG

now you can use it as a saved colormatrix as seen in this screenshot:
http://dl.phreekz.de/raw2cdng/5DIII_Greencast_Example.jpg

(!) Thats a first understanding of RGB2XYZ and this sort of problems. Shortly saying, its not impossible. I dont know if i'd solve it (when using SG) in the cdng itself.

regards chmee

@hookah
if i use this file in Speedgrade, its got a greencast - and its looking quite good with my template.
http://dl.phreekz.de/raw2cdng/Hookah_Example.jpg
Title: Re: The CinemaDNG Discussion
Post by: hookah on November 04, 2013, 05:14:45 PM
Okay

If i import direct into SG i also get a greenisy image
 - If i use your look-file it looks OK

If i import into premiere i get pinkish image - if i send it to SG via dynamiclink, its still pink.

Then i made a new look/lut with inspiration from yours (quick adjusted by hand - need some more scientific fine tuning) for use in premiere in its Lumetri effect
 - no roundtrip to SG, can play in full res, realtime inside premier.. SO COOL !!!

Here is my look/lut
http://pastebin.com/ituVygkk (http://pastebin.com/ituVygkk)

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fs8.postimg.org%2Fa00h26ghh%2Fpremiere.jpg&hash=9982e2ddef9a5040b0e2e1a08b478b07)
Title: Re: The CinemaDNG Discussion
Post by: hookah on November 04, 2013, 05:29:06 PM
delete.. double post
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 04, 2013, 05:42:53 PM
@hookah (good find)
Now its important to find out, if this premiere-lumetri-LUT is used before baking into 8bit - that means finally, you're able to highlight/shadow-recover and prevent banding issues, shortly benefiting the 14bit-bitdepth via effects/looks prepared in speedgrade. And then its not 100% fine, but it makes all the ae/converter-things obsolete.

mfg chmee
Title: Re: The CinemaDNG Discussion
Post by: mrnv45 on November 05, 2013, 02:11:18 AM
i cant seem to get a good upload.. youtube always makes my videos look like shit.  why?

Title: Re: The CinemaDNG Discussion
Post by: CinematicSyndicate on November 05, 2013, 07:34:41 AM
hookah & chmee --  Just ran some tests with this LUT this evening. It appears the LUT offsets the magenta cast somewhat, but is still a workaround as I'm getting some harsh banding and bizarre coloration. I see a greenish frame before my media loads, so it's acting like another intermediate color corrector which is another summing operation on the raw data, further destroying it. When I apply further image adjustment it just gets worse. This LUT is, however, useful as a temporary workaround - and chmee you pretty much nailed the color balance, whatever you used.

In terms of the differences between the BMCC CDNG and ML CDNG that EOSHD pointed out, is there a way to populate those fields with dummy or equivalent data to 'trick' premiere into seeing the correct color matrix?

This reminds me of what happens when I try to view 5D3 CR2 files in a lesser photo program like Picasa - EXACT SAME pink cast. Perhaps there's something there worth taking a look at?

I'm committed to helping you guys solve this! It will be a HUGE breakthrough and I'm convinced it will revolutionize the world of RAW.....dun dun dun
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 05, 2013, 09:06:07 AM
@CinematicSyndicate yesterday i tried an evening shot, and it behaves quite bad in the shadows. of course we/ i will investigate further, but as i wrote: speedgrade does not react on the correction-tags of the dng. the .look we used is a colormatrix outside the dng instead of being embedded inside the dng. did you played in premiere with lumetri-layer or in speedgrade?

i'll try to nail the color-matrix to D50 or D65 (http://en.wikipedia.org/wiki/CIE_Standard_Illuminant_D65). and it seems i have to combine the standard colormatrix (of the body) with the additional colormatrix of the colorchecker-measurements.. AND every Canon-Sensor behaves different. 5DIII-Colormatrices on 50D-Footage will not work well.

ah, and i tried with the hunter-lut (its a cube-file) - it made it worse inside speedgrade (but as a LUT-layer it pushes the colors positively on my testcharts)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: a1ex on November 05, 2013, 09:21:09 AM
Why not just subtract the black level from the raw data when generating the DNG? Since the video editor thinks it's always 0, you can try to make it 0.

So, set exif black level = 0, exif white level -= old_black, and for each raw pixel, p -= old_black.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 05, 2013, 10:46:26 AM
worth a try, but
(a) is the calculated blacklevel 100% accurate?
(b) i assume, this is not the solution for the double-valued green-sensel (greencast)
(c) these data are no more useable in other software.
(d) whitelevel isnt read by speedgrade as well.

in terms of color-calibration, matrices and measurements i'm clueless, so now its poking in the dark. it's time for a colorchecker-superman :)

-> Here's a 1.3.3.BETA(!) (http://dl.phreekz.de/raw2cdng/raw2cdng.1.3.3.BETA.zip) with Blacklevel to zero-Switch. (ahh. theres a bug.its working wrong!) patience.
-> 1.3.3.1.BETA (http://dl.phreekz.de/raw2cdng/raw2cdng.1.3.3.1.BETA.zip) (it seems, there are sensels darker than blacklevel-value, its all crazy :D)

-> Here are 2 dng's from yesterday evening (http://dl.phreekz.de/raw2cdng/phr_darkExample_a.zip). They behave quite good in Photoshop/ACR, but i cant get the flowers back in speedgrade, kind of lack of "exposure compensation"

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: a1ex on November 05, 2013, 12:12:41 PM
it seems, there are sensels darker than blacklevel-value, its all crazy :D

Ufraw simply clips them to black. The only disadvantage I can think of is that a noise reduction algorithm will no longer perform well in deep shadows, but if you expose correctly it shouldn't be a big problem.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 05, 2013, 12:24:42 PM
wanted to avoid if-routines inside the converter-loops. thats been my thoughts  ;D
Title: Re: The CinemaDNG Discussion
Post by: a1ex on November 05, 2013, 12:44:56 PM
BTW, I still get the same validation errors in your DNGs. I thought you already fixed these and still have problems.

I've explained why this error may trick a parser into using a wrong black level, so try solving this first.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 05, 2013, 01:18:49 PM
EDIT : Sorry, a1ex, found the problem. :facepalm: instead of writing the "long" 4 into the whitebalance-type, it wrote it into scale. changes a lot, because on that tag speedgrade reacts *huhh*

patience.. i have to go work for 3 days in another town, i'll be back i promise :)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: mrnv45 on November 05, 2013, 10:50:39 PM
Not a valid youtube URL [/youtube]

i must be exporting different.. cause when i play my master video.. and i play my premiere player... it looks so nice..

but once i get it on youtube it STILL looks like crap?

12mps render? an i doing anything wrong?
Title: Re: The CinemaDNG Discussion
Post by: CinematicSyndicate on November 06, 2013, 01:11:19 AM
@mnrv45 - you should post that to a different forum topic since this one is dedicated to the ongoing discussion about properly decoding CDNG files in editing applications. See the PM I sent for more help :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 08, 2013, 11:08:18 PM
raw2cdng 1.4.0.Beta online

http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

(!) It's experimental (!)
* by 99% stripped files won't work.
* optional reducing of blacklevel by 90%
* framerate fixed
* UniqueModelID now changeable
* multithreaded
* speedgradeCC-look inside
* readme inside

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: vikado on November 09, 2013, 01:43:07 AM
Not a valid youtube URL [/youtube]

i must be exporting different.. cause when i play my master video.. and i play my premiere player... it looks so nice..

but once i get it on youtube it STILL looks like crap?

12mps render? an i doing anything wrong?
h264 or prores?
raw2cdng 1.4.0.Beta online

http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

(!) It's experimental (!)
* by 99% stripped files won't work.
* optional reducing of blacklevel by 90%
* framerate fixed
* UniqueModelID now changeable
* multithreaded
* speedgradeCC-look inside
* readme inside

regards chmee
awesome work chmee, i'll do some tests over the weekend
Title: Re: The CinemaDNG Discussion
Post by: Andy600 on November 09, 2013, 10:32:50 PM
I tried setting the Unique Model to 50D and it keeps giving me 50Dchmee :D . Other than that it seems to be working ok (Win7 64bit). ACR needs Camera Model Name for DCP to work correctly. It's currently set as Canon EOS PHR.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 09, 2013, 11:45:01 PM
:) lol. ok.

So i asked in another thread. what are the important nametags for ACR? UniqueCameraModel (0xC614) or Model (0x0110). Both maybe?

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: Andy600 on November 10, 2013, 12:27:13 AM
:) lol. ok.

So i asked in another thread. what are the important nametags for ACR? UniqueCameraModel (0xC614) or Model (0x0110). Both maybe?

regards chmee

For ACR 'model' is the important one. As raw_rec is hardcoded for daylight Can you maybe add WB as a selectable field? I usually pop off a still or 2 before hitting record so manually choosing ISO/WB etc would be very useful as you can refer to the still for exposure info.

BTW, 12bit DNGs look way better in Resolve than 14bit DNG. Thanks for keeping this going Chmee! :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 10, 2013, 12:40:22 AM
For ACR 'model' is the important one. As raw_rec is hardcoded for daylight Can you maybe add WB as a selectable field? I usually pop off a still or 2 before hitting record so manually choosing ISO/WB etc would be very useful as you can refer to the still for exposure info.
it's already built in since 1.3.0 - just rename the cr2 into the same as your videofootage, it will be recognised and "measured!" wb will be taken. look into my blog description on 1.3.0 - http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
Title: Re: The CinemaDNG Discussion
Post by: Andy600 on November 10, 2013, 12:48:49 AM
How did I miss that gem?  :o
Title: Re: The CinemaDNG Discussion
Post by: timbytheriver on November 10, 2013, 12:15:42 PM
raw2cdng 1.4.0.Beta online

http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

Hi Chmee

Sorry to be a dunce, but I don't know what I should put in the 'Unique Model' field. Also, when I try to drag a Raw file in, the app does nothing (all paths are set etc.). All previous versions of your great app have worked perfectly for me! Do I need to uninstall them first maybe?

Any idea what I'm doing wrong?

Many thanks

Win 7
5DMk2
Speedgrade / Premiere Pro
Title: Re: The CinemaDNG Discussion
Post by: vaisse on November 10, 2013, 12:52:33 PM
Hi Chmee
thanks for your work

the version 1.4 o cdng crash when i start converting i have no problem with version 1.3
windows 7 64 bit

i hope you can help me
thanks
Title: Re: The CinemaDNG Discussion
Post by: vaisse on November 10, 2013, 01:54:21 PM
ok it 's ok now i change the name of model
but i can put canon eos 6d but canon eos 5d mark iii is to long

thanks
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 10, 2013, 09:01:02 PM
@timbytheriver hmm. if you drag it does not appear in the list? does it appear but the values are strange?

vaisse thanks. i will lengthen the modelname to ~24 chars. and the modelname inputform is not very inutitive i think, will change that as well.
Title: Re: The CinemaDNG Discussion
Post by: timbytheriver on November 10, 2013, 09:10:51 PM
@Chmee

Thanks for answering. It does appear in the list. But then when I press the button, the list entry turns red but doesn't process. The dng folder is created – but no files inside. :o

Also I'm really not sure what to put in the 'Unique model' box! Do you have an example I should use for my 5dMk2 please?

Thanks again for your work! :)

Title: Re: The CinemaDNG Discussion
Post by: CinematicSyndicate on November 13, 2013, 10:38:37 PM
1.40 crashes for me when I press convert...I'm on windows 7 x64.

Also - where are we in the progress of getting Premiere to natively see cDNG without the magenta/pink hue?
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 13, 2013, 11:03:11 PM
important informations for a reasonably answer:

(*) the metadata are read accurate? are resolution, frames, framerate, body accurate?
(*) maybe it's an wrong converted mlv?
(*) firmware/ml-version from when?
(*) stripped or not?

(todo for me: clickable debug-output for crash-situations)

magenta/pink cast are showing, when the whitebalance is set wrong, but not on the line of warm/cold whites, but in right angle to it (looking on a colorgraph). Thats why there are two sliders for adjusting wb.

(*) did you tested the cr2-wb-embedder?
(*) are these problems as well with ACR/Speedgrade/Resolve?

(!) havin' a lot of work - tomorrow in the german tv is the bambi. means, after that i hope i can return to patch and update to a better/usable version. btw i test and code this tool under win7/x64/c#2010 - but this time i wrote a readme because it was fast coding without heavy testing.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: timbytheriver on November 14, 2013, 11:34:52 AM
Hi Chmee

Here's some of my debugging info if it's useful:

* Metadata for White Level (16383) I think is wrongly reported. Should be 15000 (?) All other is correct (frame rate, body etc.)
* I use ML build from 10.9.2013
* Not using mlv.
* I don't know what 'stripped' means!  :-[

Screen grab of my Win 7 (It's running on Virtual Box on Mac OSX 10.7.5. All previous versions of your app have worked perfectly on this same setup!)

https://www.dropbox.com/s/96nu51crl0d3sbo/grab.jpg

Thanks again for your help.
Title: Re: The CinemaDNG Discussion
Post by: mityazabuben on November 16, 2013, 06:32:58 PM
Compare raw2cdng v.1.3.0 and raw2dng with 7D footage. Here is a result: https://drive.google.com/folderview?id=0B2NWwDUimG9cbS1mWFRtaXgzSFU&usp=sharing (https://drive.google.com/folderview?id=0B2NWwDUimG9cbS1mWFRtaXgzSFU&usp=sharing)
Left part is a raw2dng, right is a raw2Cdng. Import to Photoshop via ACR as is - nothing was changed. On a raw2cdng dng file clearly one can see color moire on a roofs (blue + yellow). What am I doing wrong?
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 24, 2013, 10:14:33 AM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

raw2cdng v1.4.5 is online
* fixed modeltag (should work now with acr-profiles)
* multithreading seems workable with stripped files
* whitelevel back to 15.000
* mlv-workaround (read in blog-description)

@mityazabuben
maybe because of profiles. in the raw2dng-picture you can see the yellow/blue artifacts as well, but less harsh
Title: Re: The CinemaDNG Discussion
Post by: timbytheriver on November 24, 2013, 02:00:13 PM
Thanks chmee!

Unfortunately, v1.4.5 is still not processing any file and/or crashing for me :( Both on Win 7 and XP. If I try running as 'Admin' on Win7, the drag and drop does not work!

I'm running on Virtual Box on Mac OSX10.7.5 – but as I mentioned before all previous/early versions of your app have worked perfectly on it!

Any ideas please chmee / anyone?

Many thanks

Title: Re: The CinemaDNG Discussion
Post by: chmee on November 24, 2013, 02:04:27 PM
is it showing any crash-error? with 1.4.0 multithreading came into source, i suppose, there's the problem.

drag&drop-problem is known - search for that.
f.i. http://answers.microsoft.com/en-us/windows/forum/windows_7-performance/drag-and-drop-not-working-in-windows-7/f6dcfa11-8c51-e011-8dfc-68b599b31bf5

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: timbytheriver on November 24, 2013, 02:06:57 PM
Thanks for the reply! :)

Win XP is showing the 'Windows has encountered a problem and has to close' box

Win7 just fails to process the files (an empty folder is created, but no green progress bars) when I press the 'convert' button

Title: Re: The CinemaDNG Discussion
Post by: timbytheriver on November 24, 2013, 02:31:55 PM
@chmee Thanks for the link. The drag and drop fix mentioned there didn't work for me.

This time I ran the app I got a crash report. Does it say anything to you?

https://www.dropbox.com/s/sf70khxpv9qgb06/grab2.jpg

Thanks again.

Title: Re: The CinemaDNG Discussion
Post by: chmee on November 24, 2013, 03:52:52 PM
next time, try to copy/paste the whole data inside the window. btw i'm writing the debug-Option.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: timbytheriver on November 24, 2013, 04:06:00 PM
Sorry! Stoopid of me. :o Here it is:

Problem signature:
  Problem Event Name:   CLR20r3
  Problem Signature 01:   raw2cdng.1.4.5.exe
  Problem Signature 02:   1.0.0.0
  Problem Signature 03:   52911779
  Problem Signature 04:   raw2cdng
  Problem Signature 05:   1.0.0.0
  Problem Signature 06:   52911779
  Problem Signature 07:   5c
  Problem Signature 08:   4ba
  Problem Signature 09:   System.NullReferenceException
  OS Version:   6.1.7601.2.1.0.256.1
  Locale ID:   2057
  Additional Information 1:   0a9e
  Additional Information 2:   0a9e372d3b4ad19135b953a78882e789
  Additional Information 3:   0a9e
  Additional Information 4:   0a9e372d3b4ad19135b953a78882e789
Title: Re: The CinemaDNG Discussion
Post by: painya on November 24, 2013, 06:47:13 PM
1.40 crashes for me when I press convert...I'm on windows 7 x64.

Also - where are we in the progress of getting Premiere to natively see cDNG without the magenta/pink hue?
1+
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 24, 2013, 06:49:52 PM
so thanks to timbytheriver we found a bug.

Dont switch the ModelID-Field on, if there's nothing in it.

if you switch on, fill in something.
OR
let it switched off, if you dont need it.

maybe you have to restart the application. patch on the way.

Title: Re: The CinemaDNG Discussion
Post by: kgv5 on November 24, 2013, 07:31:34 PM
After pressing "convert" program crashes. Win7 64bit. Tried all boxes checked/unchecked.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 24, 2013, 09:29:16 PM
v1.4.6 online
* modelID-switch-bug fixed
* switchable debuglog

http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on November 25, 2013, 12:15:44 AM
I got strange issue new raw2cdng 1.46, when you load 9 split files, see wrong info of clips
See images below.
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Ffarm6.staticflickr.com%2F5525%2F11037971356_8158f0dc3e_o.png&hash=349c6d93b5228d4f26529292cd73bc84) (http://www.flickr.com/photos/96806538@N08/11037971356/)
raw2cdng_1 (http://www.flickr.com/photos/96806538@N08/11037971356/) by davidmiazga1 (http://www.flickr.com/people/96806538@N08/), on Flickr

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Ffarm8.staticflickr.com%2F7339%2F11037880455_698bef5d87_o.png&hash=8d4e3b7da9fb50d4de5c26e24505fd96) (http://www.flickr.com/photos/96806538@N08/11037880455/)
raw2Cdng (http://www.flickr.com/photos/96806538@N08/11037880455/) by davidmiazga1 (http://www.flickr.com/people/96806538@N08/), on Flickr

Title: Re: The CinemaDNG Discussion
Post by: chmee on November 25, 2013, 12:29:07 AM
:) ok.lol. 9 Split-Files. Just give me a day to record this kind of file as well, and i will find the bug :) holy..

btw. is it working accordingly in 1.3.0?
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on November 25, 2013, 12:32:07 AM
But when you load after join manually 42GB total works ok  :D
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on November 25, 2013, 12:39:49 AM
:) ok.lol. 9 Split-Files. Just give me a day to record this kind of file as well, and i will find the bug :) holy..

btw. is it working accordingly in 1.3.0?
I don't know , i will try right now

Edit: Yes, 1.3.0 has the same issue.
But has no problem converting 42GB 2k frame size to cdng  :)
Title: Re: The CinemaDNG Discussion
Post by: painya on November 25, 2013, 02:59:56 AM
@chmee
Love the Lut you made to help combat the problem (Seen on page 13). I think it would be really helpful to put that on the First post  for people looking.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 25, 2013, 09:10:42 AM
@reddeercity You're using the 1.4.6 with debug-output. please copy/paste the debug-log into a pastebin.
Title: Re: The CinemaDNG Discussion
Post by: g3gg0 on November 25, 2013, 06:12:36 PM
@reddeercity You're using the 1.4.6 with debug-output. please copy/paste the debug-log into a pastebin.

hint: place a try-catch block around every thread function and event handlers.
and open a messagebox there displaying the stack trace.
thats the most simple and effective method i usually chose.
Title: Re: The CinemaDNG Discussion
Post by: dmilligan on November 25, 2013, 07:11:21 PM
actually, much simpler is to add a handler for AppDomain.CurrentDomain.UnhandledException during initialization and log the error from there.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 25, 2013, 07:27:11 PM
did you looked into the debug-output? to catch only exception is one thing - to see via outputlogs, what variables has been set and where it stops was my concern. i can read that.
Title: Re: The CinemaDNG Discussion
Post by: g3gg0 on November 25, 2013, 10:02:59 PM
actually, much simpler is to add a handler for AppDomain.CurrentDomain.UnhandledException during initialization and log the error from there.

had certain problems with forms not honoring that well.
this works for you?
Title: Re: The CinemaDNG Discussion
Post by: dmilligan on November 25, 2013, 10:32:29 PM
All of my projects are WPF, and I've never had it not catch anything. I actually capture most exceptions one level higher from the thread's dispatcher (a few slip through the cracks and reach the AppDomain handler though, at that point, the app is probably going to have to terminate, catching at the thread dispatcher and you can mark it 'handled' and keep going). http://msdn.microsoft.com/en-us/library/system.windows.application.dispatcherunhandledexception(v=vs.110).aspx

Creating handlers for these two events is the first thing I do when I start writing a new program.

Any particular reason you are still using forms?
Title: Re: The CinemaDNG Discussion
Post by: g3gg0 on November 25, 2013, 10:49:12 PM
thanks for that hint :)

> Any particular reason you are still using forms?
it's because it is "simple" and most likely being handled well by mono.
plus maybe because my visual studio professional license is quite old and i have the fear that WPF in .net 3.x is not good enough.

and the most important of all: my gut feeling says no :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 25, 2013, 10:54:45 PM
@dmilligan
i simply didnt started to "convert myself" into wpf. so, i didnt thought, it could be a drawback in this project.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: dmilligan on November 25, 2013, 11:05:59 PM
You're welcome ;)

WPF in 3.5 is actually fairly decent (main thing missing is DataGrid). MVVM is incredibly powerful paradigm once you get to know it. I use the WAF (WPF Application Framework), it's extremely modular.

I'd love to help with some C# stuff since that is actually my area of expertise, but I don't have access to a Windows computer and Visual Studio outside of work.
Title: Re: The CinemaDNG Discussion
Post by: g3gg0 on November 26, 2013, 01:46:58 AM
I'd love to help with some C# stuff since that is actually my area of expertise, but I don't have access to a Windows computer and Visual Studio outside of work.

you are welcome to add a monodelop project file or whatever you prefer ;)
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on November 26, 2013, 03:34:45 AM
@reddeercity You're using the 1.4.6 with debug-output. please copy/paste the debug-log into a pastebin.

I'm sure howto use & copy in to a "pastebin"

Here the log info by my Drop Box

There was more 8 spitting file actually 11 plus the .raw make that 12.
but only reported 8. run it then program crashed
https://www.dropbox.com/s/0j0i7gsofz16guk/raw2cdng_multi_file_crash_debug.log

this was after i join manually total of 46GB
https://www.dropbox.com/s/npe51sf3vz8kzcc/raw2cdng_single-46GB-file_debug.log


Title: Re: The CinemaDNG Discussion
Post by: painya on November 26, 2013, 05:03:34 AM
How long would it take to incorporate an option to create a proxy video? That would be awesome, and maybe you and the developer of rawanizer could exchange code.
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 26, 2013, 08:07:05 AM
@reddeercity
thanks. i think i know where the problem is. patch in the evening.

@painya
jpg-filesequence is prepared. one of the next patches.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: Fringuello on November 26, 2013, 03:57:56 PM
Where exhaust look-files for the lumetri-effect?
Title: Re: The CinemaDNG Discussion
Post by: chmee on November 27, 2013, 12:54:18 AM
as i now tested, the .look file i made won't work in premiere. so i read about the iridas ilut-scripting and compatible formats.
http://doc.iridas.com/index.php?title=LUT_Formats

put the code into a textfile, name it "NameItAsYouWant.cube" (ending .cube is important). its a very simple 3d-lut and i just played around for 15-20 minutes, its far from perfect, but a first insight into the lumetri-thing.
Code: [Select]
TITLE "5DIII N"

LUT_3D_SIZE 2

DOMAIN_MIN 0.315 0.0 0.158
DOMAIN_MAX 0.47 0.49 0.52

-0.2 -0.2 -0.2
1.2 -0.2 -0.2
-0.2 1.2 -0.2
1.2 1.2 -0.2
-0.2 -0.2 1.2
1.2 -0.2 1.2
-0.2 1.2 1.2
1.2 1.2 1.2
use it with the lumetri effect and push with contrasts-effects. justify the black- and whitelevels with the Domain- values for r g and b. in small steps please :) regards chmee
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 02, 2013, 09:57:24 PM
the good news:
Yeah. It works. Without some strange LUTs and Precomposings. Straight inside Premiere. Files do well in Resolve, Photoshop/ACR and Speedgrade too.

the bad news:
Not online today ;) some small bugs inside.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 04, 2013, 12:37:23 AM
raw2cdng 1.4.8 is online
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fraw2cdng.1.4.8.gif&hash=58b8e027a23761cc30715518408f5d83)
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on December 04, 2013, 04:48:50 AM
Hi Chmee  :)
I take the new Cdng for a test drive and thing are much better.
I have only done  3 test to date,
First, i still can't Link out of Premiere Pro to Speed Grade, it just Crashes.
But if you import the Cdng in to Speed Grade that works,
So far the setting that works for me (i will need more testing) is
Party mode on Gamma 1.0, at that you still need to do a fair bit of adjustment.
It take a little time to remove the some Pink cast at the high end & some Green cast at the Low end.
then i find if you use the speed grade gamma 2.2 ilut it put it in line .
Found that under the timeline options "setup" in the Calibration box .


So are still Getting the Full 16bit, CDNG or is it a Proxy file that adobe is using ?
Because it still look very flat and dose not have the same high fidelity image that i see in AE or FCPX.
 :)
 


 
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 04, 2013, 07:01:44 AM
i thought, coding a gamma-corrector/curve would optimize the flat-amount, but it it does not. it turns into red'ish or blue/greenish cast. seems, that the cdng-alogorithm of premiere (*A) has an "optimized" to somewhat-blackmagic-log-curve and (*B) it crops the 8bit to video-range. i will start a cdng-discussion in the adobe forums - its not clear for me, what they have coded..

gamma 2.2 - thanks for finding.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on December 04, 2013, 07:38:04 AM
i thought, coding a gamma-corrector/curve would optimize the flat-amount, but it it does not. it turnes into red'is or blue/greenish cast. seems, that the cdng-alogorithm of premiere (*A) has an "optimized" to somewhat-blackmagic-log-curve and (*B) it crops the 8bit to video-range. i will start an cdng-discussion in the adobe forums - its not clear for me, what they have coded..

gamma 2.2 - thanks for finding.

regards chmee
Glad I could Help
Title: Re: The CinemaDNG Discussion
Post by: mva on December 04, 2013, 08:52:09 AM
Hi, I'm trying to get started with raw2cdng, and can't get past the first step. I downloaded 1.4.6, extracted it to my desktop, double-clicked, and immediately get the message "raw2cdng has stopped working. Windows can check online... etc." I downloaded 1.3 and the same thing happened. I also tried dragging a raw file onto it like with raw2dng and got the same result. I must be making some dumb mistake. Can someone please tell me what I'm doing wrong?
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 04, 2013, 12:37:09 PM
@mva
(1) Do you have .net4.0 installed?
(2) what OS? Windows-core or emulation/virtual system?
(as every time: more information leads to more accurate answer)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: mva on December 04, 2013, 12:45:05 PM
@mva
(1) Do you have .net4.0 installed?
(2) what OS? Windows-core or emulation/virtual system?
(as every time: more information leads to more accurate answer)

regards chmee
Hi chmee,
(1) I have "Microsoft .NET Framework 4 Client Profile" installed
(2) Windows 7 64 bit (core, I suppose, since it's not running on another OS)
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 04, 2013, 12:56:26 PM
my tool is writing/reading settings-data inside the users/appdata-folder. Did you tried running the tool as administrator? (right-click)
Title: Re: The CinemaDNG Discussion
Post by: mva on December 04, 2013, 01:33:20 PM
Yes, I had tried running as an administrator. No difference.

Not sure this will help, but these are "problem details" I can view when I get the error message:

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fimg837.imageshack.us%2Fimg837%2F6389%2F6a54.jpg&hash=fd0a4f9a4cf31df4cfd44470cca644b1)

Well, chmee, thanks anyway for trying. Looks like there may be something quirky going on with my computer.

Title: Re: The CinemaDNG Discussion
Post by: chmee on December 04, 2013, 05:15:53 PM
(A) please look into this folder:
C:\Users\[Username]\AppData\Roaming

There should be a file named
raw2cdng.settings.json

if not, create an empty one. (Thats the only file that "couldnt be found" on app-start.)

(B) if this file exists, look if there's an entry named
debugLogEnabled
Code: [Select]
{"outputPath":"C:\\","uniqueName":"Canon EOS 5D Mark III","debugLogEnabled":true,"debugLogFile":"C:\\Users\\chmee\\Documents\\Visual Studio 2010\\Projects\\raw2cdng\\raw2cdng\\bin\\Debug\\raw2cdng_debug.log","gamma":1,"maximizeEnabled":true}(this is my example settings-file here on this laptop. Try to put this data into the file. Change the debugLogFile value into a useful path :)

(C) You extracted the path from the zip? really? :)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: mva on December 04, 2013, 07:06:38 PM
(A) raw2cdng.settings.json wasn't in the folder you asked me to check. Unfortunately, however, I don't know how to create an empty file. :( (I did some Googling but found too many different answers mixed with too much uncertainty.) Can you tell me how to do that? Is it done from a command prompt?

(C) Since I sometimes just drag files out of WinRAR without properly extracting them, I tried again making sure I extracted it properly, and had the same problem.

EDIT: chmee, I saw your post following this one (#397), and I'll reply soon. Want to do some reading and testing, and I'm a bit busy now to do it. Thanks!
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 04, 2013, 10:32:06 PM
Open the folder, right click and use -> "new" -> textdocument. i hope, you have switched off hiding extensions.
-> http://www.sevenforums.com/tutorials/10570-file-extensions-hide-show.html

It seems, my app does not have the permission to write this file.
http://support.microsoft.com/kb/308419

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: mva on December 05, 2013, 10:14:36 PM
chmee, I really appreciate your help, but I think I'm going to give up for now. Creating the empty file as you suggested didn't work. I tried the app on two other computers running Windows 7 64-bit in my house, and weirdly it immediately crashed each time on those too, even when running it as an administrator. I haven't had this problem with any other program/app/tool so there's something quirky going on. I Googled "System.IO.FileNotFoundException crash" and found lots of descriptions of problems similar to what I'm experiencing, with people being advised to try all sorts of things which don't work (including reinstalling .net 4.0), but I couldn't find anything I wanted to try, partly because I don't want to spend more time on this now. So I think I'll just periodically retry the app (e.g., after Windows updates) in hope that the problem has gone away, but for the time being I guess I'll be sticking entirely with DNGs. Thanks again.
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 05, 2013, 10:39:41 PM
@mva
can i ask you, if you're from a country with "abnormal" characters or did you try to change the location of the app? root folder of a harddisk for example? but non the less, for me as well an interesting thing.

Beyond that, the adobe-Forum is quite strange, on "our" part of dng-discussion they only've got the thinking, no, it seems, its not planned to open the native dng-support to other files than bmcc.. where the heck is the idea of the "widely-universal-photographic/filmdata-format"? DNG is not.

http://forums.adobe.com/thread/1326153?tstart=0

edit: i think, this is the main answer:
Quote
CinemaDNG is a huge spec, we targeted getting the colours correct from the BlackMagic camera first.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: mva on December 05, 2013, 10:55:54 PM
chmee, yes, although I do pretty much everything in English on my computer(s), all are enabled for Hebrew characters as well, I have Microsoft Office Hebrew language packs installed in them, etc. Could this be a factor?

Yes, I did try running the app from the root directory of a different drive, but without success.

Title: Re: The CinemaDNG Discussion
Post by: chmee on December 05, 2013, 11:30:05 PM
my thoughts went to unicode/ascii (chracterencoding) and something like that. maybe it cant read/write/find the proper file because of differences in right that. wrong character-encoding leads to "FileNotFound". i didnt had such an issue, so finally cant help by now, but maybe "our" worldwideweb can.. have to search and read about it.

regards chmee

Title: Re: The CinemaDNG Discussion
Post by: mva on December 06, 2013, 12:19:55 PM
my thoughts went to unicode/ascii (chracterencoding) and something like that. maybe it cant read/write/find the proper file because of differences in right that. wrong character-encoding leads to "FileNotFound". i didnt had such an issue, so finally cant help by now, but maybe "our" worldwideweb can.. have to search and read about it.

regards chmee

FYI, I updated to .NET 4.5.1 and now raw2cdng works!  :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 06, 2013, 09:50:45 PM
lol. really.. i have to check, what code-excerpt uses .net 4.5 (i'm kind of pissed, VisualStudio doesnt show up those things).
Thank you.
Title: Re: The CinemaDNG Discussion
Post by: jordancolburn on December 07, 2013, 05:15:07 PM
Thanks for this awesome tool.  I've been doing a couple of test shots just to get the workflow down.
I'm currently going: EOS-M(crop mode) -> focus pixel remover -> raw2cdng -> Premiere(editing) -> speedgrade (grading)

The latest raw2cdng w/ partymode is really great, and seems to have more lattitude in speedgrade vs h264, but I have noticed an issue where clipped values in raw show as magenta in speedgrade and premiere.  Any help or tips would be appreciated, I've tried going through this whole topic, but maybe I missed something along the way.  Thanks!
Title: Re: The CinemaDNG Discussion
Post by: timbytheriver on December 07, 2013, 05:59:25 PM
@chmee Thank you for v1.4.8! I have much better results now in 'Partymode' and Gamma 1.0 getting rid of the pink going into PPro 7.1

Something else I have noticed is this chromatic aberration (?) in Cdng. I don't see it in straight dng. Two 100% crops from the same .raw file to illustrate here (You can see it clearly on the ripples):

https://dl.dropboxusercontent.com/u/21590714/raw2Cdng.jpg

https://dl.dropboxusercontent.com/u/21590714/raw2dng.jpg

Where is this coming from?

If this is old ground already covered / fixed, I do apologise! :)

rgds

PS Both the files here were debayered in ACR from the original .raws and then cropped.

Title: Re: The CinemaDNG Discussion
Post by: chmee on December 07, 2013, 11:18:59 PM
@jordancolburn
appreciate that :) canon raw have different black and whitelevel for the components (r, g and b) it seems. if you use the partymode, all values lower blacklevel (~1023 or ~2047) will be cutted and all values higher than 15000 will be cutted as well. So, yes, it seems, there is an under- or overflow of values (i already have switches for that, but your picture shows theres another thing i missed)

@timbytheriver
you're welcome. one or two sites ago, there was an example with the same problem. it seems (in your picture as well) there is CA in dng as well, but much weaker. by now i dont have any clue to avoid that. did you try to set a according modelID? Maybe ACR is reacting well with the right Body ID?

so, sorry i cant help by now. but the tool isn't finished yet (g3gg0 just offered his help to some questions) and i hope i can throw out a nutty version till christmas :)
Title: Re: The CinemaDNG Discussion
Post by: timbytheriver on December 08, 2013, 12:33:28 PM
@chmee Good news about the xmas version! Can it have chocolate, brandy and cream included? :P

So model ID/no model ID seemed to make no difference to the chromatic aberration issue. I did notice something odd though: Mac OSX finder window reports all my dngs from raw2cdng as being dimensions: 1864 x 1042, whilst all other tools (RawMagic, raw2dng, etc.) report them as 1876 x 1054 (I shot at 1880 x 1058 ad build 4.10.13)

Odd! :o Could this be a clue?

Thanks again.

rgds
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 08, 2013, 01:10:16 PM
but the raw-infos inside raw2cdng are accurate? in dng there s an attribute for padding - i set it to zero. will look on that. thanks.
Title: Re: The CinemaDNG Discussion
Post by: timbytheriver on December 08, 2013, 04:51:39 PM
@chmee raw2cdng gui reports the dimensions as 1880 x 1058
Title: Re: The CinemaDNG Discussion
Post by: Malex on December 08, 2013, 08:16:46 PM
Hi, I have just tried 1.4.8 and great! but...
I did have few bugs, I guess it has to do with speedgrade not being stable enough with dng files ? I had this before too. When I use the direct link from premiere to speedgrade, it opens the files fine, OR it opens it with some greenish look... it's random, and I guess it has nothing to do with raw2cdng. It looks good in premiere, even though I think the white balance is off a little bit.

Great job chmee!
Title: Re: The CinemaDNG Discussion
Post by: jordancolburn on December 09, 2013, 02:46:18 AM
@jordancolburn
appreciate that :) canon raw have different black and whitelevel for the components (r, g and b) it seems. if you use the partymode, all values lower blacklevel (~1023 or ~2047) will be cutted and all values higher than 15000 will be cutted as well. So, yes, it seems, there is an under- or overflow of values (i already have switches for that, but your picture shows theres another thing i missed)
Thanks for the reply.  I'll see if I can recreate it with some more details, but for now here is an example image:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fjordancolburn.com%2Fwp-content%2Fuploads%2F2013%2F12%2FSequence-01.Still003-800x450.jpg&hash=bf68761fab2fabe8a6d37c3a049120bb)

I was trying to do a tough test vs h264 and tried to shoot inside with the lights off and a bright snowy outside, so to get the inside exposed right, the outside was very overexposed, but the highlights came back ok in SG (except the magenta area).
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 09, 2013, 05:03:33 PM
@jordancolburn

i'm searching/reading for this red/pink highlight thing. It seems, the whitelevel is higher set than needed. in RT there is a bugreport right about that, but they mention lowering the WL (after debiasing?). i'll try some of the things :)

links:
magenta highlights - https://code.google.com/p/rawtherapee/issues/detail?id=1695
WB-subtraction and RGB to XYZ - http://lclevy.free.fr/cr2/#wb

regards chmee
 
Title: Re: The CinemaDNG Discussion
Post by: a1ex on December 09, 2013, 05:11:13 PM
@jordancolburn: may I see your DNG?

(it's offtopic, I'm just experimenting with some highlight recovery algorithms)
Title: Re: The CinemaDNG Discussion
Post by: jordancolburn on December 09, 2013, 06:48:25 PM
@jordancolburn: may I see your DNG?

(it's offtopic, I'm just experimenting with some highlight recovery algorithms)
Sure, when I get home today, I'll grab the DNG frame send it to you.
Title: Re: The CinemaDNG Discussion
Post by: jordancolburn on December 09, 2013, 06:52:31 PM
i'm searching/reading for this red/pink highlight thing. It seems, the whitelevel is higher set than needed. in RT there is a bugreport right about that, but they mention lowering the WL (after debiasing?). i'll try some of the things :)
Thanks for looking into this!  Out side of that bug, raw2cdng really was the final part of the workflow I needed to get into editing and grading.  Now I can actually do a real-ish world test comparing h264 and raw with the eos-m.
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 18, 2013, 10:44:28 AM
https://infogr.am/bmcc-delinearization-curve?src=web
(some brainfood) (now both curves lin>log and log>lin inside)
full csv-data for log>lin on http://dl.phreekz.de/raw2cdng/bmccLog.csv

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: IliasG on December 20, 2013, 06:41:36 PM
https://infogr.am/bmcc-delinearization-curve?src=web
(some brainfood) (now both curves lin>log and log>lin inside)
full csv-data for log>lin on http://dl.phreekz.de/raw2cdng/bmccLog.csv

regards chmee

Chmee hi,

Do you mean that you (will) use Bmcc curves ?. At which bitdepth ?. If you need something just ask. I think the old gamma curve I provided needs a bit refinement. Of course it was proposed for 14 bit linear to 10 bit log so we will need something different if our intention is to use gamma encoding for 14lin to 12log or 16 to 12. 

Regarding magenta highlights, because the clipping White level for each model depends on ISO and apperture,  in latest RT there is now an external file (camconst.json)  where we can set per ISO White Level and a multiplier depending on apperture. I think a database is the only way to be safe while not loose highlights.

Or you can use exif (maker data) info in CR2 files, latest exiftool decodes more Canon tags  and White Levels are among them although they are a bit conservative sometimes ...

Currently, for RT, there is a need of Canon raw samples with at least a "burned" area shot with intermediate ISOs (125, 160, 250, 320 ...) to fill the database ..
5D2 and 5D3 are the only models whith complete data, all others (and 6D) are missing.
Title: Re: The CinemaDNG Discussion
Post by: glubber on December 20, 2013, 09:20:53 PM
Interesting.... i was stumbling too upon the camconst.json and the researchs of the RT community just today.
But in purpose of finding a solution for the pink highlights in dcraw/UFRAW.
=> http://www.magiclantern.fm/forum/index.php?topic=5197 (http://www.magiclantern.fm/forum/index.php?topic=5197)

As i don't have PremierePro, i wasn't aware that the pink cast problem could have the same origins as the dcraw case.
Maybe the new raw engine of Premiere is basing upon dcraw?  ;D

Edit: I just remembered seeing your name on the RT issue board.  :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 20, 2013, 10:51:17 PM
i made a second logarithmic convert-mode (12bit with bmcc gamma) - for the shadows and the whole picture grace it does a good job, but it cant eliminate the red tint, because of the different component blacklevel and amplitude - it's time to go deeper - something i wouldnt do, because i recalculate the raw-data more i supposed to. it's a real shame, that the dng-importer in premiere doesnt have a colormatrix-option or reads the colormatrix/cameracalibration-data accordingly. omg.

@IliasG just send me a binary file with 16384 int (2byte) entries (14bit->10bit) 16bit->10bit.

@glubber my thoughts, its not really about a red-overflow. if its magenta (red-blue'ish), the green-channel isnt accordingly read, if its green, it was not taking into account, there are two green sensels..

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: IliasG on December 20, 2013, 11:41:14 PM
Binary ??. I don't know how .. a simple text file with input and output columns is the best I can ..

I'll have to reopen and remember what the refinement is .. hopefully tomorrow

Did you managed to insert the linearization table in DNG exif data  ?.
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 21, 2013, 06:14:08 PM
@iliasG
yeah. do that. i'll convert it here via php. i didnt inserted the linearizationtable by now, because i assume, premiere does not take account of it. For Speedgrade and Resolve it has to exist, so yes, i'll do that today.

edit(!) IliasG, forgot to say: dont cope with the uneven black- (1023/2047) and whitelevel (~15000). take 0 as blacklevel and 65535 as whitelevel for the gammacurve. internally i do now all from 16bit to destination-bitdepth.
Title: Back to Resolve?
Post by: rommex on December 25, 2013, 03:05:30 PM
Hello Chmee,

I've been long a fan of your application, but only recently discovered this thread (stupid me).
I read it wholly before I was going to make some points.
First I want to thank you for your efforts and readiness to help.

I'll try to be brief in my points:

1. DaVinci Resolve. As I got from this forum, there was a trend to move to Adobe from Resolve due to a better de-bayering. It was true with Resolve 9 (I hated it) but Resolve 10 for me show far FAR better results than Adobe ACR. I shoot with 50D, and artifacts from Adobe are huge compared to Resolve 10.

Things move quickly, and maybe somebody should give Resolve 10 Lite a try.

2. Roundtrip. So I use R2C for Resolve and Premiere roundtrip. I read the whole discussion about naming and XML problems and while reading really had a desire to jump back into the past ))

2.1. FPS. Now, I am shooting 25 FPS, but when imported to Resolve, all clips are 24FPS. Have to change it manually. When I forget, I have a de-sycnh with audio, and have to re-render. But this is the beginning for surprises ))

2.2. Folder naming. So I have 2 clips from shooting raw video:

1.M24-1803.RAW +2 spanning files
2.M24-1806.RAW +2 spanning files

They were shot in consequence, one after another at 18:03 and 18:06 yesterday, 24 Dec.

After conversion (made today on 25 Dec at about 15:00) I have 2 folders:

131225_1235_M241803K
131224_1235_M241806N

Why 131225_ and 131224_ ? Why _1235_ and _1235_?
All this makes basically impossible to sort these folders by name.

2.3. Resolve clip naming.
When imported into Resolve, these clips show as:

M241803K.[000000-003194].dng
M241806N.[000000-004983].dng

That is a step back -- I can sort them now, which is good. But after grading when I render my dailies with "Use source filenames" checked, I have files named:

M241803K.mov
M241806N.mov

2.4. XML issue. After editing these files in Premiere, I export XML file for Resolve. And of course I have that problem where only the first clip shows ok, and others have error. Yes now I know about the solution with reelname suggested by a French-style contributor.

But I also solved this problem in my way: I searched through XML file and changed:

M241803K.mov to M241803K.[000000-003194].mov
M241806N.mov to M241806N.[000000-004983].mov

And now the problem was solved -- Resolve made all linking correctly, without reel name.

Of course you cannot do that for big projects, but I think solution is in file naming -- to make Resolve import clips and render clips with the SAME NAME.

3. 1.4.8 version.

Presently I use 1.3.0 because 1.4.8 is totally unstable for me -- it crashes in the middle of every second conversion. 1.3.0 is as solid as a rock ))

I also tried MLV conversion after placing mlv_dump.exe into the R2C folder, but the program shows only one frame from the clip. Failed to convert.

And by the way, on the screen shot you have 1.4.9. Are you hiding anything?  :D  Just kidding!

Let me say big THANK YOU for your sincere efforts! Happy Christmas!

Title: Re: The CinemaDNG Discussion
Post by: mediali on December 25, 2013, 05:13:54 PM
same problem other conver is work but raw2cdng have wrong in this :
import in premiere is same problem

 2.1. FPS. Now, I am shooting 25 FPS, but when imported to Resolve, all clips are 24FPS. Have to change it manually. When I forget, I have a de-sycnh with audio, and have to re-render. But this is the beginning for surprises ))
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 26, 2013, 11:03:49 AM
Happy Christmas, mediali & rommex.

Thanks. So, the fps-problem is a thing inside resolve. You have to adjust the basic settings of your project. It seems, resolve is not interested in the fps-metadata.

The problems Wirth the newer versions will come From the multithreaded reading. I'll rewrite that. So stay on 1.3, for resolve it makes no difference.
Title: Re: The CinemaDNG Discussion
Post by: rommex on December 26, 2013, 11:54:40 AM
Happy Christmas, mediali & rommex.

Thanks. So, the fps-problem is a thing inside resolve. You have to adjust the basic settings of your project. It seems, resolve is not interested in the fps-metadata.

The problems Wirth the newer versions will come From the multithreaded reading. I'll rewrite that. So stay on 1.3, for resolve it makes no difference.

Thanks, chmee. FPS is not big deal, really. But what about the naming issues? Any plans to resolve them?
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 26, 2013, 07:53:48 PM
Im not home, writing on a Smartphone. When im back, i'll answer the important things
Title: Re: The CinemaDNG Discussion
Post by: Canon eos m on December 28, 2013, 06:33:13 PM
Hi,

I tried to convert an old file using raw2cdng 1.3.0 and view it on PPro CC - came out all pink.
I tried doing the same thing on AE CC and came out just fine. I think it is the PPro CC that is causing the issue.
Title: Re: The CinemaDNG Discussion
Post by: Andy600 on December 28, 2013, 10:56:04 PM
@Chmee - Do you know how to change/update exif in CDNGs?

I want to add camera model to some old CDNG files converted with an old version of raw2cdng. I don't have the original .raw files. I've tried using Exiftool but it keeps throwing up errors. It seems to work with normal DNG files.
Title: Re: The CinemaDNG Discussion
Post by: Canon eos m on December 29, 2013, 07:20:14 AM
Chemee, my question too, how do I add camera model to version 1.3.0. 1.4.8 is not very stable yet.
Title: Re: The CinemaDNG Discussion
Post by: chmee on December 30, 2013, 09:48:28 AM
changing the model ID is not a one-liner, sorry. (use a hex-editor) It's the tag id's 110 (hexoffset 5e) and C614 (hexoffset 106) - it writes its string-length and the string offset into the tag-data. i dont know if i wrote the string on the end of the metadata, possibly the length of the string changes the offset of forthcoming data - makes following entries illegal/defective.

so, i tried to write a more stable and better version to christmas (you've seen the 1.4.9 :)) but the time before christmas is full of familiy-action. give me some days after newyear to stabilize raw2cdng.

toDo for the next version.
* stable read and write with mutlithreading (had a chat with g3gg0 about it)
* dng-header generation - modul rewrite
* IliasG's new LUT for 10bit/log (will work for older bodies with blacklevel 1023 as well)
* 12bit/log with bmcc like data (hopefully we're coming to an lovely workflow with premiere)

as is mentioned before, Resolve 10 is no longer a problem, so i dont want to "improve" any things there. speedgrade is working as well - but premiere and the dynamiclink-workflow lacks of being consistent/stable .

(happy new year - and: magiclantern rocked this year, so all my technical love goes to the ml-crew that made this happen)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: pind on December 31, 2013, 02:21:03 PM
thnx chmee works like a charm!!! :D

Can i ask if i need to tick partymode?


Should i tick TIF Model Tag ?

thank you

Title: Re: The CinemaDNG Discussion
Post by: chmee on January 04, 2014, 10:18:39 AM
@pind the partymode is for better pictures in premiere, "could" also help in resolve and speedgrade, but it worked without it - its up to you. the TIF Model Tag is for using profiles in ACR, leave it untouched if you dont need it.
Title: Re: The CinemaDNG Discussion
Post by: pind on January 04, 2014, 06:25:53 PM
@pind the partymode is for better pictures in premiere, "could" also help in resolve and speedgrade, but it worked without it - its up to you. the TIF Model Tag is for using profiles in ACR, leave it untouched if you dont need it.


Thank you
Title: Re: The CinemaDNG Discussion
Post by: 1% on January 04, 2014, 07:56:07 PM
To bulk edit: http://freeweb.siol.net/hrastni3/foto/exif/exiftoolgui.htm
Title: Re: The CinemaDNG Discussion
Post by: Freddy2142 on January 05, 2014, 04:07:36 PM
Hey,
I made a small GUI for "RAW2DNG".
You can download and test it here (Adfly) (http://adf.ly/cC9up) or here (Direct) (http://www.mediafire.com/download/o47z423ge69yesq/RAW2DNG%20GUI%201.2.zip).
If there are any bugs/problems PM me, thanks.


Screenshots (http://imgur.com/a/pOZR3)

Changelog:

Version 1.1:
- some bugs fixed
- add "Export DNG's to RAW file folder"

Version 1.2:
- some bugs fixed
- 500% faster
Title: Re: The CinemaDNG Discussion
Post by: ulysnep on January 13, 2014, 01:38:23 AM
Hey,
I made a small GUI for "RAW2DNG".
You can download and test it here (http://www.mediafire.com/download/bv4nv66qceog18q/RAW2DNG%20GUI.zip).

If there are any bugs/problems PM me, thanks.


Screenshots (http://imgur.com/a/pOZR3)

This is AWESOME! Thanks for putting this together Freddy!

Would it be possible to have an option to export the DNG files to the Raw file folder?

Many thanks again!
Title: Re: The CinemaDNG Discussion
Post by: Freddy2142 on January 14, 2014, 04:52:42 PM
Would it be possible to have an option to export the DNG files to the Raw file folder?

Here is the updated version: Adfly (http://adf.ly/c65aN) or Direct (http://www.mediafire.com/download/xgoh5halh0uf9si/RAW2DNG%20GUI%201.1.zip)
Title: Re: The CinemaDNG Discussion
Post by: arturochu on January 15, 2014, 12:15:43 AM
nice work here, hope to see native support for mlv format and dual iso mlv format somewhere in the near feature.
Title: Re: The CinemaDNG Discussion
Post by: arturochu on January 16, 2014, 07:39:10 AM
tried to use latest version with a bunch of mlv files, and the app crashed. bummer.
Title: Re: The CinemaDNG Discussion
Post by: arturochu on January 16, 2014, 07:46:48 AM
i was kind of wrong, it did convert all of the .mlv files to the .raw extension, it crashes when i clicked the convert to dng files.
Title: Re: The CinemaDNG Discussion
Post by: pind on January 16, 2014, 11:43:03 PM
i was kind of wrong, it did convert all of the .mlv files to the .raw extension, it crashes when i clicked the convert to dng files.


i had some crashes with this too, so i used this tehnique


Import your raw files from CF card to Data storage (HDD) and after you do this take one raw file out from HDD and convert.. when this will work click on other 50Gigz of raw files you have (on your 64GB CF card) and convert all ... thats what i do and dont have crashes...
Title: Re: The CinemaDNG Discussion
Post by: Malex on January 20, 2014, 11:16:38 PM
Can we hope for a 1.5 version that works with MLV+sound, and still works with .RAW also...? :-)
Title: Re: The CinemaDNG Discussion
Post by: Eran on January 21, 2014, 04:11:15 AM
Can we hope for a 1.5 version that works with MLV+sound, and still works with .RAW also...? :-)

That's a good point and could save us some time on the workflow.
Title: Re: The CinemaDNG Discussion
Post by: chmee on January 23, 2014, 05:57:20 PM
Yes, you can hope for a MLV-Version - sound is not on my primary list, but it would make less sense doing it without the audio part :) please be patient some more days. you're not forgotten..

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: katerlouis on January 24, 2014, 03:48:58 PM
I read the last 5 sites now. I'm sure I missed why the discussion on the pink cast in premiere with CDNG files was dropped.
Does your app Raw2CDNG fix the problem so we have access to ACR when importing files to Premiere?

Currently im using RAWMagic with some kind of Ae workaround since my director needs to do many animations and effects in Ae anyways.
Question unanswered to me yet is 8-bit, 10-bit, 12-bit thing-

Do we at least have the full RAW quality in Ae?



Thx for getting me on track :)
Title: Re: The CinemaDNG Discussion
Post by: Canon eos m on February 03, 2014, 10:03:53 AM
Could some one please help. I can convert raw footage easy with raw2cdng but cannot do this for mlv_dump files despite mlv_dump being enabled. Read chmee's website and instructions but cannot figure out what I am doing wrong.
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 03, 2014, 11:18:23 AM
@canon eos m
the more you tell about the problem, the faster you can get real help :)

* if you drag/drop mlv-files, the tool starts immediately with converting to raw.
* these temporarily converted raw-files should lay inside the raw2cdng-folder. you see them?
* after that you should see the filename and main specs of the footage in the application-filelist. do they? are they ok?
* they behave as normal raw-files (because they are :)

But, i need some help of you, guys. i'm kind of desperated, because of the adobe-behaviour (http://forums.adobe.com/message/6014956). my goal was to get a functional import into premiere (still it is) - importing into speedgrade and resolve isn't anymore a problem, it works. i thought of writing a standalone-2tif-converter(16bit) with built in ssd-demosaicing (http://www.ipol.im/pub/art/2011/bcms-ssdd/). biggest drawback is the monstrous filesize-increase, but the usability straight inside premiere could be worth it. by now, i'm stucked, and ml-raw seems to me like a "proof of concept" with a big impact on the professional vendors/scene/market, but less benefit for the workhorse, doin its job. simply i think, cdng is/was the wrong choice.. actually i rather go h.264-vanilla because of the instant workability.

so finally, whats your thoughts about efficient workflow? staticaly converting with ACR is not (imho).

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: Canon eos m on February 03, 2014, 02:11:51 PM
I am beginning to feel the same way - H.264 codec (with the .mov container) pretty much rules for casual / leisure recording.

I just got back from vacation with my family and recorded both H.264 and mlv video. The mlv video runs into 100's of MB and I don't know what to do with it now. I have been searching for a good workflow to sort the files out and render the video's but nothing seems easy. But then ML is not for those that are not ready to toil and sweat it out.

I consider (IMHO) raw_rec as a better option than mlv_rec (if you are not relying on raw_rec for sound recording).  I also prefer H.264 over ML video (both raw and mlv) in most situations. If I want to record professionally then the extra effort will be worth it ... else H.264 quality if fine for now.

Most frustrating is not being able to use premier pro to collate the dng's as this is the first and essential step for a rewarding dynamic exchange between after effect and speedgrade.

My workflow for now is:

1. View the mlv's on 'mlvrawviewer'
2. Convert mlv using 'mlvconverter' (now want to use 'raw2cdng' but struggling a bit with mlv conversion). I think hereafter I will work more with raw_rec rather than mlv_rec.
3. Collate the files on AE CC and then transfer on PPRO CC for editing. Bring back to AE CC for effects /Speedgrade for grading, etc. and finally render with PPRO CC.

Everything else seems to be a waste of time. 
 
Edit: I read that PPRO CC supports Blackmagic's version of DNG. Is it possible for the developers to somehow tinker with the Canon metadata during conversion to replicate the BM DNG specs. This will then make them readable to PPRO CC. Adobe has really let us down. DNG is an Adobe creation but .......
Title: Re: The CinemaDNG Discussion
Post by: Canon eos m on February 03, 2014, 03:08:40 PM
@canon eos m
the more you tell about the problem, the faster you can get real help :)

* if you drag/drop mlv-files, the tool starts immediately with converting to raw.
* these temporarily converted raw-files should lay inside the raw2cdng-folder. you see them?
* after that you should see the filename and main specs of the footage in the application-filelist. do they? are they ok?
* they behave as normal raw-files (because they are :)

But, i need some help of you, guys. i'm kind of desperated, because of the adobe-behaviour (http://forums.adobe.com/message/6014956). my goal was to get a functional import into premiere (still it is) - importing into speedgrade and resolve isn't anymore a problem, it works. i thought of writing a standalone-2tif-converter(16bit) with built in ssd-demosaicing (http://www.ipol.im/pub/art/2011/bcms-ssdd/). biggest drawback is the monstrous filesize-increase, but the usability straight inside premiere could be worth it. by now, i'm stucked, and ml-raw seems to me like a "proof of concept" with a big impact on the professional vendors/scene/market, but less benefit for the workhorse, doin its job. simply i think, cdng is/was the wrong choice.. actually i rather go h.264-vanilla because of the instant workability.

so finally, whats your thoughts about efficient workflow? staticaly converting with ACR is not (imho).

regards chmee

Still no luck chmee! I am missing a crucial step here. Not sure what else I can do. But will keep trying  ;) till I get it. Will let you know once I succeed.  :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 03, 2014, 03:41:09 PM
@canon eos m
can you confirm some of the steps i described?
Title: Re: The CinemaDNG Discussion
Post by: Canon eos m on February 03, 2014, 04:12:17 PM

* if you drag/drop mlv-files, the tool starts immediately with converting to raw. confirm dragging and dropping
* these temporarily converted raw-files should lay inside the raw2cdng-folder. you see them?
cannot see these filed in the raw2cdng-folder or the output folder
* after that you should see the filename and main specs of the footage in the application-filelist. do they? are they ok? cannot see these in the application

Title: Re: The CinemaDNG Discussion
Post by: Canon eos m on February 03, 2014, 06:09:08 PM
* if you drag/drop mlv-files, the tool starts immediately with converting to raw. confirm dragging and dropping
* these temporarily converted raw-files should lay inside the raw2cdng-folder. you see them?
cannot see these filed in the raw2cdng-folder or the output folder
* after that you should see the filename and main specs of the footage in the application-filelist. do they? are they ok? cannot see these in the application

Error message:

MLV Dumper v1.0

Mode of Operation:
   -   Input MLV files: 'E:/5DM3'
   -   Convert to legacy RAW
   -   Output into 'M01-1554.RAW'
[ERROR] Failed to open file 'E:/5DM3'

Edit: Problem fixed. It was something to do with my directory. Shifted to another directory and now mlv's are converted with ease  :) Thanks!

Edit 2: Another problem. The files are all pink in ACR.
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 03, 2014, 06:34:26 PM
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fraw2cdng_desc.jpg&hash=534b4d9f64151e183d2e0f38ecf326fe)

For ACR-Applications (AE,Photoshop), Speedgrade and Resolve please use
*(5) 16bit linear
*(6) no partymode, Gamma 1.0
*(4) enable and fill in the according Model ID for right profiles. Examples (http://photo.net/casual-conversations-forum/00aJ4B)
*(+) for Speedgrade you have to use a calibration-lut, i put one example into the zip.

For Premiere CC (works with Resolve as well):
*(5) 12bit linear
*(6) partymode enabled, Gamma 1.0
*(4) Model ID not needed.

Should work.
Title: Re: The CinemaDNG Discussion
Post by: Canon eos m on February 03, 2014, 06:47:00 PM
Wow! Worked wonders! Thanks a million tons!  ;D

Edit: Files okay in ACR but pink in PPRO CC.
Title: Re: The CinemaDNG Discussion
Post by: Malex on February 07, 2014, 10:09:31 AM
Hi chmee, have you seen this thread of a new MLV converter : http://www.magiclantern.fm/forum/index.php?topic=10198.0

I think that you and tonybeccar should join force to unify both of your apps.

since MLV got not the sound + this new MLV converter, many people are moving to .MLV

This converter is working fine with just few bugs with proxies. What this converter is missing is the partymode that you created! and it misses dragndrop also.
I guess just like MLdevs are working together to make ML, you guys could work together to make a great app for the ML community  ;)

Title: Re: The CinemaDNG Discussion
Post by: chmee on February 07, 2014, 03:20:22 PM
Yes, of course i've seen. quite happy that he's doin' in the time i stuck because of a lot of work. But..

The reason i cant help there is (i think) his modular way. Giving the MLV to mlv_dump, recreating DNGs with the adobe tools or Prores with ffmpeg. To get it working with premiere, you have to recalculate the raw-data and write own TIFF-Tags into the DNG-Header. By now i dont know if the adobe-converter is able to do that.

But, i appreciate and respect his work really.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: tonybeccar on February 08, 2014, 08:49:28 PM
Hey chmee and Malex, thanks so much for the appreciation!

As I said in my thread, I am NO programmer whatsoever!! Hehe.. I'm just a filmmaker who happened to do a little programming as a kid! So by now using Visual Basic wasnt that hard to understand. And chmee, you are absolutely right:

As an excited filmmaker I found this raw discovery AMAZING, yeah, as we all. So I can see that there is still work to be done, holes to fill, etc. So I decided with my little knowledge to try to make a workflow app, and yes, it points out to other programs, everything points out to either mlv_dump, dng converter or ffmpeg. My app is simply joining ALL the forces together, creating a chain and uniting the workflow. I understand that you chmee had programmed the converter from scratch, and that's very different. But maybe, my modular system can be embraced with a command line tool for cDNGs, if they could be extracted from the MLV and I get access to a command line, then we're grand. But of course, the best solution would be to implement it right inside the program.

If I'd to tell you the aim to my app, is to have just ONE app for magic lantern (excuse the ambition, hehe), but really, not to complicate things anymore.. just one app that can transform those hot MLV files into anything you want and you're ready to edit. And well I noticed that with cDNGs you get pretty much an easy workflow with Premiere.. anyway.. will be willing to help in what I can!!

Think about it ;)
Cheers!!!
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 09, 2014, 04:56:49 AM
I'd like to take the opportunity to thank both of you and the rest of the authors, programmers, and members of this community for all they have done and continue to do.  ML is an amazing piece of work!

Yes, MLV2CDNG is the "Holy Grail" for us ML users (.MLV format) and I await the day when I can just drop files onto a GUI and open up the converted files in PPro CC without any glitches or pink cast.  I was considering hiring one of those whiz kid programmers from freelancer.com and just have them write the program. 
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 09, 2014, 02:02:28 PM
:) i took a test yesterday with a 5DIII-Build from Feb01.2014 (65572fa87394a2598ef4ee75c547b750d46c6963) (http://builds.magiclantern.fm/#/) - MLV-recorded, raw2cdng 1.48 with 12bit linear and partymode gamma 1.0 looked quite good - i was not expecting this.

online: mlv-file with a testchart - 141MB - ~100 frames 1280x720
http://dl.phreekz.de/M09-0151_testchart.zip

drag,drop,do - import into Premiere CC - worked..

* actually i should implement audio-output.. done.
* and i'm sitting on better dng-colormatrices.. fi for speedgrade. (fi D50 is not implemented)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: andy kh on February 09, 2014, 06:34:52 PM
it would be so great if we can sync the audio and cdng automatically in resolve
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 09, 2014, 07:23:48 PM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

raw2cdng v1.4.9 is out. MLV-audio-implementation. please test.
mlv_dump.exe (feb08?!) with audio-support from g3gg0 is inside the zip.
* audiofile is found in the same path as dng's
* after conversion, in premiere you just have to import the dng-sequence, the audiofile will be merged automatically.

toDo for 1.5
* native mlv-implementation (but not that important, because we'll get all news from mlv_dump :)
* calibration-luts/matrices could be better.
* maybe whitelevel should be adjusted, you try and report, i hope.
* multithreads optimizing.
* things you report.

@andy_kh
resolve's got a strange (better: simple) behaviour with framerates. You have to "re-set" the project-setting to use another framerate - maybe that's the reason.. in premiere it work's well.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: andy kh on February 09, 2014, 08:27:38 PM
wow!! this is amazing. one problem with me.. i can drag and drop mlv to raw2cdng from my memory card but i cant do it from hard drive. i wil do more test. raw2cdng v.1.4.9 is fantastic. this wil make my worflow very fast. thanks a lot chemme.
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 09, 2014, 08:36:01 PM
try another pathes or partitions, just for tests and report, if it changes..
fi raw/mlv-files on root, and export-path just one path deeper..
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on February 09, 2014, 08:46:55 PM
tried the new convertor, works well . I got all white frames when I selected 12bit log bmcc
in premiere pro.
Title: Re: The CinemaDNG Discussion
Post by: andy kh on February 09, 2014, 08:55:57 PM
yeah it works when i tried another partition
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 09, 2014, 08:56:18 PM
@reddeercity
thanks for info, please try 12bit linear with partymode/g1.0

@andy_kh
like that :)
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on February 09, 2014, 09:02:40 PM
with the 12bit log bmcc I adjusted gamma to 1.0 & use partymode
that was better. can see the image now
Edit: that much better 12bit linear partymode + gamma 1.0
looks very close to ARC .
Title: Re: The CinemaDNG Discussion
Post by: arturochu on February 10, 2014, 12:37:24 AM
dual iso support plz!
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 10, 2014, 12:50:25 AM
[dualISO] will look on that. but, be sure i will use the cr2hdr.exe-converter from a1ex..

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: arturochu on February 10, 2014, 03:56:20 AM
[dualISO] will look on that. but, be sure i will use the cr2hdr.exe-converter from a1ex..

regards chmee

nice to hear that, one constant problem i have doing most of dual iso video post processing is flickering (sometimes really bad, sometimes subtle)
Title: Re: The CinemaDNG Discussion
Post by: tonybeccar on February 10, 2014, 06:07:15 AM
Hey chmee, so I finally tried your program, your workflow, AND IT IS AWESOME!!!! :D Amazing, simply amazing... the quality you get inside premiere pro with its rudimentary color correction tools it's the same as ACR, suuuuuper cool!

So was doing color correction and encounters pink edges on the picture, when you push the colors and add sharpen especially, here are 2 frame grabs, original cdng and after the grade:

Before: https://www.dropbox.com/s/5oy8fyl6bhsycnf/Before.tif
After: https://www.dropbox.com/s/ecrznjbfu1p7puk/After.tif

Do you know what's happening? I did cDNG 12 bit linear, tiff header thingy and partymode gamma 1.0.
Thanks and awesome work mate!

PS: Changing topic.. as I installed the new premiere pro, I noticed that there is an option to encode prores in windows, WOW? It doesnt work for me as I had to downgrade quicktime because it didnt let me import image sequences in ae. But oh my, is it really?!
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on February 10, 2014, 06:29:31 AM
I installed the new premiere pro, I noticed that there is an option to encode prores in windows, WOW? It doesnt work for me as I had to downgrade quicktime because it didnt let me import image sequences in ae. But oh my, is it really?!
?? ProRes on Windows in Premiere Pro CC  ???
Where are you seeing this. I have been working in there and don't see this.
The only way to encode prores on windows it though FFmeg in a standalone program
like "Cinemartin" or 5DtoRGB (I use that for all my h264)
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 10, 2014, 10:06:34 AM
@tonybeccar i cant really differentiate between recalculating-problems on my site and dng-importer/handling-failures on adobe-site anymore.. i've seen such problems with bmcc and bolex-files as well..

!!!! important to know: when working with cdng-files, you should only use 32bit-filters - the rest is 8bit and will destroy all advantages of more bitdepth.

@arturochu i didnt used dualISO yet, so we will see, if i can handle that :)
 
regards chmee
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 10, 2014, 12:28:50 PM
raw2cdng v1.49 is very slick.  It is fast!  raw2cdng>>ACR >>batch adjust/save>>Import from PPro media browser>> done.  So far this is the best option I have found, works really good chmee.  I use 12/16 bit linear/party mode/Gamma 1.0, works good, no pink cast files look good in PPro.  One question, is this supposed to work with .mlv files also?  If so, how to convert .mlv?

Very nice!  Good work.
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 10, 2014, 01:38:48 PM
Here are my tests with v.1.49...

Convert RAW files from SD card:
16 bit linear, party mode(ON) Gamma 1.0 = Result - no issues.
12 bit logarithmic, party mode(ON) Gamma 1.0 = Result - Strong green cast.
12 bit linear, party mode(ON) Gamma 1.0 = Result - no issues.
10 bit logarithmic, party mode(ON) Gamma 1.0 = Result - no issues.

16 bit linear, party mode(OFF) Gamma 1.0 = Result - no issues.  Faster conversion speeds!
12 bit logarithmic, party mode(OFF) Gamma 1.0 = Result - Strong green cast.  Faster conversion speeds!
12 bit linear, party mode(OFF) Gamma 1.0 = Result - no issues.  Very fast conversion!
10 bit logarithmic, party mode(OFF) Gamma 1.0 = Result - Slight pink cast and lots of added noise.

Question:  Does v1.49 work with .mlv files?  I tried dropping a couple on the GUI but nothing happened.  I have mlv_dump.exe in the raw2cdng folder as well.  The conversion speeds are very fast on my 1280x720 15 second RAW/MLV clips.  This is definitely a time saver.

One other thing, I noticed a few cold/dead pixels in my raw/mlv files, wondering if that is from the conversion or my raw settings?
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 10, 2014, 02:15:41 PM
@duey101
[MLV] it should. mlv_dump.exe (with audiosupport) is inside the zip. If you drag&drop mlv, you should generally see a question by windows, if it can open mlv_dump.exe. Please enable logging and look into the logfile. maybe it turns out, you have to copy the mlv-files first onto the hdd instead of converting straight from the card.

[coldpixels] i cant decide, from where the pixels are coming. Do you have tested 16bit/without partymode? (these files are openable in speedgrade, rawdigger or photoshop/acr) i do not recalculate the raw-data in this setting..
Title: Re: The CinemaDNG Discussion
Post by: tonybeccar on February 10, 2014, 04:30:00 PM
@chmee, holly molly I didnt see your new version yesterday.. will try it now! Abd oh, too bad then, but good advice, will try the 32bit filters then..

I wonder what news about mlv_dump we'll be seeing... :) And also I hope you make the command line a priority!!!

Thanks SO much chmee!!! NOW we have a workflow! ;)
Title: Re: The CinemaDNG Discussion
Post by: tonybeccar on February 10, 2014, 05:34:39 PM
?? ProRes on Windows in Premiere Pro CC  ???
Where are you seeing this. I have been working in there and don't see this.
The only way to encode prores on windows it though FFmeg in a standalone program
like "Cinemartin" or 5DtoRGB (I use that for all my h264)

Yeah, it IS true! I'm running 7.2.1 (4).. the file formats are all different now.. I have a lot of image sequences for example.. and inside Quicktime MOV I have this:

https://www.dropbox.com/s/eicb6qmz1aya7pl/prores.jpg

But whenever I hit export I get a compiler error... maybe I screwed things up myself.. but it IS there!
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 10, 2014, 05:57:25 PM
Man... all I can say is thanks!  This converts my .RAW files with lightning speed.  I'm still not sure on the dead/cold pixels issue.  I noticed they are on all my files whether shot in MLV or RAW.  I'll try some other settings tomorrow.  I'm shooting in MLV/RAW on a 60D with...
1280x720
16:9
fps override on 24fps
Global Draw off
Frame skipping off
Preview - Hacked No Prev
Extra hacks - On

The pixels appear in both MLV/RAW files.  You can see here...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fpixels.jpg&hash=3906bbec3c68401573a231d041afbd38)

The other issue is with MLV files.  Here is what my raw2cdng window and raw2cdng folder contents looks like...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fraw2cdng.jpg&hash=b252432ee16c712102d011da6b87f4e4)

But if I try to drag an MLV file onto the window nothing happens... although I can see a DOS window flash briefly as soon as I drop the file in, but nothing happens then, the file does not drop onto the GUI and Win 7 x64 doesn't ask any questions.  Not sure why I can't get the MLV files to work.  Any ideas?

The RAW files work perfectly as far as I can tell, pull right up in ACR for batch DR correction, and edit smoothly in PPro CC 7.2.1 (4).
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 10, 2014, 06:00:55 PM
@duey101 i bet, the mlv_dump does show a "file not found" error, maybe of a longer path-tree or unusual symbols in pathnames.. Please try to rename the rawcdng-folder. All points to underlines.

[deadpixels] please try another converter (fi raw2dng) and examine the pictures with rawdigger, lightroom or something else. just to clear, if its sensordata or software-creation-failure.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 10, 2014, 06:58:11 PM
I moved the folder to my root D:/ rive and now it works.  Renamed the folder to raw2cdng only and that seems to have worked, takes a lot longer to convert MLV>RAW though!  But the RAW to CDNG is really fast!   :D
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 10, 2014, 07:01:11 PM
raw->dng will be faster next versions :) (with mlv, please forgive me, i take the advantages of the reference-implementation of g3gg0)
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 10, 2014, 07:41:21 PM
Here is some interesting stuff...

Convert with raw2dng, clean no weird pixels

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fraw2dng.jpg&hash=c3a2c903e2c37cf97458522cf5878f86)

Then convert with raw2CDNG... 12/16 bit Partymode On and Off same results, funky pixels.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fraw2cdng1.jpg&hash=214b3373579069d6944f296a7d2a06d2)

And then I loaded the same file in PPro direct with Drastic's plugin installed again (after uninstall) and I get this... funky pixel hell!

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fpremiere.jpg&hash=61ba281bb6b2d43e0ead84cfcbb5bf67)

Hmm....
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 10, 2014, 07:45:30 PM
the screenshots are all from premiere? resolve? photoshop? (fi acr uses algorithms to hide these pixels, my dng's and the drastic-importer have no "repair-sensels"algorithms..)
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 10, 2014, 07:51:04 PM
The first two are from Lightroom 5, I converted the raw's in raw2dng/raw2cdng and opened both files in LR5 and took screen caps.  The last is from a single frame export from the PPro timeline.
Title: Re: The CinemaDNG Discussion
Post by: arturochu on February 10, 2014, 08:25:04 PM
my premiere doesn't even have quicktime mov option! i'm running latest version, i would love to export to prores in windows, i think you have something installed in your pc that activated those codecs.
Title: Re: The CinemaDNG Discussion
Post by: arturochu on February 10, 2014, 09:20:45 PM
Yeah, it IS true! I'm running 7.2.1 (4).. the file formats are all different now.. I have a lot of image sequences for example.. and inside Quicktime MOV I have this:

https://www.dropbox.com/s/eicb6qmz1aya7pl/prores.jpg

But whenever I hit export I get a compiler error... maybe I screwed things up myself.. but it IS there!

okj i got it, you can export from premiere cc in windows IF you install mediareactor, then a lot more codecs will appear, including prores, i've already donde this but actually i can export to prores with no problems!  :)
Title: Re: The CinemaDNG Discussion
Post by: tonybeccar on February 11, 2014, 04:27:07 AM
okj i got it, you can export from premiere cc in windows IF you install mediareactor, then a lot more codecs will appear, including prores, i've already donde this but actually i can export to prores with no problems!  :)

Lol, yes installed it and uninstalled it.. another victim of the mediareactor greatness..
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 11, 2014, 02:35:28 PM
Your converter is looking VERY NICE!!!  After more testing today I have found that to convert MLV files with raw2cdng they need to be in the same directory as the raw2cdng program and that needs to be on a root drive for example, D:/raw2CDNG works for me.  At least this seems to be the case for me.  While processing the MLV files this window pops up...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fwindow2.jpg&hash=b9675a22fb059185a874dddc41c2d7e7)

But the files finish converting (slowly) and then appear in the raw2cdng GUI, after that they convert really fast to CDNG.  The mlv_dump process is the only bottle neck in this method of converting MLV files.  Here are some more test images from today.

raw2dng, no issues.  (4:1 magnification)

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2F0raw2dng.jpg&hash=bd44688cef2854dce4959033441df52a)

raw2CDNG, very slight problem on a few pixels, or maybe blown highlights or something?  Almost undetectable.   (4:1 magnification)

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2F0raw2cdng.jpg&hash=cd05e7e2f441667841e0f67477788973)

MLV files converted through raw2cdng...

File 1

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fmlvconverted1.jpg&hash=aac60c86c40c652334fc9027a1d8fd22)

File 2

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fmlvconverted2.jpg&hash=e1afe7dca5ee24d10d65961e92be8440)

Both MLV conversions are crispy clean, no pixel problems.  I completely uninstalled DMR and Preview, not sure if that had anything to do with it.  But while it was installed I did some more conversions and got this clean raw2cdng file when opened in Light Room 5...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fraw2cdngLR5.jpg&hash=3c66f8f1bec4d24869a281c70792ae03)

But then when I opened the exact same .dng file in PPro I go these pixels showing up...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fraw2cdngAPP.jpg&hash=b098c54c02dfb0068ffb5d825ecbeb57)

Still testing...


Title: Re: The CinemaDNG Discussion
Post by: chmee on February 11, 2014, 03:56:58 PM
@duey101 thx a lot for your testing parcour.

(/me) important toDo: native mlv with splitfile-support :D

* Are these corrupted Sensels static?
* did you tried rawdigger with both files? (raw2dng vs. raw2cdng)
* are you using a proper modeltag?
* if you want, send me the test-dngs to raw2cdng@phreekz.de

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 11, 2014, 06:51:35 PM
* Are these corrupted Sensels static? 
Static as in do they move around when the video is played on the timeline?  No they do not move but remain where they are.  What does Sensel actually mean?  (sorry, new to these terms)

* did you tried rawdigger with both files? (raw2dng vs. raw2cdng)
I never used that program but downloaded just now, have no idea what it does.  LOL!!!  When I open both files in Rawdigger the raw2CDNG file looks okay, the raw2DNG file looks pinkish.

* are you using a proper modeltag? 
I'm not sure, this is how it looks when I convert...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Frawwindow.jpg&hash=999e8996897a45b32794ee43dc0b9aca)

I got that model name off of some list that I think was EXIF data for specific cameras, where do I get the correct name for a 60D?

In some of the RAW files I get these pink corrupted frames (2 or 3 per clip).

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fpinky.jpg&hash=0ca0fd0c8eec2cd2a10213ada1bc8d25)
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 11, 2014, 07:17:54 PM
After converting several formats, H.264>>ProRes (with Bigasoft Converter) Cineform>> (GoPro Studio) and a few others I have tried, these CDNG files are the best and edit in real time seamlessly even with effects dropped on them and in full res.  The native H.264 files are horrible to try and edit with and even ProRes files bog down in 1/2 and 1/4 res.  The CDNG workflow is the way to go.  It is really smooth in PPro editing wise so far.  Glitch free scrubbing and playback at full res, on my machine anyway.  You don't have to import all the clips in as a sequence or anything like that, just convert and drag into the project bin from the Media Browser in PPro.

You are on the right path, and close with this.  The best thing is how FAST it converts the RAW files!  Fast!  MLV is another story because of that bottleneck, so yes, native MLV may be the way to go.

Oh... on some clips I get weird color banding in PPro like in this image (3X magnification) screen shot from PPro single frame export.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fcolorband.jpg&hash=17429de00e67f67a125981e8454de694)

But doesn't appear if the same CDNG is opened in Photoshop.  Only looks like that in Premiere.
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 11, 2014, 07:58:49 PM
thank you for the motivational words :)

[static sensels] The Name Sensel comes from the fact, its the "pixelated" part of a sensor, but not a pixel, because 2/3 of its destinated value is calculated. Wiki says, its a blend of "sensor" and "cell". Part of an array of sensors.

[Rawdigger] has one big advantage, its good to analyze raw data - so if there are real dead pixels, they should be present in both files, as rawdigger do not reconstructs and refines the picturedata.. it seems, its the case.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.upload-pictures.de%2Fbild.php%2F50065%2Cdeadsenselsrawdigger5SGA0.jpg&hash=0e307381dce533080d2b0a08d3fcbc1f)

[banding][moiré] do you record in zoomed mode? these "problems on hard contrasty edges are logical because of the bayer-structure - in premiere more than in ACR/Lightroom (..better debayer-algorithms). BUT i've been asked messages before if its depending on my tool. Yes, it seems, my recalculation makes it more obvious - and No, its already there..

[corruptedFrames] Oh.. maybe there is an "coincidence" with the amount of splitted parts? 2 damaged Frames with 3 SplitFiles?

my 5 cents. regards chmee
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 12, 2014, 10:28:18 AM
Okay that's good to know and thanks for the education.  When that dead pixel shows up in the rawdigger does that mean my sensor actually has a dead pixel in it?!!!  :o  Or is is that during the recording in raw a dead pixel happens?  I hope my sensor isn't being damaged.  Still new to all this technical stuff.

Those shots with the color banding were shot in 5X crop mode so I get how that would effect the image.  Overall, when I shoot raw on my 60D I can get 400-420 frames in MLV mode and about 7-10 seconds in RAW mode @1280x720 which scales up quite well to 1920x1080.

As far as the pink corrupt frames not sure what the cause of that is.
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 12, 2014, 12:08:05 PM
Do a simple CR2-raw-Picture and load it into rawdigger. Look, if the deadpixels are present. And Yes, Dead or Hot-Pixels/Sensels are typical - but in your case its a lot ;) If its really that much, i would contact your distribution.. i won't believe, the local dealer will handle that, but another higher instance of canon.. as a gesture of goodwill maybe..

ahh, btw. these corrupted Frames are not caused by conversion, but seems to be a "problem" with too high recording-datarate.. MLV is a little bit slower than raw_rec.
http://www.magiclantern.fm/forum/index.php?topic=10257.0
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 12, 2014, 02:15:59 PM
I loaded a raw CR2 file and didn't see any deal pixels so what does that mean?  Why do they show up in the RAW/MLV converted CDNG's?  I don't get it.  ???
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 12, 2014, 03:00:06 PM
just one more test.
Do they show up, if you're using the reference converter? fi. raw2dng, mlv2dng, mlv_dump?
Do they disappear, if you're using lower resolutions? (datarates)

I'm not soo into it, to decide now, where the failure's coming from.. sry.
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 12, 2014, 03:06:18 PM
I just ran Rawdigger and used a picture of a white wall (off white) and then blew it up in Rawdigger.  Looks like this...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fpixeltest.jpg&hash=87e3f65856d28c16bb4fdb3c7751ac8a)

Other than these wht/grey colors there are no black/hot/weird looking pixels, they all look like this so I think my camera is fine, no dead pixels or other colored pixels at all.

Let me test the other options and get back to you.
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 12, 2014, 04:02:49 PM
Okay, after another battery of tests here is the result.

raw2dng - no issues
raw2CDNG convert raw file, no issues, convert mlv file no issues.
mlv2dng - no issues.

All the files are clean including the 10X crop.  Now...

If I take the raw2CDNG file and open it in PPro, drag it to the project window, open it in the source monitor, blow it up to 400%... THEN I see the dead/black/sensel issues.  Look here...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fppro-deadpix.jpg&hash=1a096bb3085ce7709358396bb1410d3c)

But outside of PPro these pixels from the same DNG/CDNG files are not present.  I sent you the converted files in email.  I don't know with the mlv2dng or raw2dng files cause they don't open in PPro (because they are not Cdng I assume).  EDIT: And if I output the sequence from PPro the black/dead/sensels show up in the output video, but oddly enough not in the files outside of PPro.
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 12, 2014, 05:31:14 PM
One more test, with the new version of MLVConverter 1.8.2....
http://www.magiclantern.fm/forum/index.php?topic=10198.0

In this test the output dng file is clean, but the tiff proxy file from the exact same conversion shows the same dead/cold/pixels/Sensels but ONLY in the proxy tiff files, not the dng file output.  They are also red and not black.  Any ideas on what is happening?  Very odd indeed.

DNG output...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fsame_dng.jpg&hash=a8db623f1ffc7123c40a3ee512ac9020)

TIFF output...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fsame2_tif.jpg&hash=5bde3f51bc9a0e3fdc755ff657dffb38)
Title: Re: The CinemaDNG Discussion
Post by: andy kh on February 12, 2014, 05:34:44 PM
i dont have such problems
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 12, 2014, 05:37:11 PM
[caution speculation] i'd say, because the dng's are made by adobe, there is a badpixel-tag. i dont know what it does, but it seems havin a positive effect. the same i'd say from the cr2-files straight out the body, where canon sets this tag. duey, please send me a cr2-file from your cam, just for testing purposes..

first i will optimize the actual things ->mlv with splitfiles<-, then i'd take the time for the bad-pixels opcode.



thx & regards chmee
Title: Re: The CinemaDNG Discussion
Post by: tonybeccar on February 12, 2014, 05:48:02 PM
Hey chmee, one question. I was looking into the awesome workflow of cdngs in premiere.. and the people that have the bmcc import their sequences in premiere, just like us, BUT, in the project panel, when they right click on a cdng sequence and they go to Source Settings, it opens in ACR and they can tweak things in there, close it and it updates on Premiere. Dreamy! Now.. I cannot seem to do it with your cdngs, the option is greyed out (though I haven't tried with others anyway), but I have everything up to date, premiere, phosohop, adobe camera raw (8.1 though, couldnt find the 8.2). Do you have this feature? I saw it and it is truly amazing...

PS: Please command line command line! :D I was thinking, with your app, then this is the way to go.. so when you have the command line ready i'll focus more on the interface than anything else.. drop the common dngs, ffmpeg proxies, they are no use really, slow and low quality and tedious.. but instead just make a neat interface that interlaces with your magic software.. I can put my efforts on that...

Anyway, does the premiere feature work for you?
Cheers!!!
Title: Re: The CinemaDNG Discussion
Post by: a1ex on February 12, 2014, 05:50:29 PM
it seems havin a positive effect

Yep, that's why I'm going to remove it from raw2dng ;)

In theory, the current solution (tagging the bad pixel in DNG files) is more elegant. In practice, only ACR reads these tags.
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 12, 2014, 06:12:31 PM
@tony could you give me a link to bmcc-files, where this sequence-settings are available? i've got some here, but its greyed out. this is a great finding, and would make a huge step for us.

[commandline] please (all) give me the time to primarly reach the magic milestone, raw/mlv/audio straight for premiere.  then i'd recode all into a commandline.

@a1ex lOl. ok, i should examine your way with the banding correction and the stucked-sensel correction.
Title: Re: The CinemaDNG Discussion
Post by: tonybeccar on February 12, 2014, 07:10:10 PM
@tony could you give me a link to bmcc-files, where this sequence-seetings are available? i've got some here, but its greyed out. this is a great finding, and would make a huge step for us.

[commandline] please (all) give me the time to primarly reach a first finish line, the magic milestone..  then i'd recode all into a commandline.

@a1ex lOl. ok, i should examine your way with the banding correction and the stucked-sensel correction.

Here I saw it in a video: I found sample dngs of the pocket camera here (my god how good it is!): https://www.dropbox.com/sh/44bnza1ooi34xqb/6KlnDHKWpy I wonder if they are useful.. gonna test them myself now! But yeah, that workflow is HEAVEN.

EDIT: http://philipbloom.net/2012/08/23/grading-the-blackmagic-cinema-camera-dngs-in-photoshop/
There you have BMCC sequences. For some reason I couldnt import the BMPCC single frames into Premiere.

Have you tested doing the post color in Speedgrade?

And please! Of course, first the MLV support! :D
Thanks so much and cheers mate!
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 12, 2014, 07:16:48 PM
ah. thats a video with the beta-cdng-importer from cs6 - this was crushed & buried long ago. i was wondering why/how the gpu-powered cdng-importer in CC actually should support ACR.. nope.
Title: Re: The CinemaDNG Discussion
Post by: tonybeccar on February 12, 2014, 07:21:49 PM
ah. thats a video with the beta-cdng-importer from cs6 - this one was crushed long ago. i was wondering why/how the gpu-powered cdng-importer in CC actually should use ACR.. nope.

I know it's outdated.. but you think they removed that feature?? Is there a reason? It was a good feature.. :/
I just downloaded the BMCC shots and it's still the same, grayed out. I do not have ACR 8.2 though, only 8.1, I can try and search for it...

Also, I notice that the dngs from the BMCC play faster in premiere.. and also when I play them in 1/4 or 1/2, they don't get that "greenish" look they have with your cdngs.. On full quality they look ok, but as soon as you hit play and quality isn't full the colors shift...

Will research more on that one.. to see if it's possible.
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 12, 2014, 07:44:09 PM
when they right click on a cdng sequence and they go to Source Settings, it opens in ACR and they can tweak things in there, close it and it updates on Premiere. Dreamy! Now.. I cannot seem to do it with your cdngs, the option is greyed out

The same for me, this option is greyed out.  You can see in this R3D tutorial http://vimeo.com/52037147 (http://vimeo.com/52037147) that he uses this option of source settings in PPro. 

I don't get why the DNG's came out without bad pixels but the TIFF's did?  I just ran this program http://www.starzen.com/imaging/deadpixeltest.htm (http://www.starzen.com/imaging/deadpixeltest.htm) on RAW files I took with the lens cap on, VF covered, ISO 100, and 1,2,5,15 second shots.  Converted those to uncompressed TIFF's in LR5.  Then tested those TIFF's with that Dead Pixel Test program at the default threshold setting of 60 and then halved that to 30.  No issues,  0 dead pixels / 0 hot pixels.

Anyone know a better way to test for dead pixels?  I don't see any in any images I shoot in RAW/JPG.

CR2's sent. 
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 12, 2014, 08:10:13 PM
When this gets sorted out it will be amazing!  Working with what I have right now from raw2cdng is very nice in PPro!  Glitch free playback and scrubbing, very smooth on my Toshiba with 7200 rpm HD, 8 gigs ram, GEForce 540M GPU (which I tricked PPro into thinking it's a CUDA card).  Very good work flow and fast.

Thanks to chmee and tonybeccar!   :D
Title: Re: The CinemaDNG Discussion
Post by: tonybeccar on February 12, 2014, 08:36:00 PM
When this gets sorted out it will be amazing!  Working with what I have right now from raw2cdng is very nice in PPro!  Glitch free playback and scrubbing, very smooth on my Toshiba with 7200 rpm HD, 8 gigs ram, GEForce 540M GPU (which I tricked PPro into thinking it's a CUDA card).  Very good work flow and fast.

Thanks to chmee and tonybeccar!   :D

Lol... please, thank chmee ;)
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 13, 2014, 05:09:18 AM
Truthfully, I thank every individual on this forum.  :D
Title: Re: The CinemaDNG Discussion
Post by: Malex on February 13, 2014, 12:03:54 PM
Hi there, I don't know if I understood correctly but on the magic lantern facebook page it says :
"we've fix pink frame on the 5DIII and 60D"
Does it mean that RAW files coming from the 5DIII won't need the "partymode" anymore ?
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 13, 2014, 01:32:46 PM
I dont think so. I assume, its about erroneous files and/or frames, that can be restored..
http://www.magiclantern.fm/forum/index.php?topic=8237.0

for disabling the partymode, we have to debias the blacklevel and expand the values to the full-level of destinated bitdepth. means blacklevel = 0 whitelevel = (2^bitdepth-1). i doubt, its possible inside the body.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 13, 2014, 01:44:25 PM
In the case with my 60D I think these corrupted pink frames as I posted earlier in the thread come from too slow of read/write speed for the camera itself so the frames get corrupted.  This happens on 3-10 takes so far for me on 1280x720 and my max frames are 400-415 so far in MLV mode and up to 15 seconds in RAW mode, and that is probably the best the 60D can do.  I'm going to have to save and get a 5D III!

 
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 13, 2014, 01:47:24 PM
I just discovered my sensor was filthy!  So I gave it a good cleaning with Pec pads and Eclipse so now it is clean, I'll try to shoot some more RAW and see if that had anything to do with the results I got.

chmee: you get my RAW files?
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 13, 2014, 02:02:44 PM
yes, got it. but i'm now in geneva for work. will code here on raw2cdng as well, but hadnt the time to inspect your cr2's..sry
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 14, 2014, 07:00:48 AM
A couple of more questions...  I cleaned my sensor, reshot RAW/MLV then took out my ML card and loaded a normal SD card (Sandisk 32gig 45mb/s') and then shot some plain old H.264 clips.  I loaded the new RAW/MLV clips into the timeline in PPro and marked the offending pixels locations with yellow circles (zoomed in 400X).  In all RAW/MLV files they appear in the same location.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fevilpix1.jpg&hash=91dad26b80a70dd234f9f6788253553e)


(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fevilpix3.jpg&hash=e13781b93311385f028bda2d30710907)

If I load the same SOURCE MLV/RAW files into PPro with Drastic Media Reactor's plugin I get the same pattern but the pixels look entirely different (pluses and colored squares).

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fevilpixdmr.jpg&hash=63c106105e9e84eeea128a9fac578be4)

The same pattern appears in any RAW/MLV conversion or source MLV/RAW file dropped into PPro via CDNG conversion or DMR's plugin. 

Now the part I don't understand.  Next, I loaded the regular old Canon video H.264 clip and pixel peeped it at 400X also and I see no funky pixels at all, anywhere!  Neither in the marked positions that the MLV/RAW files show them or anywhere else in the H.264 clips.  My question is, if my camera has bad/dead/hot/other messed up pixels then why don't they show up in H.264 footage? 

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fevilpix2.jpg&hash=f948293afb047aee490a7cf779c09f8a)





Title: Re: The CinemaDNG Discussion
Post by: Malex on February 14, 2014, 08:06:18 AM
I don't think it is really sensor problem even though I don't know anything about it, but in my case I got those pixel bug on many of my MLV with sound footage I had to use ACR to make them got away and convert my dng in AE (but it didn't work out with all my footage).
The point is I never had those bad pixel with Raw_rec, so I'm sticking with it for now.
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 14, 2014, 09:50:16 AM
Thanks Malex.  I get them with both, MLV and RAW.
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 14, 2014, 06:14:20 PM
Okay next test, I used raw2dng_escho.exe and got clean dng files (not cdng).  As those won't open in PPro, they do open in After Effects so I loaded them up in AE.  No funky pixels.  My camera is fine I now understand.  Keep on going, testing, winning!
Title: Re: The CinemaDNG Discussion
Post by: arturochu on February 14, 2014, 07:48:30 PM
Okay next test, I used raw2dng_escho.exe and got clean dng files (not cdng).  As those won't open in PPro, they do open in After Effects so I loaded them up in AE.  No funky pixels.  My camera is fine I now understand.  Keep on going, testing, winning!

raw2dng_escho is my go to option at the moment too.
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 14, 2014, 09:10:24 PM
thanks for the hint. here's the thread to the problem: http://www.magiclantern.fm/forum/index.php?topic=9700.0

 i put it on my todo-list.
Title: Re: The CinemaDNG Discussion
Post by: Wartburg-Entertainment on February 15, 2014, 06:53:45 PM
Hello people,

I have no problem with RAW2CDNG, as it is a really great app.
But I have still the problem with the pink highlights in Premiere CC 7.2.1 :-(
And strangely, only in Premiere! In After Effects, and in the ACR Pink higlights are not seen.
What must adjust in Premiere?
It does not matter what I set in RAW2CDNG, only In Premiere I have the Pink Horror. :-(

greeting
Title: Re: The CinemaDNG Discussion
Post by: andy kh on February 16, 2014, 05:08:00 AM
Hello people,

I have no problem with RAW2CDNG, as it is a really great app.
But I have still the problem with the pink highlights in Premiere CC 7.2.1 :-(
And strangely, only in Premiere! In After Effects, and in the ACR Pink higlights are not seen.
What must adjust in Premiere?
It does not matter what I set in RAW2CDNG, only In Premiere I have the Pink Horror. :-(

greeting
its not premiere problem. magic lantern problem. alex have been working hard to fix  it. he succesfully fix for 5d mark iii. im sory i got u now. make sure you check the partymode to get rid of it.
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 16, 2014, 06:21:47 AM
@Wartburg-Entertainment

What camera, RAW2CDNG settings are you using?  (It helps to put all your specs in your user profile signature).  I had that same issue in 1.4.8.  In 1.4.9 I do not.  Latest version is... http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.zip (http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.zip)

My settings are like this...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fraw2cdng_window.jpg&hash=d3e8bb84c211482acb425c1bc6e325f1)

I get no pinkish output but still the hot/sensels problem exists.
Title: Re: The CinemaDNG Discussion
Post by: Wartburg-Entertainment on February 16, 2014, 07:44:29 AM
Hi guys,

I use on my 7d currently still Tragiclantern because MagicLantern is not stable.
And I have version 1:49 of RAW2CDNG.
And believe me, I've got all the settings through 10,12, 16bit - with and without Tiff Model.
Only Premiere shows this pink highlights!!!

Title: Re: The CinemaDNG Discussion
Post by: andy kh on February 16, 2014, 07:46:40 AM
Hi guys,

I use on my 7d currently still Tragiclantern because MagicLantern is not stable.
And I have version 1:49 of RAW2CDNG.
And believe me, I've got all the settings through 10,12, 16bit - with and without Tiff Model.
Only Premiere shows this pink highlights!!!



as i said make sure you check  partymode
Title: Re: The CinemaDNG Discussion
Post by: Wartburg-Entertainment on February 16, 2014, 08:13:20 AM
CDNG's Without Party Mode? Really?

Edit:no Partymode - brings no Improvement.....
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 16, 2014, 11:02:46 AM
Make sure you have Gamma at 1.0, check the box for partymode and then try moving any files you want to encode into the same folder as raw2cdng, I think that is what I did, might be worth a try.  Also use the latest nightly build.  I was having that same problem and I checked/unchecked partymode and tried 10,12,16 bit and all that but I think it was when I set the Gamma to 1.0 is when it fixed it.  Make sure about the Gamma setting.
Title: Re: The CinemaDNG Discussion
Post by: Wartburg-Entertainment on February 16, 2014, 11:34:46 AM
I've always gamma 1 and level the problem only in Premiere.
Even in the media browser, the small preview files are all pink.
Will now test the build of Magic Lantern nighly, let's see
what it is so new insights there.

regards
Title: Re: The CinemaDNG Discussion
Post by: gideonplus on February 16, 2014, 12:04:45 PM
for some reason I can not make it work by dropping MLV file even from another partition/disk .
I can see the Command line window for a moment but nothing happens.
What am I doing wrong ?
Thanks
Title: Re: The CinemaDNG Discussion
Post by: Vsevolod on February 16, 2014, 01:12:03 PM
I have the same problem. Windows 8.1
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 16, 2014, 02:23:00 PM
The contents of your folder should look like this...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fraw2cdngfolder.jpg&hash=548647d92e79e4e3be4ba221c9d91dbd)

I named my folder raw2CDNG, and it looks as above.  I dragged one MLV and one RAW file in there and it started working from then.  You need to have mlv_dump.exe (http://upload.g3gg0.de/pub_files/fb196456f6a6389763cab895fb6067f8/mlv_dump.exe (http://upload.g3gg0.de/pub_files/fb196456f6a6389763cab895fb6067f8/mlv_dump.exe)) in the same folder otherwise you'll just get that brief command window popup.  In my testing you need to have everything all in the same folder including the MLV/RAW files...

Set your settings (as in the previous picture, then with all the contents of your folder looking as above, drag the MLV file onto the raw2cdng window, it should pop open the command window and look like this...

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.dueymoore.com%2Fprocessingraw2cdng.jpg&hash=a90f9fbd74c5cf77b13d4a5175027ef2)

See if that helps.
Title: Re: The CinemaDNG Discussion
Post by: gideonplus on February 16, 2014, 03:48:38 PM
no success
and it doesnt even show the file name in the Batch list
(the Op system is Win 7 64bit)
Title: Re: The CinemaDNG Discussion
Post by: andy kh on February 16, 2014, 04:52:38 PM
no success
and it doesnt even show the file name in the Batch list
(the Op system is Win 7 64bit)

copy and paste the mlv file to another drive and try drag and drop from that drive... you dont have to copy everything. copy only the mlv file
Title: Re: The CinemaDNG Discussion
Post by: gideonplus on February 16, 2014, 05:15:47 PM
I did try that and it doesnt work
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 16, 2014, 05:41:22 PM
Okay, well it certainly works and I am on the same OS (Windows 7 Ultimate x64 SP1).  Here is a video screen capture to show usage...


So just keep slugging away and you'll get it to work.  Your missing something somewhere but I don't know where.  Can you drop a .RAW file into the GUI?  Are the Output path and Tif Model tag check boxes checked?  Look at the video and duplicate what you see and it should work as you can see it does. 

1. Download raw2cdng 1.4.9.zip  http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.zip (http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.zip)
2. Extract it into a folder called raw2CDNG.  You will get 2 files in the zip, mlv_dump.exe and raw2cdng.1.4.9.exe.
3. Copy one MLV and one RAW file into the same directory as the installed files.
4. Double click on the raw2cdng.exe file, the GUI opens.
5. Drag your MLV file from the same folder as where the program is onto the open raw2cdng GUI (program window).
6. The mlv_dump.exe command box should open and start extracting the MLV and then when it is finished you will see the RAW file in the GUI.  Then all you need to do is export it where you want to.

I can drag and drop files from the desktop, SD card, C drive, D drive, it all works.

Title: Re: The CinemaDNG Discussion
Post by: gideonplus on February 16, 2014, 05:46:16 PM
Yes it works OK only with the RAW file
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 16, 2014, 06:08:06 PM
Watch the video.   ;)
Title: Re: The CinemaDNG Discussion
Post by: Vsevolod on February 16, 2014, 09:33:51 PM
Works on a different disk. Sorry.)
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 16, 2014, 10:22:34 PM
please describe it with some details, so i can fix it. on my computer i tested from different places, and it worked all the time, no matter if desktop, root or some other directories.

* filesystem (ntfs? fat32? static or dynamic hard drive/partition?)
* language?
* os?
* whole directory-chain?

if its not too intimate :)
Title: Re: The CinemaDNG Discussion
Post by: gideonplus on February 16, 2014, 11:13:42 PM
OK its Win 7 64bit ntfs static same/different drives

* language English US
Command line window just for amoment and no change on UI

example Log file:
14.02.16 19:34:29 --  ----------- Application started ------------
14.02.16 19:34:29 -- - gamma loaded and set: 1
14.02.16 19:34:29 -- outputPath: E:\Gideons Pictures\2013_12_31
14.02.16 19:34:29 -- uniqueName: ML CDNG1.4.6 PHR
14.02.16 19:34:29 -- loading LogLists for 12bit
14.02.16 19:34:29 -- setting arrays for 12bit-Loglists
14.02.16 19:34:29 -- found and enabled mlv_dump
14.02.16 19:34:29 -- loaded GUI 1.4.9
14.02.16 19:35:20 -- **-- filesList_DragDrop started
14.02.16 19:35:20 -- * dragdropped File E:\Gideons Pictures\2013_12_31\M15-0911.MLV
14.02.16 19:35:20 -- is mlv-File and will be converted E:\Gideons Pictures\2013_12_31\M15-0911.MLV
14.02.16 19:35:20 -- converted to raw
14.02.16 19:35:20 -- is raw file
14.02.16 19:35:20 -- getting RAW-Attributes
14.02.16 19:35:49 -- **-- filesList_DragDrop started
14.02.16 19:35:49 -- * dragdropped File E:\Gideons Pictures\2013_12_31\M15-0918.MLV
14.02.16 19:35:49 -- is mlv-File and will be converted E:\Gideons Pictures\2013_12_31\M15-0918.MLV
14.02.16 19:35:49 -- converted to raw
14.02.16 19:35:49 -- is raw file
14.02.16 19:35:49 -- getting RAW-Attributes

Finaly NOTHING is done in output

Title: Re: The CinemaDNG Discussion
Post by: chmee on February 16, 2014, 11:25:41 PM
..mainly "file not found error" is the problem, caused by white spaces (or other "not so fine" characters) in directory- or filename.

* small fix - raw2cdng 1.4.9.2
- mlv-convert-process-output written into debuglog
- put filenames into quotationmarks in commandline-process
http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.2.zip

i assume the space in
E:\Gideons Pictures\2013_12_31\M15-0918.MLV
will be the problem.
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 17, 2014, 06:01:29 AM
@chmee  I have been doing lots of testing and also testing for pink frames on my 60D and now it seems to have been fixed.  Did some more experimenting with conversions.  Still getting the odd pixels/sensels in raw2cdng but if I convert with raw2dng_escho.exe the files are perfect.  Here is a raw2dng_escho file for your reference (Canon 60 D / MZ 5X mode / mlv_rec.mo @ 1280x720)

http://www.dueymoore.com/000000.dng  (right click/click save link as)
Title: Re: The CinemaDNG Discussion
Post by: gideonplus on February 17, 2014, 12:13:54 PM
OK you have nailed the problem (the fix doent help it by the way but I have renamed the Dirctories)
some new thing:
in Premiere my futage the was 25fps was showing like it is 24fps
so I have checked the DMG metha data and in the FrameRate Field
it was 24fps which is wrong and should be 25fps

Another question that I have is:
Does in Resolve 10, the dng-sequence and audio file will be recognised automatically and in synch , As Well ?

Thanks
Title: Re: The CinemaDNG Discussion
Post by: andy kh on February 17, 2014, 02:00:46 PM
OK you have nailed the problem (the fix doent help it by the way but I have renamed the Dirctories)
some new thing:
in Premiere my futage the was 25fps was showing like it is 24fps
so I have checked the DMG metha data and in the FrameRate Field
it was 24fps which is wrong and should be 25fps

Another question that I have is:
Does in Resolve 10, the dng-sequence and audio file will be recognised automatically and in synch , As Well ?

Thanks

no it wont. only c2dng syncs that too only in premiere pro. c2dng is  the best and i always use in all my projects since u can cut directly in premiere pro.its even better now by syncing audio which makes my workflow much easier and fast
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 17, 2014, 08:38:02 PM
@gideonplus thats correct, i nailed it to static 24fps, because there've been more important things to solve - and changing framerate is the less difficult problem in fi premiere.. its the same with timecode - i could use "timeofday" but by now i start simply at 01:00:00:00

by the way: could you show a debuglog with mlv_dump-error with version 1.4.9.2? i thought, this could solve the problem :)

regards chmee

Title: Re: The CinemaDNG Discussion
Post by: Wartburg-Entertainment on February 17, 2014, 09:32:57 PM
@ chmee

Can you please help me with my Premiere CC CDNG problem.
So the MLV files all come from a 7D with last Tragic Lantern.
I can open the DNG in Premiere, but the settings of ACR are not included in Premiere.
And now the main problem, as already described here I have only completely pink in the highlights and in the over-exposure.
Today I again several MLV transformed with the new 1.4.9.2. - No improvement. Is it Windows 8.1 or Tragic Lantern or or ...?
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on February 18, 2014, 12:25:17 AM
There no ACR support in premiere pro cc right now, dynamic link to A.E. or speedgrade
is still there , but time consuming. Better to use A.E. first save the project xml import that
by dynamic link in to Premiere Pro CC .  :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 18, 2014, 01:50:34 AM
@wartburg-entertainmant
i feel a liitle bit sorry.. 7D is another workplace (i'm using and testing 5DIII), and as you stated, tragic-lantern-mod is much older, so you cant get all benefits from actual versions, it can hit you right there.. simply, i cant help you.

@a1ex you stated in another thread i'm neither giving source open (via fi github) nor i respect gpl-copyrights.  i worked on that, solved that and gave all informations, to code fi premiere support more than on time. finally, its a free software. that's not really fair..
Title: Re: The CinemaDNG Discussion
Post by: Wartburg-Entertainment on February 18, 2014, 06:19:22 AM
@chmee: Thank you anyway, even if it is not good news.

@reddeercity: Thanks for the idea. I also think that it is currently the best workflow.
Title: Re: The CinemaDNG Discussion
Post by: a1ex on February 18, 2014, 08:45:13 AM
@chmee: it wasn't directed at you, it was a doubt about all the other closed-source apps from this forum that do (more or less) the same thing as raw2dng.

This is an open source community, we share the knowledge with everyone else, so I think it's fair that everybody who uses our software and expands upon it should share their knowledge instead of hiding it in binary blobs.

I understand you are not using any of my code, so there are no GPL issues. But what about the fundamental aspect of open source, collaboration?

The part about raw2cdng was that a user asked me to fix a bad pixel problem in your program, I told him your program does not appear to be open source, so there's nothing I can do.

Sorry for the misunderstanding.

Quote
finally, its a free software
Since we do not have the freedom to see your source code, to study it or to reuse bits from it, your program does not count as free software (http://www.gnu.org/philosophy/free-sw.html), sorry. And the same is true about all the other closed-source programs (freeware != free software).
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 18, 2014, 09:17:47 AM
@a1ex ok. understandin' - so let me reorganize this codechaos and then..
Title: Re: The CinemaDNG Discussion
Post by: a1ex on February 18, 2014, 09:21:22 AM
Thanks, no need to hurry. That's also a practical aspect of open source - since you upload it in the open, you tend to be a bit more careful about code quality, readability and so on (so it's a good idea if you are trying to sharpen your skills).
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 19, 2014, 12:41:15 AM
raw2cdng 1.4.9.3 online
* second attempt fixing this directory-filenotfound-thing
* modeltag and uniquemodel are separated (@Andy600 told about problems in cinelog (http://www.magiclantern.fm/forum/index.php?topic=10151.0))
* optimizing some things (fi tiff-dng-tag-creation)

http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.3.zip

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: peterbrne on February 20, 2014, 02:06:03 AM
Hi,

Thanks chmee for great work....

Im totaly not familiar with how to make apps,..

Just a question ... in future ... will it be available for OSX also?

Thanks,

Peter
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 20, 2014, 02:51:01 AM
@peterbrne
actually not from my side, i m Mr ignoramus in terms of mac-world:) so, just hope for the day, i will opensource the code and others will use that for cross-compilations. :)
Title: Re: The CinemaDNG Discussion
Post by: anandkamal on February 22, 2014, 06:20:40 PM
hi.. may i please have the link/instructions of the latest tutorial on resolve premiere pro roundtrip? or the new premiere pro cdng to resolve workflow? thanks..
Title: Re: The CinemaDNG Discussion
Post by: andy kh on February 22, 2014, 07:11:25 PM
there are so many ways to do that. import cdng in premiere pro.cut n export xml. import xml in resolve. do your grading and export xml for premiere pro and now you can add titles, effects etc
Title: Re: The CinemaDNG Discussion
Post by: anandkamal on February 22, 2014, 07:53:50 PM
hey thanks.. but xml could not be imported in resolve "the file M17-1805.000000.dng" can't be found" pops up... i should do some renaming. Please explain that stuff so that resolve can import the xml from ppro...
Title: Re: The CinemaDNG Discussion
Post by: andy kh on February 22, 2014, 08:20:11 PM
no. you dont have to rename it. open your project. click go to media page and click "davinci resolve" on top right. click preference. click "+" and add your folder where your cdng is located. there are so many basic tutorials on youtube
Title: Re: The CinemaDNG Discussion
Post by: anandkamal on February 23, 2014, 08:51:27 AM
has any faced problem with the following workflow? CDNG to resolve and edit all the footage in resolve itself, color correct and export it to premiere pro (with timeline xml) for perfect trim, transition effects and do mastering from premiere..
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 24, 2014, 12:02:20 AM
since the beginning this was one of the favoured workflows.

* first we manipulated canon-raw to 16bit to work in resolve9
* we found the right switches for xml-roundtrip
* then resolve 9.15 came with native support of 14bit-dng's

the drawback was and is the complete import and recode of the footage, just to work with it in other software. time and space-consuming.
Title: Re: The CinemaDNG Discussion
Post by: kaco on February 25, 2014, 07:45:58 PM
There no ACR support in premiere pro cc right now, dynamic link to A.E. or speedgrade
is still there , but time consuming. Better to use A.E. first save the project xml import that
by dynamic link in to Premiere Pro CC .  :)

Could you please explain your workflow a little more?
Title: Re: The CinemaDNG Discussion
Post by: reddeercity on February 25, 2014, 09:31:10 PM
Could you please explain your workflow a little more?
Sure no problem its very simple, open Cdng's and  or dng's image sequence
in After effects ,now you have  access to the ACR panel  save your project as cs6 xml
import A.E. Project in to your premiere pro project.
I posted a short video tutorial , link below
http://www.magiclantern.fm/forum/index.php?topic=8177.msg86395#msg86395
Hope this helps  :)
Title: Re: The CinemaDNG Discussion
Post by: kaco on February 25, 2014, 10:09:14 PM
Sure no problem its very simple, open Cdng's and  or dng's image sequence
in After effects ,now you have  access to the ACR panel  save your project as cs6 xml
import A.E. Project in to your premiere pro project.
I posted a short video tutorial , link below
http://www.magiclantern.fm/forum/index.php?topic=8177.msg86395#msg86395
Hope this helps  :)

big thanks
Title: Re: The CinemaDNG Discussion
Post by: duey101 on February 28, 2014, 06:35:28 AM
@chmee

Question: Is the mlv2cdng app going to work in the same way as the raw2cdng app?  What I mean is will we still have to convert from MLV>RAW or will it be drop MLV files straight into the GUI and convert to CDNG.  The one thing I like about raw2cdng is the SPEED!  It is blazing fast on my system.
Title: Re: The CinemaDNG Discussion
Post by: chmee on February 28, 2014, 05:41:42 PM
If i implement mlv, it will be native.. And also the speed should stay equal. I'm thinking about a processing module, for cs2x2 for example, but it would lower the Speed massively
Title: Re: The CinemaDNG Discussion
Post by: zcream on March 07, 2014, 12:27:39 PM
Would you consider posting the LUT or the formula used to convert 12-bit linear RAW to 10-bit log ?
I am still trying to figure this out.

Cheers!.
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 07, 2014, 12:47:35 PM
@zcream
i convert from 14bit to 10bit/log. you find the discussion about lin/log in the early posts inside this thread, IliasG helped me out.
http://www.magiclantern.fm/forum/index.php?topic=5618.msg47480#msg47480

beside that, try it with a simple lin/log-conversion as described in a lot of math/physics-forums..
http://www.physicsforums.com/showthread.php?t=304633

here are the old calculations by IliasG
14lin10log_gamma_rec709.zip (http://www.file-upload.net/download-8689468/14lin10log_gamma_rec709.zip.html)


@all
mlv/audio native is on the way.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: arrinkiiii on March 07, 2014, 12:52:29 PM
@all
mlv/audio native is on the way.

Sounds good =))
Title: Re: The CinemaDNG Discussion
Post by: andy kh on March 07, 2014, 01:11:54 PM

@all
mlv/audio native is on the way.

that wil be great. for me i stil wil have to convert to raw first to remove pink dots since i use 650D now.
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 07, 2014, 02:03:59 PM
the pink dot removing costs a lot of speed - but i still think about a optional button for that.
Title: Re: The CinemaDNG Discussion
Post by: andy kh on March 07, 2014, 02:22:47 PM
the pink dot removing costs a lot of speed - but i still think about a optional button for that.

oh wow!! that wil be fantastic if there is optiional button for removing pink dot. no other application wil be needed  in the workflow if you do so. drag and drop all mlv files in raw2cdg. remove pink dots and convert to cdng. this wil be the best application for sure
Title: Re: The CinemaDNG Discussion
Post by: duey101 on March 08, 2014, 07:38:35 AM
@all
mlv/audio native is on the way.

This look promising, +1 for the pink dot button.
Title: Re: The CinemaDNG Discussion
Post by: CrW on March 08, 2014, 10:22:44 PM
Just a little bug report (sorry if this has already been posted):
On Windows (8, 64-bit), the drag-and-drop functionality breaks if the path of the dragged file contains spaces (mlv_dump gives error).
E.g.:
C:/this/path/works/video.mlv
C:/this/path/does not/work/video.mlv

In the latter case, mlv_dump would say that "C:/this/path/does" does not exist.
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 09, 2014, 12:14:36 AM
@CrW What Version you're using? Please try 1.4.9.3 and tell if its still apparent..
http://www.magiclantern.fm/forum/index.php?topic=5618.msg102531#msg102531
Title: Re: The CinemaDNG Discussion
Post by: kaco on March 10, 2014, 12:18:28 AM
@CrW What Version you're using? Please try 1.4.9.3 and tell if its still apparent..
http://www.magiclantern.fm/forum/index.php?topic=5618.msg102531#msg102531

I was about to report same bug. With 1.4.9.3 it works for me.
Title: Re: The CinemaDNG Discussion
Post by: kaco on March 10, 2014, 12:30:11 AM
@all
mlv/audio native is on the way.

When developing native wav extract, could you please test importing the final CDNG into Davinci Resolve? For me the DNG sequence and the WAV file is always being imported as two separate files. :-/
Title: Re: The CinemaDNG Discussion
Post by: AD DP on March 10, 2014, 05:39:25 AM
@CrW What Version you're using? Please try 1.4.9.3 and tell if its still apparent..
http://www.magiclantern.fm/forum/index.php?topic=5618.msg102531#msg102531

Using 64bit win 8.1 here, no response when attempting to drag and drop a MLV file on version 1.4.9.3 of the converter, am I missing something?  :'(

Edit:
I can convert from MLV to RAW first then use raw2cdng and it works - sometimes (all crash logs saved) - but there seems to be a huge loss in quality when importing to premiere for editing, compared to the other option of converting MLV's to DNG's with Mystic then importing to After Effects, Any ideas? ;/
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 10, 2014, 09:18:00 AM
@AD DP
The difference is the DNG-Wrapper. Premiere uses another code than AFX. Because i dont have Win8.1 i cant check, whats wrong there. Please wait some more days for native MLV-Support, then give it another try.
Title: Re: The CinemaDNG Discussion
Post by: kaco on March 10, 2014, 10:41:10 AM
Using 64bit win 8.1 here, no response when attempting to drag and drop a MLV file on version 1.4.9.3 of the converter, am I missing something?  :'(

Edit:
I can convert from MLV to RAW first then use raw2cdng and it works - sometimes (all crash logs saved) - but there seems to be a huge loss in quality when importing to premiere for editing, compared to the other option of converting MLV's to DNG's with Mystic then importing to After Effects, Any ideas? ;/

I'm also on Win 8.1
The 1.4.9. version was not working with spaces in directories. With 1.4.9.3 it works for me (although the windows with mlv_dump will stay black during the extraction process).
Title: Re: The CinemaDNG Discussion
Post by: arrinkiiii on March 10, 2014, 10:57:11 AM
Please what some more days for native MLV-Support...

Wonderful =))
Title: Re: The CinemaDNG Discussion
Post by: AD DP on March 10, 2014, 07:57:52 PM
@AD DP
The difference is the DNG-Wrapper. Premiere uses another code than AFX. Because i dont have Win8.1 i cant check, whats wrong there. Please wait some more days for native MLV-Support, then give it another try.

@chmee

That's Great! I did have success converting to RAW and then using the program to make CDNG's, just not with the MLV's. Cant wait for the updates! Thank you for all your hard work. :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 12, 2014, 11:25:57 PM
raw2cdng v1.4.9.5 is online
* native mlv+audio

http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

PLEASE! i wont believe, that all is working fine. so, test and tell all bugs, glitches, problems. my wish is to get a stable 1.5 version..
Title: Re: The CinemaDNG Discussion
Post by: melimelo on March 12, 2014, 11:30:58 PM
I'm a total newbie so my questions may be quite simple but have a few doubts about the raw2cdng workflow and quality implications:

1. To edit directly in PPro (with no pink cast) I just extract the cdngs setting bitdepth at 12/linear as I hve read in chmee's blog. Everything goes smoothly but my question is: is there any quality loss vs extracting at 16 bit and then grading in resolve and sending back to premiere via xml for edit and back to resolve for export (or using the 16 bit extract at ACR)? (I'm not a pro in any way, so the quality I get with the 12/linear export in PPro is more than enough for me...but just wanted to know...)

2. When I open the CDNG sequence in premiere for color grading I use the speedgrade direct link but when I've triedto color in resolve (via xml from PPro) I get the problem that the CDNGs have embeded a 24fps frame rate. In premiere I just interpret footage as whatever it actually is (i.e. 25fps) and sequence is adjusted accordingly. But if I export that 25fps sequence as xml, Resolve just does not import the clips as it says the clips do not match the sequence timecode (I guess the dngs have a fixed 24 fps timecode embedded). ANything I'm doing wrong? If not, how can I change that "internal" fixed 24 fps setting?

Thanks
Title: Re: The CinemaDNG Discussion
Post by: Jakobmen on March 12, 2014, 11:31:22 PM
raw2cdng v1.4.9.5 is online
* native mlv+audio

http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

PLEASE! i wont believe, that all is working fine. so, test and tell all bugs, glitches, problems. my wish is to get a stable 1.5 version..

link to 12.03.2014 – v1.4.9.5 – native mlv-support is downloading v1.4.9.3 first glitch..  :P 
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 12, 2014, 11:33:10 PM
just ve seen it :) fixed.

@melimelo
(1) i didnt recognised any real disadvantages on 12bit versus 16bit. primarily i left the 16bit because of acr and resolve-users. and some saying, its a disadvantage by numbers :)
(2) will look deeper on that.
Title: Re: The CinemaDNG Discussion
Post by: melimelo on March 13, 2014, 12:23:52 AM
just ve seen it :) fixed.

@melimelo
(1) i didnt recognised any disadvantages on 12bit versus 16bit. primarily i left the 16bit because of acr and resolve-users. and some, saying, its a disadvantage by numbers :)
(2) will look deeper on that.

Thanks for your extremely quick reply...have downloaded the new version, I guess this native feature must be a big improvement (as I've read you've been working on this for quite long) but could you elaborate for "illiterates" like me what are the main improvements/advantages of this version of this vs previous versions? Thanks!

PS: the only thing I may add is that in the new version the default tif model tage still says version 1.4.9.3...not much help but that is the most I can contribute ;-)
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 13, 2014, 12:43:11 AM
:) main advantage is the native mlv+audio support. before that i had native raw-support and mlv+audio was done thru chaining to external app (mlv_dump). it was a double copy/recode-process. i just wanted to do an app, you're able to transcode straight from the card, without copying and merging splitted files, with batch support (dropping more than one raw/mlv-file) - support in premiere, because thats my favoured edit-app - and finally something with a kind of speed. i dont like the timeconsuming process with acr/photoshop/lightroom/aftereffects, nonetheless its still the way to get the best quality.

Quote
still says version 1.4.9.3...not much help but that is the most I can contribute
thx.
Title: Re: The CinemaDNG Discussion
Post by: Jakobmen on March 13, 2014, 04:14:00 AM
so is there a option for 23.976? or its just 24.000fps
Title: Re: The CinemaDNG Discussion
Post by: AD DP on March 13, 2014, 06:23:05 AM
raw2cdng v1.4.9.5 is online
* native mlv+audio

http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

PLEASE! i wont believe, that all is working fine. so, test and tell all bugs, glitches, problems. my wish is to get a stable 1.5 version..


@chmee

I think there is a loss in quality when going the CDNG route, but it may also be my inexperience with utilizing speed grade and not really being able to pull all the detail back out. Here are the results I was able to get using the previous version 1.4.9.3 by going the MLV>RAW>CDNG then importing into Premier for my regular workflow:
https://vimeo.com/88723935 (https://vimeo.com/88723935)
Shot on the t2i with the most recent nightly build [v2.3 March] at 1152x482 with a 2.39 aspect ratio, also at 1080p / 24 frames sec selected in the canon menus. Stretched just abit (14%) to fit. Not amazing quality, but eh for the t2i its pretty good! and its leagues better than the h264. :)
Will continue some testing [this time on my 5DIII] and post results. Excited to see the new version, will definitely give it a shot with my next test!
Title: Re: The CinemaDNG Discussion
Post by: kaco on March 13, 2014, 12:15:30 PM
raw2cdng v1.4.9.5 is online
* native mlv+audio

http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

PLEASE! i wont believe, that all is working fine. so, test and tell all bugs, glitches, problems. my wish is to get a stable 1.5 version..

Great work, thanks!
In my case i am having problems with FPS detection (the MLV was shot as 23.976, in older versions it worked OK).
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi59.tinypic.com%2F2dlyx4.jpg&hash=a15e9cc1b5561d38d708d9fbea868be2)
Title: Re: The CinemaDNG Discussion
Post by: kaco on March 13, 2014, 12:37:26 PM
Next problem is that the resolve is importing this CDNG as 24p (bud I can change this via Clip attributes). But the audio will always import as separate file and it is few seconds longer.
I assume this is some MLV stuff and not a problem of CDNG?
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi60.tinypic.com%2Fn5ovuh.jpg&hash=38abf04e29352e8a5350770732c0ec6b)
Title: Re: The CinemaDNG Discussion
Post by: Olivier57 on March 13, 2014, 01:51:42 PM
Hi everyone !  :)

Yes, I have the same problem than Kaco with the fps detection. Also: how do you activate the chroma smoothing function ? (Because it's "disabled")

Thanks
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 13, 2014, 02:34:48 PM
(1) fps-read from mlv. on the list. (something since yesterday while coding :))
(2) according framerate (also timecode) into cdng. on the list.
(3) chroma smoothing - not useable now, on the list for 1.5
(4) audio length. please check, if theres really async or if the audiopart is just longer.

edit:
6074 frames / 24fps = 4min 13sec 2frames
6074 frames / 23.976fps = 4min 13sec 8frames + ~6 frames audio often recorded by mlv (seen in another thread)

(5) audio/video resolve - maybe a nameproblem (according to the path/filename-thing found entries ago)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: kaco on March 13, 2014, 03:27:03 PM
(5) audio/video resolve - maybe a nameproblem (according to the path/filename-thing found entries ago)

probably a timecode problem.

OK, I found I way how to sync audio and video in Resolve. You can do this in Media Pool by syncing the clips viac timecode.
Read page 122 from this document: http://software.blackmagicdesign.com/DaVinciResolve/Manual/DaVinci_Resolve_Colorist_Reference_Manual_Nov_2013.pdf

Now the problem is, that all my clips from CDNG has Start timecode 01:00:00:00. I have to manually change clip attributes to 00:00:00:00 otherwise the sync won't work. I can't change this for all clips in one click, I have to do this for every DNG sequence.
@chmee: Could we somehow change the start TC in raw2cdng?

You can see the wrong start TC column here:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi60.tinypic.com%2Fn5ovuh.jpg&hash=38abf04e29352e8a5350770732c0ec6b)
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 13, 2014, 03:36:06 PM
(5a) :D thy will be done. static sync-TC set to 00:00:00:00 (because no one ever said something about it)
and/or
(5b) button switch for TimeOfDay?
and/or
(5c) instead of wav -> bwf (wav with timecode)
timecode in bwf is strange thing.. no.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: kaco on March 13, 2014, 03:45:50 PM
(5a) :D thy will be done. static sync-TC set to 00:00:00:00 (because no one ever said something about it)
and/or
(5b) button switch for TimeOfDay?

regards chmee

Tested it with bigger project. Manually changed 50+ clips to 00:00:00:00 and applied Resolve's "Auto-sync Audio Based on Timecode" function and it doesn't work very well. It links wrong audio and video files from time to time. So, maybe the ML does not add correct (same) timecode to audio and video? Or is this raw2cdng problem?

I know it works good in Premiere (probably based on filename), but for many the Resolve is more important.
Title: Re: The CinemaDNG Discussion
Post by: kaco on March 13, 2014, 03:52:18 PM
For now the only working solution for audio/video sync in Resolve is to manually select every audio/video and click chain icon to link the files:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi58.tinypic.com%2F2uny14z.jpg&hash=6a0e265cbbf12aada12d4ce7e05be271)

(quite annoying if you have 100+ clips)
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 13, 2014, 03:58:58 PM
because plain wav cant hold timecode, its fixed to 00:00:00:00. so, the problem should be managed in other ways. it could be the naming-definitions for path and mediafiles inside. will look on that. has something to do with the clipNo, that should be in directory and filename. (will investigate)

Directory : M021234_C0001
Pictures : M021234_C0001.[0000-9999].dng
Audio maybe : M021234_C0001.wav

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: kaco on March 13, 2014, 04:06:35 PM
because plain wav cant hold timecode, its fixed to 00:00:00:00. so, the problem should be managed in other ways. it could be the naming-definitions for path and mediafiles inside. will look on that. has something to do with the clipNo, that should be in directory and filename. (will investigate)

Directory : M021234_C0001
Pictures : M021234_C0001.[0000-9999].dng
Audio maybe : M021234_C0001.wav

regards chmee

Thanks! I can provide you working BlackMagicDesign CDNG if you like.
Title: Re: The CinemaDNG Discussion
Post by: tonybeccar on March 13, 2014, 07:01:35 PM
We're getting there!

For me it didnt work at all.. the frames, fps and camera model tags are wrong. The fps seems like it adds the previous value, so you end up with a huge number in the end. Fps shows 1.00.. maye that's the multiplier factor in mlv metadata? And when I hitted start, "raw2cdng stopped working". Oh also, i dragged and dropped 62 mlv files.. short ones, 9 seconds each.. and it took very long to add. Also, it added some wav files that were hanging around because of mlrawviewer.. it'll be wiser to filter it only for mlv's (and raw's).

But anyway, simple mistakes.. it's looking very promising!!!

Myself I'm coding at the moment.. making an interface update, to include dual iso, black image substraction, full mlv support and much much more.. please tell me you didnt forget about the command line? ;) Cause then it will be perfect!!!

Cheers chmee, thanks so much for the hard work! :)
Tony.
Title: Re: The CinemaDNG Discussion
Post by: andy kh on March 13, 2014, 07:11:54 PM
it seems like i cant add for than one mlv. its works if i add only one but if i do add two or more mlv and hit start "raw2cdng" stop working
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 13, 2014, 07:24:18 PM
ok, thanks for reports.

(1) framesCount reset . todo.
(2) Fps-show 1.00 - fixed
(3) right framerate in cdng - fixed
(4) modeltags are not taken from mlv, by now. todo.
(5) chaos-wav's - cant get the problem. think, will be fixed with other filename-definition.
(6) 69 mlv are a lot - every mlv has to be opened, properties have to be read and written into dngheader into ram while dragdrop-action. so, no great wonder.
(7) raw's dont have any wavs - because there is no mlv_dump-action anymore.
(8 multiple mlv's and raw. todo.
(9) command line maybe in 1.5

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 13, 2014, 07:54:37 PM
@g3gg0 and @a1ex
its strange, it doesnt fit to the definition of droppedframes: you write 23976/1000 into Nom/Denominator for 23.976fps. but by specification it should be 24000/1001. honestly saying, both leads (in my case) in premiere to 24fps. (page 10 in cdng-specs 1.1 (http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/devnet/cinemadng/pdfs/CinemaDNG_Format_Specification_v1_1.pdf))

(fixed: 0x0a instead of 0x05 for C764)

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: ted ramasola on March 14, 2014, 04:15:54 AM
chmee,

I tested v 1.4.9.5 but it keeps crashing after outputing a folder with 4-5 frames. Same files I tested extracted ok using raw to dng converter by tonybeccar.

I tried doing it one by one. Sometimes it would convert a file and in another it would crash.

I'm trying to test how long it would take to covert files from a single 64gig cf card comparing it to raw converter 1.9 to dngs.
For the Dng converter it took me 29 min. I'm trying to see if converting via raw2cdng would be faster.

edited:
additional info,
I am using 7D and all files are MLV.
Title: Re: The CinemaDNG Discussion
Post by: melimelo on March 14, 2014, 02:52:58 PM
no new bug to report but additional evidence for crashes commented on posts above:

1. unable to extract several mlvs together, always crashes
2. individual files, sometimes works, sometimes crashes (when it stops, it has already created the extract folder with the first 4-5 DNGs already extracted)

I'm on windows 8 (not 8.1) with mlvs coming from 5D2

pink in highlights as in previous versions (at least for me), only visible in premiere pro, not in ACR or Resolve

thanks for your work
Title: Re: The CinemaDNG Discussion
Post by: tonybeccar on March 14, 2014, 03:21:31 PM
Hey chmee, strange.. in my converter I also look for like 6 fields of metadata while adding, and it's extremely fast.. maybe it's the method you're looking for the bytes? I always look for an exact byte, I know there's a command that searchs all bytes in the files which takes forever.. anyway, you're an awesome programmer so I might just be talking nonsense.. hehe
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 14, 2014, 06:33:10 PM
tony, thats not the point. i#m not an awesome coder, i just have ideas and try to realize them, often with a lot of oldschool-sequential-code :) i decided to read the whole file-blocks (VIDF/AUDF-Count) while drag/drop-action - just to get my properties right, when they are in the list. if its that important i'd do them into the convert-part..

btw. 1.4.9.6 is on the way, maybe today, maybe tomorrow.
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 16, 2014, 01:00:31 PM
short info: testing all output-formats on several apps. think, today 1.4.9.6
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 16, 2014, 03:40:20 PM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

16.03.2014 – v1.4.9.6 – bug fixing
* timecode set to 00:00:00:00
* framerate in dng fixed
* mlv-batchmode fixed
* framesCount in GUI fixed

* 16bit and 12bit linear partymode are ok (in photoshop, acr, resolve and premiere, i think speedgrade as well, not tested)
* 12bit log/bmcc and 10bit are not really useable by now. investigating.

* linking video/audio in resolve via rightbutton link. searching for the according filename-conventions to get it automatically. 
Title: Re: The CinemaDNG Discussion
Post by: rrrmusic on March 17, 2014, 06:39:34 AM
Today tried to convert mlv to dng directly with raw2cdng v1.4.9.6 with a canon 5dmk3 and problem continues with pink highlights in Premiere CC 7.2.1
Partymode enabled with gamma 1.0 bitdeph 16 linear
Tiff model tag Canon EOS 5D Mark III
The view in ACR looks OK and audio is in sync

Magic Lantern version v2.3.NEXT.2014Mar11.5D3113

 I don't know what i'm doing wrong. Please let me know

The only problem i have is the pink highlights in Premiere cc 7.2.1\
sorry for my bad english
Thanks
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 17, 2014, 10:48:47 AM
Yes, there are still pink highlights in Premiere. the dng-properties seems ok, because in acr, photoshop, afx its all ok.. by now im fiddling about the way to avoid them in premiere, but not to cut too much highlight-data.

all infos & hints about this are highly appreciated.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: vikado on March 18, 2014, 12:30:16 AM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

16.03.2014 – v1.4.9.6 – bug fixing
* timecode set to 00:00:00:00
* framerate in dng fixed
* mlv-batchmode fixed
* framesCount in GUI fixed

* 16bit and 12bit linear partymode are ok (in photoshop, acr, resolve and premiere, i think speedgrade as well, not tested)
* 12bit log/bmcc and 10bit are not really useable by now. investigating.

* linking video/audio in resolve via rightbutton link. searching for the according filename-conventions to get it automatically.
awesome, thanks chmee!
your tool has been by far my favorite.
i'll be sure to donate to your cause!
btw, would it be possible to import the same cdng footage back into raw2cdng and compress it down even more?
Title: Re: The CinemaDNG Discussion
Post by: anandkamal on March 18, 2014, 04:19:12 AM
hi chmee, whenever raw2cdng crashes or even when i extract cdngs, there are additional empty folders with somewhat similar name styles created in the same directory. Example: M121643J_C0001_B16LINMAX (when software crashed while extracting .raw file), M121644K_C0001_B16LINNOR (empty folder1 after mlvs were extracted), M121643J_C0001_B16LINNOR (empty folder2, after mlvs were extracted)... the actual folder's name M121653K_C0001_B16LINNOR (contains all the cdngs from mlv file).. am i doing something wrong?
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 18, 2014, 11:05:27 AM
:) because i didnt tested the actual version with RAW, this could be a silly bug. thanks for info.
did you enabled the logfile? please link/show it via pastebin.
are you using a newer firmware with audio-support?

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: Nang on March 19, 2014, 04:50:37 AM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

16.03.2014 – v1.4.9.6 – bug fixing
* timecode set to 00:00:00:00
* framerate in dng fixed
* mlv-batchmode fixed
* framesCount in GUI fixed

* 16bit and 12bit linear partymode are ok (in photoshop, acr, resolve and premiere, i think speedgrade as well, not tested)
* 12bit log/bmcc and 10bit are not really useable by now. investigating.

* linking video/audio in resolve via rightbutton link. searching for the according filename-conventions to get it automatically.

Not sure if I'm doing something wrong, but I downloaded this version and placed them into my raw folder. I checked "Partymode", 1.0 for gamma, changed output path, and drag my raw files onto the Batchlist. When I click on convert to CDNG, a pop-up appears that says 'raw2cdng has stopped working'. I tried doing it again and change bitdepth and such, but still getting the same pop-up. Any suggestions?

Thanks
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 19, 2014, 07:19:54 AM
@nang
Quote
did you enabled the logfile? please link/show it via pastebin.
Did You used RAW or MLV-Files?
Quote
because i didnt tested the actual version with RAW, this could be a silly bug.
Quote
are you using a newer firmware with audio-support?
Title: Re: The CinemaDNG Discussion
Post by: DitOnTheLoose on March 19, 2014, 11:49:33 AM
My Debug Log for the instant crash on 1.4.9.6
Used with raw_rec, no audio.

And I have one feature request:
Can we have an option to change the output folder and filenaming?
Something like "Name folder and files after the original .raw" would be perfect.

Code: [Select]
14.03.19 11:40:01 --

----------- Application started ------------
14.03.19 11:40:01 -- - gamma loaded and set: 0
14.03.19 11:40:01 -- outputPath: M:\140318_Day_15\152
14.03.19 11:40:01 -- uniqueName: ML CDNG1.4.8 PHR
14.03.19 11:40:01 -- loading LogLists for 12bit
14.03.19 11:40:01 -- setting arrays for 12bit-Loglists
14.03.19 11:40:01 -- loaded GUI 1.4.9.6
14.03.19 11:42:14 -- **-- filesList_DragDrop started
14.03.19 11:42:14 -- * dragdropped File X:\140315_Day_13\138\138_M0001.RAW
14.03.19 11:42:14 -- is raw file
14.03.19 11:42:14 -- getting RAW-Attributes
14.03.19 11:42:14 -- ready. got all RAW-Attributes
14.03.19 11:42:14 -- set FileNameShort: 138M0001
14.03.19 11:42:14 -- added to fileData - batchlist
14.03.19 11:42:14 -- found Splitfiles: 1
14.03.19 11:42:14 -- new ListViewItem: 1920x1080 - 2016 frames - 25,00fps - 2047 - 15000 - Canon EOS 5D Mark III
14.03.19 11:42:14 -- added ListViewItems to GUI
14.03.19 11:42:14 -- **-- filesList_DragDrop Ended
14.03.19 11:42:18 -- **-- clicked startConvert and starting Workerthread _doWork
14.03.19 11:42:18 -- **-- _doWork Workerthread started
14.03.19 11:42:18 -- yes, there are Files in the list: 1
14.03.19 11:42:18 -- counting Frames
14.03.19 11:42:18 -- counted AllFrames: 2016
14.03.19 11:42:18 -- changing GUI because of work :)
14.03.19 11:42:18 -- * converting File: 138_M0001
14.03.19 11:42:18 -- changing bitsPerSample
14.03.19 11:42:18 -- bitdepth set to : 16
14.03.19 11:42:18 -- blackLevel Old - New : 2047 - 2047
14.03.19 11:42:18 -- whiteLevel Old - New : 15000 - 15000
14.03.19 11:42:18 -- set and change DNG-Values
14.03.19 11:42:18 -- changed uniqueName: ML CDNG1.4.8 PHR
14.03.19 11:42:18 -- byte Array DNGHeader written
14.03.19 11:42:18 -- set and changed OutputPath: M:\0000_Test\1
14.03.19 11:42:18 -- * Opening File and setting *delta for Splits*
14.03.19 11:42:18 -- - File 138_M0001.RAW split No 0
14.03.19 11:42:18 -- - delta Begin: 0
14.03.19 11:42:18 -- - delta End: 192
14.03.19 11:42:18 -- - framepos: 2016
14.03.19 11:42:18 -- * delta SplitValues done
14.03.19 11:42:18 -- created OutputPath: M:\0000_Test\1\138M0001_C0001_B16LINNOR
14.03.19 11:42:18 -- created helperClass parameter for Threading
14.03.19 11:42:18 -- set MaxThreads to 4
14.03.19 11:42:18 -- started CountdownEvent. starting threadloop for frame-converting.
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 19, 2014, 12:23:25 PM
[raw] ok, thanks a lot. i'll look on the raw-converter-modules - there will be a bug inside.

Quote
..Name folder and files after the original .raw..
just give me some days. by now there's on the list "pink highlights" (nearly done) and "raw".

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: Nang on March 19, 2014, 01:43:12 PM
@nangDid You used RAW or MLV-Files?

I tried on RAW files and I'm using August 13th build. Maybe that's the reason why? I haven't updated because everything's been working perfectly since August.
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 19, 2014, 02:09:06 PM
it should work with this older version as well - for sure its a bug in my code for raw-data - but, yours is really old :)
Title: Re: The CinemaDNG Discussion
Post by: Nang on March 20, 2014, 12:20:13 AM
it should work with this older version as well - for sure its a bug in my code for raw-data - but, yours is really old :)

I'll look into updating to a more recent build. My concern is if something goes wrong. The August build has been working flawlessly thus far, but maybe it's time to update. Might give mkv a shot, but I record my audio externally so I never really had a need for this.

I tried version 1.4.9 and for the most part it works, but it crashes randomly.
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 21, 2014, 11:12:04 PM
raw2cdng1.4.9.7 is online pink highlights fix
* this is a testversion, if pinkhighlight-fix works everywhere
* logarithmic approaches freezed
* partymode is now on by default, no click. gamma changes not needed as well.
* raw-bug fixed (fps.denominator was the bad boy)
* downside: it has lost much of its speed. (but: theres space to optimize a lot)

short tests with premiere, acr/photoshop and resolve looked ok. (5dIII)

http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: anandkamal on March 22, 2014, 06:26:58 AM
chmee: "because i didnt tested the actual version with RAW, this could be a silly bug. thanks for info.
did you enabled the logfile? please link/show it via pastebin.
are you using a newer firmware with audio-support?

regards chmee"

I am sorry, but what is logfile and how to enable it?
I used .raw files. Also i request for fixing bigger (greater than 6GB) raw files getting crashed while converting.
i am using the latest firmware, both in ML and raw2cdng. I am using raw and not mlv, so no audio. By the way, which is preferable for the converter, .raw or .mlv?? thanks
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 22, 2014, 06:56:08 AM
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fraw2cdng_desc.jpg&hash=534b4d9f64151e183d2e0f38ecf326fe)

No 7

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: Wartburg-Entertainment on March 22, 2014, 07:06:52 AM
Hello Chmee,

since the last two versions of the app does not create more folders.
It used to be that I got MLV per file, create a folder.
Now creates a folder and it contained only by a MLV the CDNG.
The others are generated, they are not found. (overwritten)

greeting
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 22, 2014, 07:09:18 AM
?written so complicated..? (try german?!) do you mean the batch-convert is broken?
Title: Re: The CinemaDNG Discussion
Post by: Wartburg-Entertainment on March 22, 2014, 07:15:24 AM
Yes deutsch, :)

in den letzten Versionen war es immer so, dass ich pro MLV einen Ordner angelegt bekam. Das war super.
Seit 1.4.9.6 und 7 entsteht nur ein Ordner von einem MLV File.
Das Programm arbeitet aber munter weiter und scheint die anderen CDNG's zu erzeugen, aber im Ordner sind nur die Files von einer MLV-Datei.
Habe noch nicht gecheckt von welcher - erste oder letzte in der Batchliste.
Windows 8!!


Gruß aus Thüringen

EDIT: Es sind die Dateien vom letzten MLV-File in der Batchliste!! Es werden die vorgehenden überschrieben. Deshalb kann ich sie nicht finden!
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 22, 2014, 07:50:14 AM
Ok danke. Schau ich mir heute abend an.
Title: Re: The CinemaDNG Discussion
Post by: ted ramasola on March 22, 2014, 08:56:06 AM
tested latest with several MLV files. Seem to work well so far.
Title: Re: The CinemaDNG Discussion
Post by: anandkamal on March 22, 2014, 08:41:11 PM
version 1.4.9.7 works well with .raw files.. i tried with files greater than 7 GB and multiple files.. great Chmee... thanks
Title: Re: The CinemaDNG Discussion
Post by: random on March 22, 2014, 09:46:21 PM
anybody has an idea why the cdngs ignore the white balance I chose?
I get horrible warm images in Premiere independent from which Kelvin I record with.
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 22, 2014, 10:05:41 PM
because i set static values. its a code-module before mlv, that now has whitebalance-data.. long time ago (v1.3.0) i wrote a whitebalance-importer, that takes the values from a cr2-photo. its still inside raw2cdng. but, thanks for the hint, seems, its time to change that.

@wartburg-entertainment
please send me a logfile from a multiple file-convert-process. raw2cdng[at]phreekz.de or use pastebin.com
Title: Re: The CinemaDNG Discussion
Post by: AD DP on March 22, 2014, 11:46:22 PM
raw2cdng1.4.9.7 is online pink highlights fix
* this is a testversion, if pinkhighlight-fix works everywhere
* logarithmic approaches freezed
* partymode is now on by default, no click. gamma changes not needed as well.
* raw-bug fixed (fps.denominator was the bad boy)
* downside: it has lost much of its speed. (but: theres space to optimize a lot)

short tests with premiere, acr/photoshop and resolve looked ok. (5dIII)

http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/

regards chmee

Hey Chmee,

Went through another short project using your raw2cdng, no problems here! I had some pink banding on afew frames - but I believe this happened during recording the MLV's. I dropped the affected frames.
Here is the video: https://vimeo.com/89770585 (https://vimeo.com/89770585)

Will be migrating to the 5DIII from here on out. :)
Thanks!
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 23, 2014, 12:07:22 AM
@AD DP
thanks for sharing (and using raw2cdng). if you use lower exposure and control highlights below a point, they havent be seen in earlier version as well. i made tests with dimming lights to see, where the pink highlights begun, and finally tried to investigate where this level is. on this point green is clipped, but red and blue are going further. but to say it once more clear, its the fault of the cheap cdng-code in premiere. its horrible. with this step we have lost highlight recovery. in v1.5 there will be back the "old" 16bit transcoding (beside the highlights-hack). and finally i hope, adobe will do something clever: code the importer as dng was founded, an open format with descriptive metadata inside it.
Title: Re: The CinemaDNG Discussion
Post by: ted ramasola on March 23, 2014, 02:32:32 AM
tested 1.4.9.7 to convert 3 mlv files recorded at 60P and 2 of those files caused the app to crash. I noticed that selecting 12 bit causes the crash. 16bit no problems.

I went back to 1.4.9.6 and all 3 converted fine using 12 bit.
Title: Re: The CinemaDNG Discussion
Post by: kirklasalle on March 23, 2014, 09:22:05 PM
OMG! Where the hell have I been! I just found this today! I just wanted to give a big Thank You!

I've been trying to find some kind of exit strategy for the 19lights GingerHDR workflow I've been using, which just shut down (drag and drop with ML Raw http://19lights.com/wp/tutorials/magic-lantern-raw-video/ was a great solution for a bit, fast workflow with direct import into PP with very few drawbacks)... After that, First option was the ACR workflow with 8bit output and that just wasn't going to cut it (for bigger jobs)... but, now I know I can get solid 12bit cDNG on my timeline (with a pro NEC monitor) and I couldn't be happier.  Awesome deal. Great Work man! :D
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 23, 2014, 10:28:11 PM
@ted ramasola
thanks for information. did it converted some of the pictures or does it crash right in the beginning? could you send me first ~10mb of the failing file? (use for example Hxd for cutting that into another file)

@kirklasalle
you're welcome. tell me/us a little bit about the workflow with your nec monitor, primarly about your real advantages. thats a interesting thing. and of course, how the cdng-files behave..

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: zacshome on March 24, 2014, 03:47:46 AM
Hey guys, Using 1.4.9.6, I have been encoding with 16bit and 1.0000 gamma, i did some test of encoding with and without 1.0 gamma and it seemed the same, not sure what benefit I am getting from that..

Also when I export as 12bt LOG for resolve, all my files come out pure white, but 16bit linear works perfect. I thought with resolve I am meant to be exporting them as LOG files for my lut's to work but doesnt seem to be working :) (sorry noob post but have searched online for an hour or so and can't find a solution)
Title: Re: The CinemaDNG Discussion
Post by: kirklasalle on March 25, 2014, 01:26:38 AM

@kirklasalle
you're welcome. tell me/us a little bit about the workflow with your nec monitor, primarily;yy about your real advantages. that's a interesting thing. and of course, how the cdng-files behave..
regards chmee

The cDNG's I get are drag and drop which is great... workflow is what you'd expect, import the folder from prelude (when needed, which auto-pulls up media encoder and allows transcode for offline, so you'll need some space and some time, again, IF NEEDED, I say if needed when working between web project @720p and  feature length project @ full color gamut... otherwise, just direct drag N' drop folder import into PP CC) all the way through to a final edit and color. [EDIT: btw - only first pass on color in PP, 2nd and final polish in Davinci]

Really the monitor (http://www.bhphotovideo.com/c/product/880645-REG/NEC_p232w_bk_23_Widescreen_Professional_Graphics.html) not that big of a deal, it just allow me to do 10bit color with a wide AdobeRGB gamut and has some function for a basic LUT in the field... the cool thing is your software allows me to choose 12bit , that's what I really got excited about (16bit crashes on me, I'll pull logs if you need) so, I have two 7Ds that I can keep now (no buying a new camera) and get some kickin RAW out of them.

[EDIT: The thing is, right now, for the price, (especially since I already own two 7D's) you can't beat 12bit RAW @ 1080p with the 7D, it saves me on a monthly rental... you deserve a huge (and more of a) thanks!   Please don't stop! this is some of the best ML covert software I've seen so far]

I'll donate... I promise that! [EDIT: Although, I need an English version... there's no translate with paypal I guess]
Title: Re: The CinemaDNG Discussion
Post by: ted ramasola on March 26, 2014, 03:21:04 AM
@ted ramasola
thanks for information. did it converted some of the pictures or does it crash right in the beginning? could you send me first ~10mb of the failing file? (use for example Hxd for cutting that into another file)

Its hard to replicate the crash, but I've also made comparison with 1.4.9.6 and 1.4.9.7 and it seems the .6 version produces similar quality with 14bit DNGs side by side in resolve. When I use 1.0 Gamma and NO Party mode. The CDNGs from .7 version is darker and I need to push up exposure in resolve. What is the default gamma for .7 ?
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 26, 2014, 08:02:56 AM
the gamma is 1.0 as well, but i have to lower the values by (assume) ~1ELV to fix the highlights for premiere. in next versions will be back the normal convert without partymode - for usage in speedgrade, acr and resolve.

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: DitOnTheLoose on March 26, 2014, 12:21:57 PM
I am back on 1.4.9.

The new CDNG naming convention interferes with my workflow. Which is to bad since I really liked not having to change the FPS every time I imported into Resolve.
Title: Re: The CinemaDNG Discussion
Post by: anandkamal on March 27, 2014, 05:46:37 PM
hi, i am trying to open .cdng files in Nuke. With J_Ops plugin, I could only open a single frame of .cdng file in Nuke 7.0. The sequence could not be opened as usual, like in .exr pics. Pls help me out. How do i import the entire .cdng sequences? thanks
Title: Re: The CinemaDNG Discussion
Post by: Olivier57 on March 27, 2014, 07:26:42 PM
anandkamal: Why do you want directly open cdng in Nuke ? ... In the cinema industry: the digital compositors work their shots a lot with the dpx 10 bits log and Exr for the 3d sequences.  ;)

I read in the past even with J_Ops plugin, Nuke is not able to read dng.
Title: Re: The CinemaDNG Discussion
Post by: anandkamal on March 28, 2014, 10:57:21 AM
thanks Olivier57.. that would be a good alternate. Also Nuke is good with .exr. I thought because of quality degradation, converting cdng to dpx10 or exr would be lossy. Anyways, that being a standard, I will have to do that.  ;)
Title: Re: The CinemaDNG Discussion
Post by: anandkamal on March 28, 2014, 09:49:32 PM
hi.. sometimes my footage does not play in MLRawViewer, the screen is fully black, I thought its a mistake and proceeded to convert the .raw to cdngs. The converter shows "0 FPS" and crashes. Is that footage a shooting mistake? i don't think so bcos other footage is good. Am I wrong at something? Why would that happen? (the take I okay-ed is unfortunately showing this error  :-[)
Title: Re: The CinemaDNG Discussion
Post by: kaco on March 28, 2014, 11:11:09 PM
@chmee: Any news about the vertical banding (fixed pattern noise)? I've shot a lot 50p footage and the banding is pretty bad. So I have to convert MLV's to RAW's, then export dngs  with raw2dng and then have to edit the exif camera model with irfanview batch for ACR -- very slow and complicated process. It would help a lot if you'll have a time to implement FPN correction. 0:-)
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 29, 2014, 12:03:25 AM
..because i went to an x/y-view to the data (instead of simple array rewriting) theres a good chance to implement anti-banding and cs.
Title: Re: The CinemaDNG Discussion
Post by: ted ramasola on March 29, 2014, 01:02:07 AM
@chmee,

does the new fix in raw2dng have any effect on your raw2cdg app?

A1ex fixed a bug posted here;
http://www.magiclantern.fm/forum/index.php?topic=5404.msg109330#msg109330
Title: Re: The CinemaDNG Discussion
Post by: GooDween on March 30, 2014, 10:04:54 AM
can someone explain why in after effects is so slower playback of dng?
in davinci i got around 10-15 fps.
in adobe 0.5 i think i even can't scrub trough
Title: Re: The CinemaDNG Discussion
Post by: chmee on March 30, 2014, 10:23:41 AM
apart from the fact that after effects in general is slower (its a compositing software, not a clip-based edit tool), dng is imported by a slow, but high-quality onTheFly-converter (acr).

please be patient, i'm gathering all experiences, fixing and optimizing raw2cdng. there was work as well (echo2014 award) https://www.flickr.com/photos/chmee/

@ted will have a look on it. i never saw this kind of artifacts..

regards chmee
Title: Re: The CinemaDNG Discussion
Post by: arrinkiiii on March 30, 2014, 10:38:16 AM

Nice pic's Chmee  :)
Title: Re: The CinemaDNG Discussion
Post by: zacshome on March 31, 2014, 05:24:45 AM
Is there a way to get rid of vertical noise with this app? I dont get vertical noise in raw2dng but I do in raw2cdng
Title: Re: The CinemaDNG Discussion
Post by: dangerzonerj on April 02, 2014, 03:41:52 PM
yeah I'm also having those vertical banding issues.

There's another app (mystic) which has a fix for it, just check the function and bang, problem solved. Perhaps something like it on raw2cdng?

Thanks chmee for the lovely job on this software!
Title: Re: The CinemaDNG Discussion
Post by: chmee on April 02, 2014, 05:08:11 PM
i described it above. to solve vertical banding or deadpixels you have to look on it as a picture (x/y-pixels) not as an array of bytes. i'm on it, but i guarantee, the conversion will slow down. thus it will be an optional button..

(still recoding) regards chmee
Title: Re: The CinemaDNG Discussion
Post by: chmee on April 03, 2014, 10:48:08 AM
by the way. we'll see what advantages the next CC-Version will bring (NAB2014) - maybe the whole work is not needed anymore :)
http://www.cinema5d.com/news/?p=24528
Title: Re: The CinemaDNG Discussion
Post by: kaco on April 03, 2014, 10:54:15 AM
by the way. we'll see what advantages the next CC-Version will bring (NAB2014) - maybe the whole work is not needed anymore :)
http://www.cinema5d.com/news/?p=24528

Still lot of people use CS6 and I think this is the Canon raw from C500, not ML. But we'll see :)
Title: Re: The CinemaDNG Discussion
Post by: chmee on April 03, 2014, 11:11:50 AM
i do not count on the canonRAW-entry in the video, but on optimizing/opening the raw-module. thats all i want :) (but yes, i think about converting ml_raw to arriraw or canonraw, but these tests will come after the recode of raw2cdng). After that (middle of 2017) think i'll find the time to read the CC-sdk to write an native importer :D
Title: Re: The CinemaDNG Discussion
Post by: DFM on April 03, 2014, 01:04:35 PM
by the way. we'll see what advantages the next CC-Version will bring (NAB2014) - maybe the whole work is not needed anymore :)
http://www.cinema5d.com/news/?p=24528

The forthcoming release of Premiere Pro CC and Media Encoder CC will support *some* types of ML-generated CinemaDNG footage but not all, and they have to be real cDNG frames not just a folder of 'stills'. I can't give a detailed matrix yet as the code is still in flux, for example the build I'm testing now has problems with stuff from the 7D and won't read 14-bit files.
Title: Re: The CinemaDNG Discussion
Post by: DFM on April 03, 2014, 01:07:36 PM
... by the way, if folks want me to run a test on a particular 'flavor' of cDNGs, upload them someplace and ping me a link. Don't need more than a second's worth of frames.
Title: Re: The CinemaDNG Discussion
Post by: DavidSh on April 03, 2014, 01:14:32 PM
... by the way, if folks want me to run a test on a particular 'flavor' of cDNGs, upload them someplace and ping me a link. Don't need more than a second's worth of frames.
Hi DFM,
I use MlRawViewer to export cdng, it opens really good when i import to pp cc and use direct link to speedgrade. but when i import straight to speedgrade it has heavy green cast.
Does the new version has solution to this issue?
Title: Re: The CinemaDNG Discussion
Post by: chmee on April 03, 2014, 02:04:11 PM
For Speedgrade you should build your own Matrix or LUT - i made a Matrix-example in raw2cdng 1.4.5. Use this one or tweak the Matrix, and write them into the file i provided. thats not a real issue.

btw. did @baldand solved the pink-highlight-problems?
Title: Re: The CinemaDNG Discussion
Post by: DFM on April 03, 2014, 02:36:04 PM
Sg in the current version of CC expects slightly different things in footage; DNGs in the other apps previously went through Camera Raw but in the forthcoming version they import natively, and you don't get the ACR dialog if you open the cDNG footage's "source settings" - instead there's a Lumetri quick-adjust tool. That will impact on workflows that use the ACR engine to do heavy lifting (lens corrections, noise reduction, etc).

Here's the dialog being fed a 16-bit linear cDNG from chmee's app:

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fmedia.uvsar.com%2FAdobeCC%2FPPCC-Lumetri-dialog.jpg&hash=02b0a9d4084b357856027018db04daf0)
Title: Re: The CinemaDNG Discussion
Post by: DavidSh on April 03, 2014, 02:52:52 PM
For Speedgrade you should build your own LUT - i made an LUT-example in raw2cdng 1.4.5. Use this one or tweak the Matrix, and write them into the file i provided. thats not a real issue.

btw. did @baldand solved the pink-highlight-problems?
Thanks chmee i will look for your example, even when i fix the green cast it looks like there is no highlight information to work with.

Regard baldand MlrawViewer,  on pp cc only saw them once at high contrast areas.

Title: Re: The CinemaDNG Discussion
Post by: DavidSh on April 03, 2014, 03:47:26 PM
chmee your matrix made my day!
Title: Re: The CinemaDNG Discussion
Post by: arrinkiiii on April 03, 2014, 04:03:29 PM
Sg in the current version of CC expects slightly different things in footage; DNGs in the other apps previously went through Camera Raw but in the forthcoming version they import natively, and you don't get the ACR dialog if you open the cDNG footage's "source settings" - instead there's a Lumetri quick-adjust tool. That will impact on workflows that use the ACR engine to do heavy lifting (lens corrections, noise reduction, etc).

Here's the dialog being fed a 16-bit linear cDNG from chmee's app:

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fmedia.uvsar.com%2FAdobeCC%2FPPCC-Lumetri-dialog.jpg&hash=02b0a9d4084b357856027018db04daf0)

Looks good. It's a start... i guess.
Title: Re: The CinemaDNG Discussion
Post by: chmee on April 03, 2014, 04:08:53 PM
@DFM thanks for the insight - looks like the arriraw-settings window :)

@DavidSh you're welcome

@baldand had an issue entry on bitbucket about pink highlights. i emphasize his opinion, we have to wait for the next CC-Version. my actual fix is not the ideal solution.
https://bitbucket.org/baldand/mlrawviewer/issue/55/premiere-renders-dng-highlights-pink
Title: Re: The CinemaDNG Discussion
Post by: baldand on April 03, 2014, 06:17:27 PM
btw. did @baldand solved the pink-highlight-problems?

Unfortunately not. I'm still hoping Adobe solve this in the next version of PP since it's clearly their issue.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on April 08, 2014, 11:52:04 PM
Hi all , there a new update to premiere pro today at least to me. Version CC 7.2.2(33)
I guess with NAB 2014 now in full swing there was bound to be some updates .
Anyways I was trying some cdngs and at least to me it the closes I have seen yet.
I used raw2cdng Version 1.4.9 with 12bit party mode gamma1 it looks right.
With 16bit its still pink hi-lights, so looks like improvement for 12bit , if someone could verify
this the would be great . :) 
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: DFM on April 09, 2014, 11:43:47 PM
With the forthcoming versions, 'party mode' isn't required anymore. The screenshot I posted earlier isn't of a 'party mode' file.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 10, 2014, 12:31:33 AM
@DFM good to hear.

btw. raw2cdng 1.5 is on the way, one or two days to finalize coding and then there will be a beta.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 12, 2014, 01:08:39 AM
ok. please see that really as a beta. testing, telling. all stuff you remark, problems, ideas and so on.

raw2cdng 1.5.0BETA

* PLEASE before the first start rename/delete the old settings-file in user/[user]/appdata/roaming ->raw2cdng.settings.json
* mlv and raw native (quite fast :))
* audio of course
* changed GUI to wpf (just saw a design-glitch on another pc)
* mouseover on preview ;) the preview is not accurate, its just a preview if the footage is the right one.
* the checkbuttons in the list are not implemented yet.
* options are disabled by now excepted pink highlights (but it doesnt work well by now :) i assume only in 16bit)
* there is a possibly empty field under the bitdepth-buttons. mouseover. you can refine your filename to your own needs..
* only for testing general purposes!
* this version is only linked here

http://dl.phreekz.de/raw2cdng/raw2cdng.1.5.0.beta.zip

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ted ramasola on April 12, 2014, 01:32:34 AM
how about the ev levels chmee? is it default like 1.4.9.6 ? or lowered like 1.4.9.7 ?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 12, 2014, 01:48:48 AM
the ev levels just lowered if you're using pinkhighlight-fix, as i assume. by now i wait for the next adobe update - and i hope we dont need any pinkhighlight-fix :) last days i didnt really tested all circumstances.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on April 12, 2014, 05:52:51 AM
Had a lots of App Crashes, worked for about 10 file (loaded same file but change settings)
now it just crashes when I try to convert. Delete the old settings-file in user/[user]/appdata/roaming ->raw2cdng.settings.json
to get it to work after it crash, but now it just stop working, the preview work ok and the metadata was visible but no go.
Of what I did get to work I found Cdng 16bit maximized to work the best no pink, but very flat looking but editable.
12Bit maximized  worked also but with a gamma shift up and all high lights were pink and "delta pink high lights" didn't make any difference one way or another.
But it wasn't flat it had good contrast.
what's the differences between maximized and non and what doses "delta pink high lights" do ?
Note * it crashes on Raw & MLV. Deleted the .json file again and use 1.4.9 and back to normal for raw.
Windows 7 Ultimate , Toshiba i7 laptop 8GB Ram, SSD. the only error messages I could get are as follows:
"C:\Users\i7 Toshiba\AppData\Local\Temp\WERDA.tmp.WERInternalMetadata.xml
  C:\Users\i7 Toshiba\AppData\Local\Temp\WER3784.tmp.appcompat.txt
  C:\Users\i7 Toshiba\AppData\Local\Temp\WER3795.tmp.mdmp"


I do like the new look very nice job, the preview very fast. I though came to mind as I'm writing this, the preview stop working on the last 3 files
of MLV so I did not click to preview I just converted but then after a few more it stop working all together.
I also have dual video cards on my laptop (Intel HD400 & NvidiaGeforce GT 630M) Could this be a problem with debayering in preview ?
Good job! :)

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 12, 2014, 09:25:12 AM
:o geez.. ok, i have to code the debuglog into the new version :) it seems, it has to do with the IO-read/write-routines. did you used some heavy subpath-things?

please try it one more time with one path from root and put into the filenamegenerator
Code: [Select]
[S]_ (for shortfilename). just to hear if it helps..

(btw. weeks ago i searched for some filename definitions from other filmcameras, i will optimize the filenamegenerator to this, but its quite possible to write the same filenames with raw2cdng)
http://pastebin.com/bE5jsBUY
Code: [Select]
Arri like filenaming
[CardNo][ClipNo]_[Date]_[Base36SerialNo]
A004C010_20100610_R1JL.mov

could be in raw2cdng
A001C001_[D]_[S]

regards and thanks a lot chmee
Title: Re: The CinemaDNG Discussion
Post by: Midphase on April 12, 2014, 07:48:09 PM
Unfortunately not. I'm still hoping Adobe solve this in the next version of PP since it's clearly their issue.

Wish you would have been at the CDNG NAB meeting on Wednesday. Some good info was being given. Frustratingly though, Adobe failed to send any of their people over. There are some good things potentially coming to the CDNG standard.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 14, 2014, 12:46:15 PM
i had been busy shooting so couldnt test the new version and finally i hav tested today. it keeps on crashing and i am unable to use the application at all. however i wil love this new application once all the check buttons are implemented. this new version will save a lot of time. i have been using the old version v 1.4.9.3 because i have to convert to raw first and remove pink dots.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 14, 2014, 12:56:48 PM
please give me some more informations.

* when does it crash? right in the beginning? after drag/drop? while showing preview? when starting to convert?
* maybe once again a path/naming problem?

@ted ramasola [pinkhighlight]
i just made my thoughts about the light-loss because of the highlight-workaround. finally it does not corrupt the SNR, because the values are only mathematically lowered (in 12/16bit). so if you brighten the frame, it should not have more SNR than before.. it "only" loses kind of highlight recovery ability, so it does not affect the simple premiere workflow, because there is no way to recover highlights..
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 14, 2014, 01:27:06 PM
i can drag and drop. after that i cant click anywhere as it would crash if i do so
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 14, 2014, 01:49:13 PM
@andy_kh
would you quote the path, your mlv/raw are saved? i'll do the test here now as well with a bigger directory-tree.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 14, 2014, 02:33:28 PM
@chmme

i have tried different paths but it dint work and i cant even click the check box to preview. it keeps on crashing

i think (reddeercity) has mention the same problem above
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 14, 2014, 02:53:48 PM
if you dragdrop, its imported into the list and the values are right? resolution, frames/framerate?

but, nonethelesse, i will recode the debug-logging, to get faster informations.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 14, 2014, 04:23:27 PM
if you dragdrop, its imported into the list and the values are right? resolution, frames/framerate?

but, nonethelesse, i will recode the debug-logging, to get faster informations.

yeah values are right
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: tonybeccar on April 14, 2014, 06:43:29 PM
Amazing changes!!!! Interface is a beauty!! As anotherone said here.. a lot of crashes with the app, that for me have to do with the interface. Most reproducable when adding multiple files.. after that when I scroll, or tick a box, or click a column, crash. The preview is out of this worldddd!!! It'll be just nuts to have the function like premiere, to scrub with the mouse when you move to the right, hehe. One last thing I think this program is missing that is vitally important, is multifolder support, subfolders I mean. When you are on a big shoot you always have your footage organized into folders. Oh and also, the naming convention, very nice! But is it possible to have the option to have the filename untouched? The shortened filename is weird and I never figured out what the last 'random' letter means!

Anyway, amazing stuff!!!
Cheers and thanks chmee!!!!!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 14, 2014, 07:06:31 PM
thanks.

* Yeah, wpf is a new world for me, i have to figure out, what event is doin the crash.
* i thought about scrubbing over the picture (left frame 0, right last frame), we'll see :)
* subfolders.. hmm. means, you'd change the subfolders per sequence. in this case, you put the mlv/raw files into according named subfolders and use the "source-path alias" inside the namegenerator.. it wont generate subfolders, but it will ease the sorting by name (maybe i find an intuitive way to implement it)
* realFilename comes to the alias list. thx.
* the shortened filename was born out of the fact, EDL-Names should be 8 chars long - and the mlv-name [a-z0-9] is just 7 - the eighth char is calculated from the first 7, means its not random, but not really useable :)

but to be honest. i wont code a commandline - if this ones will work, i'll start coding a premiere native importer-plugin - after some longer tests i have to admit, that the transcoding process is time consuming as well - and in the end not needed..

if someone's got crashes, please pastebin the saved crashreports (not the minidump of course :)) they will be in the temp-path with some strange name.xml. (see above, entry from reddeercity)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 14, 2014, 10:23:32 PM
@chmee: we have discussed the wav+dng import into resolve, so I know you are working on this. Have you tested this in Adobe AE? When I drop the CDNG folder into AE it imports just the video, not audio in my case. Maybe I'm doing smth wrong or it is not working? But it would save a lot of time if this will work.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 14, 2014, 11:29:42 PM
i promise i will look on that :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 15, 2014, 10:45:59 AM
btw. i just made some small boring tests. recording, transcoding (16bit maximized, no pinkhighlight-fix) and straight using in premiere.

* the transcoding-process is (for me) much too long. as i stated theres only one solution. writing a mlv-importer for AE/premiere, since its the same SDK-Base
* importing the filesequences is a bit annoying, because you have to mark sequence by sequence. have to figure out, if premiere can cope with kind of containers, marking all pathes, thats it.. (would be solved with the importer :))
* but finally premiere works well with the data, i had no pink highlights, maybe i didnt overexposured :)

(vimeo) 1280*448 at 50fps - down to 23.976fps

(youtube) 1920*1080 at 23.976fps

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 15, 2014, 07:11:54 PM
@chmee, it is very slow and complicated process (AE or resolve), but it will give you more production options. For me, the basic color grading tools in Premiere CC are not enough (I think they are more color correction than color grading). So even the process is slow and complicated, it is sometimes worth it.

I usually export with raw2cdng (or MLV converter if I need to fix vertical bandings). Then depending on the type of a project:

1. Create proxies in Resolve (faster then realtime). Import into Premiere. Make my edit, export XML to resolve, create final grade.

2. Create proxies in Resolve. Import into Premiere. Make my edit. Then duplicate the edit in the another layer. Nest every clip. Replace with AE composition (Adobe dynamic link) and import CDNG sequence. Then I make color grading on the first frame with Lightroom, save metadata and reload footage in AE. It will also update the clip in Premiere. Very slow, but higher quality debayer process then in Resolve.

3. Grade with Lightroom, import all into AE and directly export Prores/DNxHD -- all clips graded.

The second workflow is highest quality and with ability to render just the clips used in the final edit with ACR. But very annoying that I have to manually nest and relink every clip to AE.

Still though, I have problems with audio :D You should really look on that. If you have 1TB project with 250 shots, I spend day or two linking just the audio. :D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 15, 2014, 07:15:06 PM
Also, using Premiere CC to grade raw makes no sense to me when you can use Resolve which is for free. The process is not so much more complicated, isn't it?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 15, 2014, 07:19:26 PM
@kako

i dont understand why dont you guys import cdng in cc and start your cut? you guys neednt sync audio manaully in premiere pro thats the reason why i love raw2cdng. after your cut you can grade in resolve or after effects without having any problem
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 15, 2014, 07:22:03 PM
@kaco
with cdng editing raw is as easy as editing h264. i wil do a tutorial very soon to show you what i mean
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 15, 2014, 07:35:12 PM
@kako

i dont understand why dont you guys import cdng in cc and start your cut? you guys neednt sync audio manaully in premiere pro thats the reason why i love raw2cdng. after your cut you can grade in resolve or after effects without having any problem

It's because I am still on CS6. If there would be ACR involved into Premiere CC I would definetely upgrade. But just because of CDNG "proxies" and then also round tripping to Resolve it's not worth the money for me.

If the CDNG audio works with Premiere, I think it might somehow work in AE and Resolve too. If it works with Blackmagic CDNG files. I would say that the Premiere just checks the filenames while AE and Resolve also timecode or what -- that's why the audio and video imports as two separate files.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 15, 2014, 07:39:14 PM
If the CDNG audio works with Premiere, I think it might somehow work in AE and Resolve too. If it works with Blackmagic CDNG files. I would say that the Premiere just checks the filenames while AE and Resolve also timecode or what -- that's why the audio and video imports as two separate files.

it doesnt work in resolve
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: tonybeccar on April 15, 2014, 07:43:05 PM
thanks.

* Yeah, wpf is a new world for me, i have to figure out, what event is doin the crash.
* i thought about scrubbing over the picture (left frame 0, right last frame), we'll see :)
* subfolders.. hmm. means, you'd change the subfolders per sequence. in this case, you put the mlv/raw files into according named subfolders and use the "source-path alias" inside the namegenerator.. it wont generate subfolders, but it will ease the sorting by name (maybe i find an intuitive way to implement it)
* realFilename comes to the alias list. thx.
* the shortened filename was born out of the fact, EDL-Names should be 8 chars long - and the mlv-name [a-z0-9] is just 7 - the eighth char is calculated from the first 7, means its not random, but not really useable :)

but to be honest. i wont code a commandline - if this ones will work, i'll start coding a premiere native importer-plugin - after some longer tests i have to admit, that the transcoding process is time consuming as well - and in the end not needed..

if someone's got crashes, please pastebin the saved crashreports (not the minidump of course :)) they will be in the temp-path with some strange name.xml. (see above, entry from reddeercity)

regards chmee

Awesome awesome awesome!!!

Wow, an MLV importer would be like... mmhhh there is no real world comparison, lol! Regarding the subfolders system, take a look at how I've done it in my mlv converter... it's pretty straightforward, you maintain alphabetical order and everything. I do not show ONLY the filename when it's multifolder but that info I guess can be stored in a property value somewhere...

Anyway, amazing work man!! Thanks so much!
Cheers!!!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: tonybeccar on April 15, 2014, 07:51:46 PM
btw. i just made some small boring tests. recording, transcoding (16bit maximized, no pinkhighlight-fix) and straight using in premiere.

* the transcoding-process is (for me) much too long. as i stated theres only one solution. writing a mlv-importer for AE/premiere, since its the same SDK-Base
* importing the filesequences is a bit annoying, because you have to mark sequence by sequence. have to figure out, if premiere can cope with kind of containers, marking all pathes, thats it.. (would be solved with the importer :))
* but finally premiere works well with the data, i had no pink highlights, maybe i didnt overexposured :)

(vimeo) 1280*448 at 50fps - down to 23.976fps

(youtube) 1920*1080 at 23.976fps

Exquisite quality! But what about those green edges?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 15, 2014, 07:55:16 PM
it doesnt work in resolve

CinemaDNG + wav from Blackmagic Design cameras work. So it's probably just a question of correct filename convetions and maybe some metadata.

@chmee -- I can provide you some BMD footage for the "reverse engineering" if you are interested.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 15, 2014, 08:09:31 PM
@kaco (for sure) you can help with the BMD naming conventions:

Pathname
+--->dng-filenaming
+--->wav-filenaming

ill try to rewrite/implement the filenamegenerator, as every can construct this filenaming-convention for himself :)

@tony in both videos? please tell video/timecode..

ahh. PLEASE DONT FORGET TO GIVE ME SOME INFORMATIONS ABOUT THE CRASHES. (pastebin the xml-files) its difficult to find this bloody bug. i tried it on 4 pc's here at home, it does not crash :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: CF on April 15, 2014, 10:31:52 PM
The app opens, I then proceed to drag in a .RAW file... I click convert, the filename turns RED and the app crashes with below log

Here is my Crashlog:

  Problem Event Name:   CLR20r3
  Problem Signature 01:   raw2cdng.1.4.9.7-2.exe
  Problem Signature 02:   1.0.0.0
  Problem Signature 03:   532cb395
  Problem Signature 04:   raw2cdng
  Problem Signature 05:   1.0.0.0
  Problem Signature 06:   532cb395
  Problem Signature 07:   e1
  Problem Signature 08:   352
  Problem Signature 09:   System.NullReferenceException
  OS Version:   6.1.7601.2.1.0.256.1
  Locale ID:   1033
  Additional Information 1:   0a9e
  Additional Information 2:   0a9e372d3b4ad19135b953a78882e789
  Additional Information 3:   0a9e
  Additional Information 4:   0a9e372d3b4ad19135b953a78882e789

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 15, 2014, 10:36:53 PM
for this version (1.4.9.7) enable the debuglog and link it via pastebin. please do a test with 1.5.0.beta (linked some entries before) - its important to pastebin the crash-xml-files from the temp-path (if it crashes :) )
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on April 16, 2014, 04:33:01 AM
Gave it a go again to see if I could get it work, still crashes but now when click on the batch window header bar it crashed right away with out any files.
Try it  for my desktop, My downloads folder & even tried from my 2nd SSD drive on the root in my laptop. Crashed when I click on the batch window header bar, but
then I tried it off my USB 3.0 4TB raid "0" box and it fired up loaded a file I could see the preview play the file just like you should but crashed when I tried to convert it.
The "outputPath":"D:\\New folder. So I inputted a unique name this time and it work, :o  "MLV 5D2" , So do you Need to input a name first of it will crash ? I guess so
I would have thought it would just use the file input name. The App & test file was on USB 3.0 drive in a root folder & output path was to the 2nd SSD drive on the root folder. Just run it from my 2nd SSD drive and ok now a long as you have a unique name in that field, but it will crash if there is now file in the batch window and you click on any thing on that side. Well at least it work just need to get a few bugs out.  :D


Edit:
Code: [Select]
- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider Name="Windows Error Reporting" />
  <EventID Qualifiers="0">1001</EventID>
  <Level>4</Level>
  <Task>0</Task>
  <Keywords>0x80000000000000</Keywords>
  <TimeCreated SystemTime="2014-04-16T02:27:51.000000000Z" />
  <EventRecordID>27556</EventRecordID>
  <Channel>Application</Channel>
  <Computer>i7Toshiba</Computer>
  <Security />
  </System>
- <EventData>
  <Data />
  <Data>0</Data>
  <Data>CLR20r3</Data>
  <Data>Not available</Data>
  <Data>0</Data>
  <Data>raw2cdng.1.5.0.beta.exe</Data>
  <Data>1.0.0.0</Data>
  <Data>534871ee</Data>
  <Data>mscorlib</Data>
  <Data>4.0.30319.18444</Data>
  <Data>52717edc</Data>
  <Data>24d</Data>
  <Data>18</Data>
  <Data>System.ArgumentOutOfRange</Data>
  <Data />
  <Data>C:\Users\i7 Toshiba\AppData\Local\Temp\WER4359.tmp.WERInternalMetadata.xml</Data>
  <Data>C:\Users\i7 Toshiba\AppData\Local\Microsoft\Windows\WER\ReportArchive\AppCrash_raw2cdng.1.5.0.b_bef85dffdf135db157af789b841bb5ef8296687_291a4a4c</Data>
  <Data />
  <Data>0</Data>
  <Data>b37f3eb0-c50e-11e3-8dcb-6036ddb07e2f</Data>
  <Data>1</Data>
  </EventData>
  </Event>
XML error reporting also getting "System.NullReferenceException" so times
Win 7 Ultimate SP1, hope this helps
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 16, 2014, 05:24:46 AM
Problem signature:
  Problem Event Name:   CLR20r3
  Problem Signature 01:   raw2cdng.1.5.0.beta.exe
  Problem Signature 02:   1.0.0.0
  Problem Signature 03:   534871ee
  Problem Signature 04:   mscorlib
  Problem Signature 05:   4.0.30319.18063
  Problem Signature 06:   526766b5
  Problem Signature 07:   24d
  Problem Signature 08:   18
  Problem Signature 09:   System.ArgumentOutOfRange
  OS Version:   6.1.7601.2.1.0.768.3
  Locale ID:   1033
  Additional Information 1:   0a9e
  Additional Information 2:   0a9e372d3b4ad19135b953a78882e789
  Additional Information 3:   0a9e
  Additional Information 4:   0a9e372d3b4ad19135b953a78882e789
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on April 16, 2014, 06:11:02 AM
I import the new cdngs 16bit maximize to my Mac and they have a heavy  green over cast , but the 16bit liner look normal  in the mac preview.
Tried a direct import in SpeedGrade on the Mac and heavy green cast which I seem not be able to cleanup, but with dynamic link from Premiere Pro it look ok.
A.E. look ok either way but with a gamma shift up on the 16bit maximize, but in Apple Products - Motions, Compressor, FCPX the 16bit maximize file option Will not work as a Heavy Green Cast over the image. So in Apple products use "non 16bit maximize". Never tried the 12bit thou so may be better I will have to test that later.

Edit: 16bit maximize look Ok in Photoshop on the Mac.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 16, 2014, 08:44:21 AM
thanks a lot. thats something i can work with.

btw. with speedgrade there was a greencast before, you have to use/tweak a .look file with values for the colormatrix (example in 1.4.5 zip). anyway, thanks, i will look on that as well.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: duey101 on April 16, 2014, 04:45:19 PM
Hi chmee,

I've been away on holiday for a while, just got my 5D Mark III and I am happy to report that all previous issues I had in the past are now gone.  No hot/bad sensels or anything else.  MLV/RAW files import perfectly into PPro CC iwth no color cast issues so far.  I only shot a couple of things in the last day or two since I got my camera but so far it all looks good.  I'll do some more testing and report back.

The interface it great and it work a treat, fast too.  Great work!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 16, 2014, 05:06:42 PM
i've seen on my today's tests, the pink highlights are still there in 16bit maximized, but the pink-highlight-button does his job quite well, the general brightness does not change. so, avoid massive highlights while recording :) (and we're waiting for the adobe-update)

(coding on beta2 1.5.0, found some silly bugs, leading to these crashes i think - today or tomorrow)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 17, 2014, 09:19:54 AM
raw2cdng 1.5.0.beta2 online
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fraw2cdng.1.5.0.beta2.jpg&hash=b504e208f1db8b56e62b796f09d142e8)

test and tell. if this beta2 crashes on first start, try it a second time (on first start it writes the settings-file, i think, on todo-list)
(some buttons are light transparent, because not yet implemented)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 17, 2014, 10:09:24 AM
i drag and drop 25 mlv files and converted all 25 files succesfully. in your older version when convertion is taking plce the particular file is highlighted with green color which makes easier for us to know how many files are converted and how many files are stil left. it would be great if you can stil add that feature but not a big deal we can open the save path folder and see. i wil do more test and report if i find any problem. i cant wait to test the chroma smoothing as well. this application is simply amazing
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 17, 2014, 10:30:18 AM
:) ah. yes. yesterday i still thought about it :) on todo list.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: tonybeccar on April 17, 2014, 10:57:29 AM
Oh my god, amazing!!!! Working smoothly importing 30 mlv's. Filenaming is amazing now! Just don't click any item on the list while converting, it will crash.. hehe. Maybe disabling the interface while converting? The preview is amaaaazing! Working flawlessly now!!!!

Just, WOW.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 17, 2014, 12:14:20 PM
lol, oh yeah, disabling gui-events while converting.. on todo-list. thx.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 17, 2014, 12:22:58 PM
the new interface is perfect. great job
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: DavidSh on April 17, 2014, 12:37:14 PM
Looks so good, too bad i am on a mac :(
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: visceralpsyche on April 17, 2014, 12:39:39 PM
Thanks for your efforts Chmee :)

I just tested beta2 and am still seeing the vertical lines in the DNG files.

Currently RAWanizer 0.6.1 (the one I've been using up until now) exports without the vertical lines, but I'm limited to shooting RAW v1 in camera and it doesn't create CinemaDNG files (which are awesome by the way - thanks).

So hopefully you can figure out the issue because it would make your program the number one choice for ML RAW conversion!

Cheers from Tokyo,

Paul :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 17, 2014, 12:46:32 PM
vertical banding and chroma smoothing are under gpl-license. i have to talk with @g3gg0/@a1ex/@1% about using their code, because in this case i'm too lazy to solve it myself.. but, they're mostly recoded and inside :)

@DavidSh As i've read, ThomasWorth is preparing his new RawMagic-Version.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: a1ex on April 17, 2014, 01:31:23 PM
I'm OK with you using my code for free software (http://www.gnu.org/philosophy/free-sw.html) under the terms of GPL (nothing to do with price).

Maybe you can tell Thomas Worth too; he is currently using my GPL'ed code in his proprietary app.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: duey101 on April 17, 2014, 03:28:54 PM
This is great chmee! (1.5 beta)  Superb program and interface, no problems here just good CDNG files to import direct to PPro CC, no vertical banding or pink highlights, thank you. 
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 17, 2014, 03:32:02 PM
vertical banding and chroma smoothing are under gpl-license. i have to talk with @g3gg0/@a1ex/@1% about using their code, because in this case i'm too lazy to solve it myself.. but, they're mostly recoded and inside :)

@DavidSh As i've read, ThomasWorth is preparing his new RawMagic-Version.

I'm OK with you using my code for free software (http://www.gnu.org/philosophy/free-sw.html) under the terms of GPL (nothing to do with price).

Maybe you can tell Thomas Worth too; he is currently using my GPL'ed code in his proprietary app.

If I understand this correctly, does this mean that we won't have vertical banding correction soon?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 17, 2014, 10:09:31 PM
nope, it means, if i "free" my software, i can use other gpl-licensed sourcecode as well. (so give me one or two days)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 18, 2014, 02:14:31 AM
it will be great that we will be using only one application after implementing vertical bending and chroma smoothing. for now i m using atleast three applications in my workflow
1.convert to raw
2.remove pink dots
2.raw2cdng
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 18, 2014, 09:59:13 PM
so. i've done a git with the sourcecode. therefore its now free software (this software will never cost a dime, its up to you to donate me some cents). i hope, some of you will help me to optimize code, to put some new options into it and to still let me learn the next stage of coding :) tried to comment most points.

https://bitbucket.org/chmee/raw2cdng

(i'm now a little bit tired, have to take my time for my children and for two other projects)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Nautilus on April 18, 2014, 11:32:46 PM
so. i've done a git with the sourcecode. therefore its now free software (this software will never cost a dime, its up to you to donate me some cents). i hope, some of you will help me to optimize code, to put some new options into it and to still let me learn the next stage of coding :) tried to comment most points.

https://bitbucket.org/chmee/raw2cdng

(i'm now a little bit tired, have to take my time for my children and for two other projects)

regards chmee

hey chmee,

Is it possible for you to integrate a pink dot removing feature? I love the fact that cdng files can be directly imported into premiere pro thus simplifying the workflow. But i can't benefit from that because my 650D procuses raw footage with pink dots so i have to use g3gg0's raw2dng to convert from raw to dng and then import into AE and then into premiere... That takes too long and some quality lost in the process.

In this thread: http://www.magiclantern.fm/forum/index.php?topic=5404.0 g3gg0 released a raw2dng executable that does chroma smoothing which removes pink dots as well.

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 18, 2014, 11:37:00 PM
please read some more thread entries. the last 10 will give you the answer.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Nautilus on April 19, 2014, 12:36:57 AM
please read some more thread entries. the last 10 will give you the answer.

ok my bad. i hope you solve license related issues soon.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ted ramasola on April 19, 2014, 12:40:59 AM
ok my bad. i hope you solve license related issues soon.

If you read the previous posts, it actually said there are non.  :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Nautilus on April 19, 2014, 12:48:38 AM
If you read the previous posts, it actually said there are non.  :)

Sorry i couldn't see that part. He says he has to ask alex, g3gg0 and others to use their codes. And Alex replied it's ok.

Quote
nope, it means, if i "free" my software, i can use other gpl-licensed sourcecode as well. (so give me one or two days)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 19, 2014, 12:52:37 AM
:) you quoted the interesting part..
Quote
..so. i've done a git with the sourcecode. therefore its now free software ..
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 19, 2014, 12:34:28 PM
so. i've done a git with the sourcecode. therefore its now free software (this software will never cost a dime, its up to you to donate me some cents). i hope, some of you will help me to optimize code, to put some new options into it and to still let me learn the next stage of coding :) tried to comment most points.

https://bitbucket.org/chmee/raw2cdng

(i'm now a little bit tired, have to take my time for my children and for two other projects)

regards chmee

Bravo! Thanks a lot! I hope will see the vertical banding checkbox soon! :))
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on April 19, 2014, 11:19:09 PM
i dropped 4 .raw files, each 6-11gb. The converter crashed. something needs to fixed? thanks
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 20, 2014, 09:52:44 AM
better informations lead to better solutions.

WHEN does it crash? while dragdrop? while preview? while convert? Does it affect one file? did you tested, if maybe one's corrupt?

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: painya on April 21, 2014, 12:32:01 AM
Chmee, here is an issue I have encountered with my RAW video. Looks to be aliasing? Note: other shots do not see this.
Details:
50d
Lexar 1000x (new)
1568x882
Processed individually
Running CDNG version 1.4.9.7
Directly imported into Premier
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.upload.ee%2Fimage%2F4013824%2FM201109F_C0001_B16._000000-000246_.dng.Still001.jpg&hash=940b557ecd3d0002c770f656f1f22bb5)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: painya on April 21, 2014, 12:34:19 AM
Here's the h.264 equivalent if you are at all interested.
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.upload.ee%2Fimage%2F4013827%2FM201109F_C0001_B16._000000-000246_.dng.Still002.jpg&hash=342258a95817a2ee44a83dd1c95c160f)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Frank7D on April 21, 2014, 12:56:12 AM
@painya,

DSLR video is subject to aliasing due to line-skipping. RAW video does not negate this issue. The only way you can avoid it is to use crop mode.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: painya on April 21, 2014, 12:57:58 AM
@painya,

DSLR video is subject to aliasing due to line-skipping. RAW video does not negate this issue. The only way you can avoid it is to use crop mode.
Right! Thank you very much. I forgot that it had to be in crop mode to fix that.
Silly me.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: tonybeccar on April 21, 2014, 02:50:05 AM
Right! Thank you very much. I forgot that it had to be in crop mode to fix that.
Silly me.

Aliasing and moire are much more aggresive because you're shooting at a lower resolution than it's max also!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: painya on April 21, 2014, 04:01:46 AM
I was shooting at the 50d's max no-crop resolution for RAW.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on April 21, 2014, 01:04:49 PM
chmee, it crashed while converting the .raw files to cdng.. i did nothing, did not even try to preview. Just drag and dropped, clicked convert, it processed for a few seconds and got crashed, the 1.5.0 beta 2 version
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: rrrmusic on April 22, 2014, 02:34:43 AM
The application crashed with new mlv nightly builds (Canon 5d Mk3 113). Tried to convert a old mlv from March 15 and worked fine. The new MLV that i recorded with recently nightly builds the application crashed with only drag and drop.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ted ramasola on April 22, 2014, 03:57:11 AM
Though I was able to convert all my 48 mlvs to cdngs heres my notes on v1.5.0beta2

1. Took me 2 times to initially start the app. 1st it crashed.

2. I was able to drag n drop all 48 files but while converting I was moving my mouse over in active buttons and it crashed at the 45th file.

3. I repeated the process this time I left it alone, it successfully converted. But it has a new name structure which presents a problem for me in resolve.
with v 1.4.9.6 the files can alternate dng and wav in the media bin in resolve making it easy to click the audio link button to sync dng and wav files. This version however will not display in alternate with dng and wav.

I look forward that you will finally be able to link the wav and dngs with your upcoming version.

Best of luck!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 23, 2014, 04:46:44 PM
minutes ago i put a beta3 (http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/) on the server.

done:
* wav files have now an according timecode for resolve fi
* some more infos in debug-log
* some more variables for the filenamegenerator. [D2] is interesting for resolve filenames
Code: [Select]
[C]_1_[D2]_C0000(_)
found a bug seconds ago:
* an "Argument out of Range" exception leads to crash. investigating right now..
* inconsistency with raw-files - if they re wrong shown in preview, they will be converted wrong.
* (fixed. will be in beta4 - if there are more than one .raw, all .raw's take the blocklistdata of the last one, silly bug..)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ted ramasola on April 23, 2014, 09:34:28 PM
Chmee!!!!

AUDIO AUTO SYNC NOW WORKS!!!!  AAARRRGGGHHHHH!!!



Thank you!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jakobmen on April 23, 2014, 09:51:10 PM
Chmee!!!!

AUDIO AUTO SYNC NOW WORKS!!!!  AAARRRGGGHHHHH!!!



Thank you!

how you doing this lol ?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: dubzeebass on April 24, 2014, 01:54:02 AM
AUDIO AUTO SYNC NOW WORKS!!!!

Confirmed and replicated :-)

Great job, chmee! Now if only I could make it work under Wine.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ted ramasola on April 24, 2014, 02:02:50 AM
how you doing this lol ?

In Resolve, Autosync the files in media bin
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jakobmen on April 24, 2014, 02:08:11 AM
What's the workflow? We talking about resolve?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ted ramasola on April 24, 2014, 02:17:26 AM
after the cdng extraction using beta3, open resolve.

Drag the folder containing all the extracted dngs and wavs into the Media Pool.

In the Media Pool right click on Master folder, or if you crated a New Bin right click the Folder containing the footage files.

Choose AutoSync based on timecode.

In Edit Tab, Create New Timeline, uncheck create empty timeline, all your footage with linked audio shows up on new timeline.

Sit back a while and smile.

Go back online and click donate button on chmee's site.  :P
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 24, 2014, 08:00:14 AM
@dubzeebass
[linux, wine] do you have some info for me? i dont have any linux based pc here.. 
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: gideonplus on April 24, 2014, 10:28:26 AM
when using Black Magic camera CDNGs, the audio snaps automaticly
and any addional action is not requaierd.
is that the case for MLV DNGs when using v1.5.0beta3 ?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 24, 2014, 11:03:22 AM
nope, i didnt have found the key for that. but it s a big step into the right direction, it syncs accordingly, with just one or two more clicks. and i assume, its something about the wav-file, because, if you delete the wav file, it changes its behaviour in the mediabrowser and the path changes into a mediafile.

please link some more BM-Mediasequences with audio, so i can analyze it.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Metsadah on April 24, 2014, 11:10:58 AM
I see this app also fixes the pink cast in premiere cc. Is there no mac version of this program?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 24, 2014, 11:46:09 AM
you'll have to wait for
RawMagic-Update
and/or
the next adobe update (things will be better)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Redrocks on April 24, 2014, 05:28:24 PM
@chmee have you abandoned proxies? I know Premiere is switching to better support for dng, but even a top end workstation is going to struggle with projects over an hour in length and it would be great to have proxies for editing. The only codec I can find that does not require previews in Premiere is .m2v, which I can create in AE but that means going through ACR with every clip to get a proxy...
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 24, 2014, 05:46:52 PM
would be
* jpg filesequence
* with half resolution
* not color accurate
ok?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on April 25, 2014, 06:19:44 AM
Test out the beta3 , every thing work perfectly not crashes . The best results for Premiere Pro yet with 16bit maximized Cdng's , Hi-Light were handled very will &
with the "Auto Level" filter it look dam good ! I would say at this point I can go mobile with my i7 PC Laptop and a small USB 3.0 raid 0 box  and  rough edit on the road or small project with only a few hours for a turn around (e.g. new stories current events etc...) , being I still use FCPX as my main NLE , but now I have a very good second choice.
At least now I can use more that A.E.cc and get my money worth out of my subscription from adobe :D
Thanks chmee  :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: evanbuzzell on April 25, 2014, 08:26:02 AM
had a crash with a large amount of files being imported.

The debug doesn't say much, just that where it stops. It can't seem to load a file that is 11gb. It just doesnt do anything when I try to add it to the batch on its own.

**Update* But it loaded a 5 file one just fine. I wonder if that file is corrupt now that I'm looking

I'm on 1.5.0 beta3 btw
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: piloui on April 25, 2014, 11:46:35 AM
There is gold every day on each topic of this forum ! I'm going crazy !
But wait, what's the difference between DNG and CDNG ?
How can cdng sequences playback be so smooth in premier pro cc ?

@chmee, you make the whole process much more easier. No offline editing and intermediate rendering any more.

There is some bugs of course for a beta :
Crashes during extraction
sometimes the destination path is not working and no way to know where the dngs are extracted. I'll try a long extraction with many MLV.
CDNG 16 bit make premiere pro cc crash during playback. CDNG 16 bit maximized are all right.

EDIT : no crashes at all with .MLV files queue. Drag and drop multiple RAW not working. Drag and drop multiple MLV working. When drag and drop multiple file, the loading time seems a bit long, but maybe it's normal.

Thanks !

win 8.1 x64, premiere pro cc, updated.
ml 63b2f145cb3b 2014/02/14.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: evanbuzzell on April 25, 2014, 02:03:39 PM
audio not syncing in resolve as well
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Redrocks on April 25, 2014, 04:41:58 PM
@chmee JPEGs still produce a yellow bar for me, so I don't want to use them for proxies for the same point Reddeercity raised: Premiere doesn't need to create previews for .m2v (MPEG2 format in output module of AE), so I can edit anywhere on a basic laptop with Premiere installed. All I need is a way to get from MLV to MPEG2. Thanks all the same though, perhaps others would be happy with JPEGs.

I've tried the last few versions of your app and kept getting crashes. Beta3 is a lot faster and seems to be using most of my computers raw power, but I can't drag the folders into AE like I always have. Tried with the same MLV file ran through RAW Video Converter 1.9.1 and had no problem.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 25, 2014, 06:32:11 PM
mates, thank you for testing and response.

* RAW-Error is fixed, will be in beta4, found it days ago.
* Resolve-Sync - i tried to realize it, it seems to work. @evanbuzzell please check, if both mediafiles have the same timecode - this was my first intention - the second one was, to change the path/files behaviour into the regular path-as-mediafile. i didnt found the key for that, i need more samples, i said it before..
* @RedRocks. ok. in beta4 there will be a proxy-jpeg-output, as i stated, they are not color accurate, have a wrong gamma, but for simple cutting it should do. and: ffmpeg has implemented mlv, this could be your way (maybe mine too for other codecs).
* ah, btw. do you use mercury-playback on gpu? my yellow marked sequences play quite fluid, even if i use gpu-accelerated effects.
* clearly saying, my intention was/is to get the user as fast as possible editing his stuff
* @piloui "sometimes destinationpath not working". only under raw or with mlv as well?

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: piloui on April 26, 2014, 12:34:06 AM
MLV workflow seems to be quiet stable indeed. No crash, no destination path bug. Actually the only limitation i see comes from adobe  ::)
Cannot drag n drop multiple files in the bin like in resolve. For 100 shots, you have to do 100 drag n drops. Can't belive such obvious thing are missing.
Great job chmee. Any tips concerning loading time when dropping several MLV in raw2cdng ?

Some ideas :
it would be great to have some highlight of the current mlv extracted, it's purely visual.
Maybe some button to pause, stop, resume.
And a great thing you never find in batch processing software : the magic check box "turn off computer when finished"
Just some ideas, the tool as it is is already a gift !
;)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 26, 2014, 01:34:57 AM
thanks a lot.

* the loading time on drag/drop is because of reading all attributes of every file, how many frames are there really, is there audio, creating frame-lists for the preview, lensdata and so on.

ideas:
* highlight active converting-file - its already on the todo-list - as it was already in the older versions..
* "turn off computer when finished" really? ok.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Redrocks on April 26, 2014, 04:12:38 PM
Thanks Chmee, I'll try the JPEGs and see how that works out. I do have gpu enabled for PP & AE, running through a 3 GB GTX 660 Ti. When I first open a project yellows are fine, then I get 20 mins down the timeline and things get laggy. A project of .m2v seems to run smoother than any other codecs I can spit out of AE. Like you say, ffmpeg will hopefully give us more options.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: piloui on April 28, 2014, 03:15:32 PM
Files naming tool is great too. date of creation is a really good way to differentiate shots. However, date of creation, under window, changes when you copy the file. This means, if you shoot let say 10 MLV, on a 3 days period, they will all have, on CF card, a different creation date.
BUT
If you copy all of them at once ( as everybody do i hope ^^ ) they will all get an new and similar creation date with a little difference.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.cinemersion.fr%2Fimages%2Fdivers%2Fmlv_date.jpg&hash=a84dce4ba62143e31ddec8586a6abe84)

The MODIFICATION DATE always stays the same. Weird, for me, creation date would be the shooting time, and modification would be, copy, export etc ... Try figure :/ windows world !

Anyway, maybe an other tag option like [DM] "modification date" would be cool.

Thanks
;)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: piloui on April 28, 2014, 04:32:37 PM
Actually the only limitation i see comes from adobe  ::)
Cannot drag n drop multiple files in the bin like in resolve. For 100 shots, you have to do 100 drag n drops. Can't belive such obvious thing are missing.

There is a simple solution for several cdng sequences to drop in premiere pro in one move :
Go where your DNGs are excracted. You should have all your folders containing all your DNGs. In the search bar, top right corner, type  *000.dng
This should list all but only your first DNGs from all your sequences.
Select all of them, drag n drop and bim bam, all your sequences in premiere in one drop.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jayzed on April 28, 2014, 05:45:47 PM
OK. Another, no doubt often asked question (I couldn't find anything in a search, though). I have many hundreds of DNG files from the last few months. Now that CHMEE's lovely CDNG tool is the latest answer to my latest prayers, I was wondering how to convert these DNGs to CDNGs, as CDNGs are more compatible with a wider range of video apps and I'd like to use them as my standard workflow thingies. I've deleted all my MLVs, and before them all my RAWs so I can't go back to the source.

Any ideas?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: revast on April 29, 2014, 03:36:11 PM
@dubzeebass
Quote
Great job, chmee! Now if only I could make it work under Wine.
here you go: http://pastebin.com/SUHCXm8X

@chmee
Quote
[linux, wine] do you have some info for me?

Plenty of infos about raw and linux here (http://www.magiclantern.fm/forum/index.php?topic=9335) collected by me..
also how to run your program via wine...

WINE is acutally and opensource implementation of the windows api, thus a layer of windows libraries upon linux. That has the advantage to be able to run stuff in native speed, as it's not emulated. The disadvantage against a crossplatform raw2cdng aparrt from the obvious is that it needs plenty of space, 600+mb with dependecies, and then in wineprefix ( a bottled environment where a windows system is presented to the application) is annother 500mb, with dontnet4 installed etc, just for running a little program like yours; also, it is not guaranteed that programs will work at all or with every version of wine, or possible that changes you make to your program would then also break it in wine.

One can ship around this by using a dedicated wine version from playonlinux, just for one app, and set it up so that everything a windows program needs to run is selfcontained in one folder, but it's quite a job to do that.


Hence my question - Would it be possible to go crossplatform with raw2cdng, if it's just to build against mono? I have not looked at your code if it's that easy..


 
Quote
  I dont have any Linux based pc here.. 
In terms of developing, I guess you don't have to...
If you would go for crossplatform and build raw2cdng against mono, the .exe build for windows should run on Linux/Mac, too
g3gg0 with MLVViewSharp e.g. does it that way.

Normally, a virtual machine would suffice, e.g Virtualbox as portable solution:
http://www.vbox.me/ (portable has the advantage that you are not polluting your system at all)
and a ready to use image:
http://downloads.sourceforge.net/virtualboximage/xubuntu_1204.7z
(username/password): xubuntu/reverse

Thats only one solution, as Linux also runs form usb-stick, and it even saves the changes you do on the stick then (e.g install mono, in terminal, enter "sudo apt-get install mono-runtime, libmono-corlib2.0-cil libmono-winforms2.0-cil" ) ...
If you want to try that, I would recommend unetbootin, quite bulletproof thing (check persistence)








Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 29, 2014, 04:37:56 PM
@revast
thanks a lot for the short introduction. its time to get on this ship because of coding for raspberry and other linuxbased things, this sounds the right way. i'll try my best..

please some patience for beta4, next days..
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: rrrmusic on April 29, 2014, 05:03:46 PM
Hi
This weekend I have a short that I want film with magic lantern mlv format. Recently i have problems with latest nightly builds Canon 5D MK3 1.1.3 using raw2cdng 1.5 b3 with constant application crashes.

Maybe this is a question for chmee very grateful and congrats for your great job with the raw2cdng tool but i want to know what nighly build you test that is working better or more stable with mlv format using raw2cdng 1.5 b3.

Please let me know
Thanks
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 29, 2014, 06:15:44 PM
i had to look into my download-directory :) it was a version from april 14. but, i m no reference for that, i ve got a bunch of mlv's i recorded some day and i use them to check my code..

@piloui thanks, will implement it, because its the more useful date :)

@jayzed sorry, didnt thought about it, and by now i wont. lack of time.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on April 30, 2014, 08:24:24 PM
friends I could not render out the footage correctly from Resolve. The audio gets jumbled and only one out of two different footage is rendered. I shot using .mlv with audio enabled, beta3 converter, 5dIII, march 2014 update. The footage preview was good, had syncing audio in mlrawviewer. I also ticked "autosync based on timecode" in resolve... anything wrong that i ve done? also the last audio file is slightly lengthier than the video file.. thanks
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: artyg on April 30, 2014, 10:41:44 PM
Go back online and click donate button on chmee's site.  :P

Alredy done:) Chmee really cool!!!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on April 30, 2014, 11:16:27 PM
[F] mode of naming had the problems i previously posted. But changing the naming mode to [F]_[D] cleared the problem now...

@chmee.. thanks for the great application. Could you also please check whether the flickering issue in Resolve is due to the converter? because 1.4.9.7 version had no flickering issues after rendering the color corrected output, however there were audio syncing complaints which was rectified in beta 3. But, in beta 3, I get the flickering issues in resolve once again. Audio syncing is perfect. Roundtrip is good except that i have to manually link the footage, as usual. Because the flickering was absent when i used cdngs from 1.4.9.7 and present when i use beta 3, i think there is an association. Pls see to that friend... thanks
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on April 30, 2014, 11:44:11 PM
hi again, I used the naming code [F]_[D] in beta 3 converter. Selected cdng 16bit maximized. Did a roundtrip in resolve and premiere cc. Worked fine, there was no flickering issue in the color corrected outputs. Thanks.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 30, 2014, 11:59:03 PM
@anandkamal good to hear.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: evanbuzzell on May 01, 2014, 09:14:41 PM
mates, thank you for testing and response.

* RAW-Error is fixed, will be in beta4, found it days ago.
* Resolve-Sync - i tried to realize it, it seems to work. @evanbuzzell please check, if both mediafiles have the same timecode - this was my first intention - the second one was, to change the path/files behaviour into the regular path-as-mediafile. i didnt found the key for that, i need more samples, i said it before..
* @RedRocks. ok. in beta4 there will be a proxy-jpeg-output, as i stated, they are not color accurate, have a wrong gamma, but for simple cutting it should do. and: ffmpeg has implemented mlv, this could be your way (maybe mine too for other codecs).
* ah, btw. do you use mercury-playback on gpu? my yellow marked sequences play quite fluid, even if i use gpu-accelerated effects.
* clearly saying, my intention was/is to get the user as fast as possible editing his stuff
* @piloui "sometimes destinationpath not working". only under raw or with mlv as well?

regards chmee

So the timecode for the audio and the video are different. I could have sworn that the audio files were synced in Premiere just a few days ago and now I'm not seeing any audio sync in premiere or resolve. This just made this project a million times harder aye! I've tried changing the file names to something different when exporting, any other ideas?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 02, 2014, 12:18:26 AM
@evanbuzzell
re-check: what's the framerate for this videos? please write some video/audio timecode pairs, if theres maybe a correlation.. is the audiofile playing fine in any audio/mediaplayer?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: evanbuzzell on May 02, 2014, 12:59:15 AM
@evanbuzzell
re-check: what's the framerate for this videos? please write some video/audio timecode pairs, if theres maybe a correlation.. is the audiofile playing fine in any audio/mediaplayer?

23.976 on all the clips.
Video length is 02:18:10, audio is 02:17:38400
V: 01:12:01, A: 01:11:38400
V: 01:43:07, A: 01:42:19200
V: 02:13:11, A: 02:13:00000

Different day, More Recent than most:
V: 00:23:06, A: 23:09600
V: 02:12:18, A: 02:12:00000
V: 03:13:07, A: 03:12:09600
V: 00:46:12, A: 46:28800

Most recent build:
V:00:21:16, A: 21:28800
V: 00:39:07, A: 39:19200
V: 05:43:03, A: 4:25:28800
V: 02:08:05, A: 02:07:38400

older one
V: 00:28:23, A: 28:19200
V: 00:56:17, A: 57:00000
V: 00:51:23, A: 51:19200

The audiofile plays fine for the most part, it is the sync that is the issue. 5d2 is the camera I'm using apr 23, 14
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 02, 2014, 11:40:58 PM
the timecode (first frame) is made of the creationtime of the file (lastWrite is the better attribute, its coded already, will be in beta4). does one of the both timecodes match to the filetime? could you send me examplefiles to raw2cdng@phreekz.de? (just one dng and the audiofile) i have to code some more debug and helper-things for those problems.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Wartburg-Entertainment on May 03, 2014, 04:51:40 PM
Hello,
how can it be that my 1.5.beta3 makes no CDNG,
but nut DNG?
At least recognizes Premiere not the files.

greeding
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 03, 2014, 06:37:31 PM
@wartburg no, they should 100% work. maybe you're looking into the wrong tab/window in premiere :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on May 04, 2014, 02:40:24 PM
chmee, i was telling about the flickering issue in resolve. Now when i turn "delta pink highlights" in raw2cdng and process further, i get flickering issues during roundtrip between resolve and premiere. When i turn the delta pink highlights off, the flickering issue solves... Just wanted to say the converter influences everything (an advantage of course) ;) With [f]_[d] naming, 16bit maximized and everything else turned off, the roundtrip is good without flickering in resolve. Of course i had to link the media pool manually (problem between adobe and resolve :()
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 04, 2014, 03:34:21 PM
the pink highlight-button is ONLY for direct editing in Premiere. you dont need it (i think) if you start and end in resolve. the premiere roundtrip is finally only for editpoints.

but: the interesting point is, if its flickering:  where does it flicker? in resolve as well? in premiere? what data/recodring mode you're using?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on May 04, 2014, 07:03:12 PM
I use .mlv files for processing. Latest build 5d3 113. Beta 3 converter.

Flickering is caused by Resolve. Flickering is like lightning occurring on the whole frame (not at the top or bottom, but as a whole). The color correction does not apply to certain frames and those frames show, among the other colored frames, like flickerings or lightning. This is seen in the rendered output and obviously when roundtrip-ped back into premiere.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 04, 2014, 07:15:02 PM
irfanview is able to render dng's, please look into the folder, if you find those bad pictures. send it with another functional dng to raw2cdng@phreekz.de
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on May 04, 2014, 08:06:04 PM
thanks, with irfanview, cdngs are fine. But the flickering issue is NOT while importing the cdngs. The problem arises after I do color corrections (in playbacks & rendered outs). The color changes won't apply to certain frames. For ex: among 700 frames, the erroneous frames are around 300 or something. They keep on blinking indicating that the color corrections have not been worked on those 300 frames. I found that it has to do something with raw2cdng. When certain parameters are changed in raw2cdng, the flickering reappears. Every time (during updates or whatever), I optimize the roundtrip - resolve is a headache :(

The flickering is a very common problem in resolve (i googled it) and the below link shows the same problem that I face:

There are so many reasons for flickering. Mainly, it is related to frames and frame rates, i think. With older versions of raw2cdng, when I faced problems in 23.976 conversion, there was flickering. Now in beta 3, everything is fine, unless I stick to certain keys, like naming in [f]_[d] and 16bit maximized options.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 04, 2014, 09:09:17 PM
please take acount into the timecode of "broken" frames. are they regular in distance or on a certain timecode/frame?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Ramon on May 04, 2014, 09:28:14 PM
please take acount into the timecode of "broken" frames. are they regular in distance or on a certain timecode/frame?

Hey, was going to test this converter.. but I can't seem to import .raw files :(
The beta3 and beta2 open fine, but when i drag drop a .raw file it gives this (https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fwww.oxygenxml.com%2Fdoc%2Fug-editor%2Fimg%2Fcursor_unavailable.png&hash=d00000cde9022e466ef5b548be8943b1) thing :(
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on May 04, 2014, 09:49:40 PM
I ll notice the timecode next time when it flickers. I think those are random flickerings based on the amount of color change that we do. I do not get flickering, what so ever, if I use LUTs and do nothing else. I face the problem only when i manually color correct the footage.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 04, 2014, 10:47:42 PM
@ramon this symbol should only appear if you drop on wrong position. strange.. maybe thats a common problem with dragdrop in windows sometimes? try other app-position? or start app as admin..
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 06, 2014, 09:39:19 AM
does anyone else have issues when trying to convert mlv files with numerous .001, etc (split at 4GBs due to using FAT32). I found beta3 isn't recognizing my media past around 3 files , any larger and the program doesnt see the clip at all ?? The application essentially did not recognize when I dropped the first MLV of the sequence of 8 files. However, If i drop a single MLV file or and MLV file with a single .001 file, it reads and converts no problem. Otherwise, it essentially does not see when I drop files with more "splits"? Any suggestions? Tried beta 2 and beta3.

Also, very excited for the new mlvtocdng :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: artyg on May 07, 2014, 01:03:23 PM
Hello Chmee! I test beta3 crashes with big files. I shot files from 250 frames to 6000 (4 minutes with 25 fps) and all OK! But with 8 minutes (12000 frames) program crash :( One time on 2900 frame, and one time on 3900 frame... this is very strange, because 4 minutes shot with 6000 frames convert ok... Somebody test beta3 with big files 8 minutes and more.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 07, 2014, 11:46:28 PM
thanks for info. please forgive, busy in jobs. the next beta will have a debug-button for better insight into the things happening..
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: arturochu on May 08, 2014, 06:26:08 AM
Just shot 376gb in dual-iso mode, ready to test new beta.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: artyg on May 08, 2014, 09:29:19 AM
Hmmm... with bild magiclantern-Nightly.2014May07.5D3123 and raw2cdng 1.5beta3 pink come back...
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi7.pixs.ru%2Fthumbs%2F7%2F2%2F6%2FM071324140_7884203_12020726.jpg&hash=d3c798bb3bef760bd9ee2777a67eed56) (http://pixs.ru/showimage/M071324140_7884203_12020726.png)

Then i use MLVViewSharp http://www.magiclantern.fm/forum/index.php?topic=8447.0 to convert to DNG and no pink appeare... Strange
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 09, 2014, 01:14:46 PM
@artyg
please send me a short MLV example to raw2cdng@phreekz.de. actually i'm a bit confused because of the actual changes regarding the blacklevel. i didn't saw those problems with earlier ml-builds..

https://bitbucket.org/hudson/magic-lantern/pull-request/484/black-level-fix/diff

please @all
the actual ml-builds are quite new (especially regarding the blacklevel-things), some of them (1.2.3 for 5DIII) are bleeding edge. dont change your system if you want to work productive. if you need them, use an older build, something between ~feb09(mlv-audio) and ~apr27.. i am still testing my code with a 5DIII 1.1.3 build from ~apr20, because its workin' and stable. you are not bound to use the newest builds!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: a1ex on May 09, 2014, 05:49:24 PM
The black level bug was introduced in February 11 (http://www.magiclantern.fm/forum/index.php?topic=10443). I am currently waiting for code review for the fix before merging it.

If April builds are any better than May ones (especially regarding black level), please show exactly where the difference is (bug report).
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on May 09, 2014, 09:52:11 PM
dose this problem refer to 5D3 Only ? as there has never been a problem with 5d2
but I'm still on Feb16th NB, no problems.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: a1ex on May 10, 2014, 07:28:07 AM
There are more causes for this bug.

The one I've talked about earlier only affects 5D3 and 60D (it appears when changing video modes and recording right away, and I could reproduce it and fix it).

It can also appear if a pink (corrupted) frame appears right when ML is computing the black level (though this one is quite hard to reproduce).

It can also appear because of some Canon code overwriting the raw buffer - remember that raw mode is just a debugging flag written by Canon, but they didn't design it for everyday usage. For example, in 500D, if you take a picture in LiveView and hold the shutter button for a little longer, the raw buffer will be overwritten by LiveView data and the raw zebras will be wrong. This has nothing to do with raw video, but it's a test case that can be reproduced (very useful for testing the fix).

It may also appear because of race conditions (I've tried to avoid it and make the raw backend thread-safe, but I'm not an expert in multithreaded programming, so I may have missed something).

The workaround from MLV (which is enabled by default - not by me (https://bitbucket.org/hudson/magic-lantern/commits/02c9af698af933dfea31212713d5216cb0626ff8)) is incorrect for 5D2 and many other cameras. The pull request removes the workaround and fixes the raw backend properly IMO. It checks the optical black area, so it should also detect bad black levels caused by a pink frame (though not with 100% chances).

On the pull request page, I've listed the relevant tests needed to make sure the fix is portable, and got zero feedback on these for about one week. I simply didn't have the time to do these early tests myself, and without code review and early feedback, I'm not going to include the fix in the nightly builds.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 10, 2014, 09:47:45 PM
Aside from the black level issue, I cant get the application to process MLVs from my 7D without it crashing. I've tried several sets of MLV files recorded at different times, on different cards, with different version of ML\TL. Is there anyway I can provide a log of source of diagnostic info for this? I have no other alternative mlv2cdng workflow, as this has worked so well

Many thanks!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 10, 2014, 10:18:33 PM
@jbowdach please be patient for 1 or 2 days, i will a provide a beta4 with more debug-infos (and some bugfixes)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 11, 2014, 01:16:22 AM
@jbowdach please be patient for 1 or 2 days, i will a provide a beta4 with more debug-infos (and some bugfixes)
@chmee many thanks, you are awesome! Please let me know how I can help provide any debug info
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 11, 2014, 10:52:53 PM
first: @a1ex it was not my intention to leave bad words on testing and generally on your work - understanding bugs in my code with a glitch on ml-side makes it very difficult to find the problem. so please @all it was no call-up to not test new builds.. your tests are of course necessary.

beta4 is online
* raw-bug (i hope) fixed
* more templates for filename generating
* proxy jpegs
* more debug infos in log (debug-button on top)
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 11, 2014, 11:21:42 PM
first: @a1ex it was not my intention to leave bad words on testing and generally on your work - understanding bugs in my code with a glitch on ml-side makes it very difficult to find the problem. so please @all it was no call-up to not test new builds.. your tests are of course necessary.

beta4 is online
* raw-bug (i hope) fixed
* more templates for filename generating
* proxy jpegs
* more debug infos in log (debug-button on top)
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

regards chmee

Thank very much, good sir! I am already testing it now to see if it fixes any of the crashing issues I had w beta3 and I have debug enabled so I can provide you debug info. Your work is VERY much appreciated as it's easily the best method for MLV -> DNG + WAV at the moment.

EDIT: Please find my debug log on pastebin below and issue files on bitbucket. It crashed as I was converted MLV files to 16bit CNDNGs (trying to avoid extra options so not using 12bit downconversion or any other options yet) -
Debug Log - http://bit.ly/1jw3160 (http://bit.ly/1jw3160)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on May 11, 2014, 11:24:05 PM
Thank chmee, Just downloaded the new version (1.5.0 beta 4) and with give it a work out  :)
One question thou, Is there a way of getting the normal 16bit Cdng as in Version 1.4.9 ?
I can't edit the Cdngs or use them in any Apple produces, as there are all pink cast (for Adobe)
but I like the new version with the file name generator etc.. . I can use the older version it fine but just wondering .  :D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 11, 2014, 11:44:29 PM
Thank chmee, Just downloaded the new version (1.5.0 beta 4) and with give it a work out  :)
One question thou, Is there a way of getting the normal 16bit Cdng as in Version 1.4.9 ?
I can't edit the Cdngs or use them in any Apple produces, as there are all pink cast (for Adobe)
but I like the new version with the file name generator etc.. . I can use the older version it fine but just wondering .  :D

off topic real quick:  thanks for your videos online. Helped me get started w the workflow when VERY few mlv videos working w sync sound seem to be online
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 11, 2014, 11:59:01 PM
@reddeercity  :o huh. didnt noticed such a glitch. please send me a wrong and a working dng. i'll do my best to get you into the position using the new gui :)

@jbowdach you as well, send me a short ( short! :) ) not functional mlv from your 7d to analyze it on my pc.

--> raw2cdng@phreekz.de

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 12, 2014, 02:21:54 AM
@reddeercity  :o huh. didnt noticed such a glitch. please send me a wrong and a working dng. i'll do my best to get you into the position using the new gui :)

@jbowdach you as well, send me a short ( short! :) ) not functional mlv from your 7d to analyze it on my pc.

--> raw2cdng@phreekz.de

regards chmee

hmmm....short..let me see what i can find that errors out thats not huge.

EDIT: Okay, I have NOTHING thats not massive. GOnna shoot a test tonight thats not too big for you to analyze . Many thanks :)

EDIT 2: Now of course, all my very short clips are working. 5-13 seconds. Possibly something with large files and split M01s, etc?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on May 12, 2014, 05:55:28 AM
Testing Beta4, Had crashes with jpeg proxy enabled & 16bit output or when I previewed then try to convert without proxy enabled crashed also
will test some more.
@chmee here is a screen shot of FCPX Right hand side is from the Beta 4 Raw2cdng & on the left is the 1.4.9 ver. raw2cdng.
http://500px.com/photo/70187941/fcpxca~-by-david-miazga?from=user_library
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 12, 2014, 06:06:27 PM
Tried with 10 separate clips and couldn't get them to crash, OF COURSE....dont want to upload 4GB MLVs, so will try again in or day or so
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 12, 2014, 06:21:42 PM
@jbowdach
my longest testfile is a 3strip-sequence. so, i will record a longer one today and do some tests. looking, if there are "out of memory" or some IO-exceptions doin this.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 13, 2014, 06:47:22 AM
thanks! Ive been trying to shoot full cards, so I 59GBs of MLVs + M001, M002, etcs. Seems be errors on various clips tho, not just REALLY long ones. my specific tests for you were all below 20 seconds tho

EDIT: Posted another log to bitbucket
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: artyg on May 13, 2014, 07:52:23 PM
@artyg
please send me a short MLV example to raw2cdng@phreekz.de. actually i'm a bit confused because of the actual changes regarding the blacklevel. i didn't saw those problems with earlier ml-builds..

https://bitbucket.org/hudson/magic-lantern/pull-request/484/black-level-fix/diff

please @all
the actual ml-builds are quite new (especially regarding the blacklevel-things), some of them (1.2.3 for 5DIII) are bleeding edge. dont change your system if you want to work productive. if you need them, use an older build, something between ~feb09(mlv-audio) and ~apr27.. i am still testing my code with a 5DIII 1.1.3 build from ~apr20, because its workin' and stable. you are not bound to use the newest builds!

I downgrade my 5D3 firmware to 1.1.3 and install ML 24Apr. And pink highlites stay... Then I run beta4 in CDNG 12bit (not maximezed) mode and programm work correct! No pink. I test my old files, and they all convert correct! So, I think 1.2.3 firmware work correct too... But some strange things with 1.2.3 happen. Sometimes 5D3 does not turn on!.. I decided to stay in 1.1.3 - this firmware more stable. Thanks, chmee!

So, one bug i find. In 12bit maximized mode pink highlights appear... (post on bitbucket)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 15, 2014, 08:13:54 AM
please @all i need more testfiles from other bodies than mine (5DIII 1.1.3).

(*) do a short (max 100MB) mlv or raw record, 10 frames
(*) with high dynamics, it should have overexposed and dark areas as well
(*) some greyish things and some colored (colorchecker, if you have)
(*) some kind of testpattern (search via google)?
(*) write down build date (checksum?) and body with canon-firmwareversion
(*) the files will be collected and released/listed on a site

if you want, you can send them to me (raw2cdng@phreekz.de)

regards chmee

@artyg 12bit pinkhighlights, yes i know. thats the reason i ask for more videofiles from other bodies.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ted ramasola on May 15, 2014, 08:17:24 AM
Do you need a mkII file?

What setting should the BLACK FIX be ON or OFF ?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 15, 2014, 08:56:23 AM
yes, 5DII as well. best is to deliver both and name them.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ted ramasola on May 16, 2014, 09:08:23 AM
Sent you 4 mlvs from 5D2 and 7D to raw2cdng@phreekz.de

2 settings for each cam. With black fix ON and OFF.

Used ML nightly may 14 build for both cams.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 16, 2014, 09:48:58 AM
i'm already downloading. thanks.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 18, 2014, 09:59:09 PM
raw2cdng 1.5.0 beta5 online

* stabilizing code (memory leak if bigger files fixed, memory leak if preview was used fixed)
* proxy jpgs now with gamma 2.0
* busy-progressbar while reading data from drag/dropped files

http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

now there are two important things for me:
* @reddeercity please send me (as written above) these two different dng's - want to fix that.
* 12bit output isnt fine at all - have to fix that.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 19, 2014, 07:22:58 AM
many thanks, will test and report back
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on May 19, 2014, 08:26:23 AM
@chmee I will get those file to you later on today when I wakeup (eight hours from now)  :D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 19, 2014, 09:36:52 AM
@chmee tested out beta5 and def much improved. Still eventually ran into a crash, but got much further along. Posted as a new issue ( https://bitbucket.org/chmee/raw2cdng/issue/16/crashes-when-converting-to-16bit-cdng-from (https://bitbucket.org/chmee/raw2cdng/issue/16/crashes-when-converting-to-16bit-cdng-from)) on bitbucket for your review when you have a chance. Ive been using the same files for testing so I will shoot some fresh test footage to future testing to eliminate that as an issue

EDIT: 2 crashes. but eventually success!!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 19, 2014, 10:06:31 AM
@jbowdach
looked into the debug.. it crashes while/right after audiofile-creation. it converted 5146 Frames. does the converter show this amount of frames if you dragdrop it? open the taskmanager and look while converting on the resources (memory) of the process raw2cdng. does it come near 1.5GB?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 19, 2014, 10:41:40 AM
@jbowdach
looked into the debug.. it crashes while/right after audiofile-creation. it converted 5146 Frames. does the converter show this amount of frames if you dragdrop it? open the taskmanager and look while converting on the resources (memory) of the process raw2cdng. does it come near 1.5GB?

Let me try another conversion and see.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 19, 2014, 10:50:49 AM
@jbowdach
looked into the debug.. it crashes while/right after audiofile-creation. it converted 5146 Frames. does the converter show this amount of frames if you dragdrop it? open the taskmanager and look while converting on the resources (memory) of the process raw2cdng. does it come near 1.5GB?

Doesnt appear to be coming near, its hovering at 40-100MBs. It just crashed with mem usage around 73 MBs. Newest log uploaded to bitbucket. Heading to sleep for the night, but let me know anything else you need and Ill get it for you tomorrow. THanks for the beta5, def making progress! Ur hard work is much appreciated  :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 19, 2014, 06:29:33 PM
@jbowdach try to find a tool for us (maybe i code it). raw2cdng first reads the file, all blocks inside, and somewhere frame 5200 it does not take the real framenumber from the block, but a zero, some are doublets..

please use mlv_dump to write an indexfile (idx) and link to that file.
http://www.magiclantern.fm/forum/index.php?topic=7122.0
Code: [Select]
./mlv_dump -x in.mlvi want to see if the blocks inside the mlv's are ok.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ted ramasola on May 20, 2014, 07:48:59 AM
Chmee,

tested the latest beta5. 42 clips totalling 8+ minutes.

In resolve when I use autosync based on timecode some wav files will link with the wrong video files.
This will be indicated by wavforms with wrong displays, showing blank spaces.

Aligning them manually and the wav files are ok.


(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fscontent-b-dfw.xx.fbcdn.net%2Fhphotos-frc3%2Ft31.0-8%2F10344292_480990862031703_6166731265983228991_o.jpg&hash=bd229f71b359aaa8412609b67c69cff7)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 20, 2014, 08:50:35 AM
@ted please do a second try with changed filename-structure. it seems you use
Code: [Select]
[F]
-->
/M16-2015
M16-201500000.dng
M16-2015.wav

try
Code: [Select]
[F](_)
-->
/M16-2015
M16-2015_00000.dng
M16-2015.wav
remember, Resolve wants idealy a filename like that:
Code: [Select]
[F]_[D]_001_C0000(_)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 20, 2014, 09:42:11 AM
@jbowdach try to find a tool for us (maybe i code it). raw2cdng first reads the file, all blocks inside, and somewhere frame 5200 it does not take the real framenumber from the block, but a zero, some are doublets..

please use mlv_dump to write an indexfile (idx) and link to that file.
http://www.magiclantern.fm/forum/index.php?topic=7122.0
Code: [Select]
./mlv_dump -x in.mlvi want to see if the blocks inside the mlv's are ok.

regards chmee

Does it mention which filename?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 20, 2014, 09:53:58 AM
filename of your recording, fi
Code: [Select]
mlv_dump -x M16-2018.MLVshould output a
Code: [Select]
M16-2018.idxin the same folder.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ted ramasola on May 20, 2014, 11:06:20 AM

try
[F](_)


Tried this and the wav files are still not linked correctly. Strange thing is that some files are ok, some are not.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 20, 2014, 11:13:50 AM
as i mentioned entries before, look on the files in the mediabrowser. you should see the embedded timecode. please tell me both tc-values. did you recorded nondrop or drop?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ted ramasola on May 20, 2014, 11:32:49 AM
Using [f](_) convention and testing using just 2 files still will not link properly using autosync.

resulting files is named as:

WAV file = M16-2036.wav
DNG file(1stframe) = M16-2036_00000.dng
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 20, 2014, 11:41:51 AM
yeah, i think, there are some circumstances, where tc of video and audio do not match. and by now i cant catch these moments..
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: glubber on May 20, 2014, 01:11:46 PM
Just a small bug report:

raw2cdng.1.5.0.beta5.exe crashes while drag&drop a "early build" RAW-File (i guess very first build for 550D from May 2013) while v1.3.3.1 is doing fine.
I'm on Vista 64.

Code: [Select]
Beschreibung:
  Stopped working

Problemsignatur:
  Problemereignisname: CLR20r3
  Problemsignatur 01: raw2cdng.1.5.0.beta5.exe
  Problemsignatur 02: 1.5.0.0
  Problemsignatur 03: 53790dc6
  Problemsignatur 04: raw2cdng_v2
  Problemsignatur 05: 1.5.0.0
  Problemsignatur 06: 53790dc6
  Problemsignatur 07: 5d
  Problemsignatur 08: 0
  Problemsignatur 09: System.DivideByZeroException
  Betriebsystemversion: 6.0.6002.2.2.0.256.1
  Gebietsschema-ID: 1031

The Raw file is over 900 MB big so i can't just attach it :)

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 20, 2014, 02:33:45 PM
@glubber are you crazy?  :D confronting me with a nearly 1 year old build on a body i had no samples yet :) when does it happen? right while dragdrop? while converting? so, it shows a "DivideByZeroException", some values - assume - couldnt be read. please do me a favour, do a short record as its described here (http://www.magiclantern.fm/forum/index.php?topic=11899.0) and send it to me - i would use it for the samples gallery as well.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: glubber on May 20, 2014, 03:01:05 PM
Yep right while dropping into raw2cdng window.

I'll try to dig out that old built and give You a short sample.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: artyg on May 20, 2014, 09:02:15 PM
Chmee,

tested the latest beta5. 42 clips totalling 8+ minutes.

In resolve when I use autosync based on timecode some wav files will link with the wrong video files.
This will be indicated by wavforms with wrong displays, showing blank spaces.

Aligning them manually and the wav files are ok.

Hmmm, I test autosinc today and all work OK...
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 20, 2014, 09:12:44 PM
yeah, often its ok (on my footage i hadnt problems as well), but here were two guys with problems. the Timecode is taken from datetime of file. for the wav file i have to recalculate to audiosamples AND adjust with nondrop or dropframe - sometimes this adjusting does a wrong thing. by now i cant grab that bug (i should do some more debug infos :))
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on May 20, 2014, 11:11:51 PM
@chmee, just sent you 2 Cdng's converted by raw2cdng v1.4.9 & v1.50 beta5.
As I said v1.4.9 display right but v1.5.0 beta5 has a pink cast in MAC Finder or preview app.
with 16bit linear .

Edit: ML build feb16 NB, 1872x1012. Extracted with MLVBrowseSharp to Raw v1.0 then
converted with v1.4.9 but not on beta5 just drop in your app and converted.
Note* the only difference I notice in ACR in v1.4.9 has a WB of 4950-k and a Locked embedded camera profile
while v1.5.0 beta5 has a WB 5000-k and the Camera profile default Adobe Standard but other profiles are selectable.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 20, 2014, 11:57:34 PM
sent you my thoughts on your examples :) the finder-result does not interest me, honestly saying. assume, its because of the linearizationtable, thats inside to work with premiere..

the reason, you couldnt use profiles in acr is, you changed the name to "5D Mark2 Cdng 1.4.9". "Canon EOS 5D Mark II" works.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on May 21, 2014, 12:30:45 AM
sent you a screen capture of finder with the Cdng's
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 21, 2014, 12:41:17 AM
jepp. in this case (app-world and apps) i cant really help - just to say it officially once again: i assume its about the linearizationtable inside the dng's. will think about it (much later) if i just dont use them, when only linear 16bit dng's are wanted.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on May 21, 2014, 12:46:57 AM
Ok, I use PC because it's faster then mac in converting raw and better tools!
I can use the older version I just wanted the new name generator feature on the new one that's all.  :) 
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: glubber on May 21, 2014, 12:50:20 PM
Hey Chmee,

i was able to repoduce the behavior of 1.5.0beta5 vs. 1.3.3 with a RAW file of Luke Neumanns sheeps :D
http://neumannfilms.net/2013/05/15/magic-lantern-raw-video-workflow-tutorial-downloadable-files/ (http://neumannfilms.net/2013/05/15/magic-lantern-raw-video-workflow-tutorial-downloadable-files/)

Nevertheless I'll try my best to upload my 550D samples the next 2-3 days.

Below a screenshot of the crash showing the progressbar while dropping in the file.

 (https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fimg213.imagevenue.com%2Floc1145%2Fth_68007_raw2CDNG1_5_crash_LNsheep_122_1145lo.jpg&hash=22afdd851f7a1e3fe3dbdc5a65dbad33) (http://img213.imagevenue.com/img.php?image=68007_raw2CDNG1_5_crash_LNsheep_122_1145lo.jpg)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 21, 2014, 01:32:27 PM
i just got a file with the same behaviour (strange i hadnt that on other files, the problem is so obvious.) when raw's have no framerate (did they ever had this metadata?) , it will crash while calculating the duration of the video. if framerate is zero, i set it now to 24fps. today i will put a beta6 on the server.

raw2cdng 1.5.0 beta 6 online

* 12bit maximized should work again (with pinkhighlight-fix as well)
* RAW crash fix (divideByZero-Exception)
* more debugdata

http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 22, 2014, 08:01:36 AM
Will 12 bit regular return? What's the major difference between maximized that requires the huge increase in processing involved to create them?

. never could find that file you mentioned but ill see if i have any future crashes or issues

Thanks as always! Giving your suggestions regarding filenames using f(_) a try for audio sync in Resolve, as I'm having some issues auto syncing.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: glubber on May 22, 2014, 09:23:33 AM
Hi Chmee,

is it intentional that that the headline of the app shows "rawcdng 1.5" instead of "raw2cdng 1.5"?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 23, 2014, 12:27:35 AM
@jbowdach the main difference normal versus maximized is i dont use the original values from canon ( example 5DIII: 2048-15000 from max 16384) but i spread them to the maximum values (0-65535 with 16bit). this is needed for premiere and gives no drawback on other apps. with 16bit we have no problem with changing valuerange, because we're not lowering it. in 12bit we do and every bit we have we should use.

ahh btw. with beta6 i write more from the audioprocess into the debuglog. i want to find this bug for autosync in resolve as well :)

@glubber thanks, will change that :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on May 23, 2014, 12:35:14 AM
@jbowdach the main difference normal versus maximized is i dont use the original values from canon ( example 5DIII: 2048-15000 from max 16384) but i spread them to the maximum values (0-65535 with 16bit). this is needed for premiere and gives no drawback on other apps. with 16bit we have no problem with shortening valuerange, because there isnt. in 12bit we're lowering the bitdepth and every bit we have we should use.

@glubber thanks, will change that :)

Makes sense  thanks for the explanation. Seems to save approx 20% space compared to 16 bit conversions based on my tests last night. Will know of sync worked when the  get home and can check the render output.

Does premiere not accept 12 bit non maximized cdngs sequences? Haven't played w native cdngs in premiere pro yet
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 23, 2014, 12:46:33 AM
premiere does not work with biased files (zero is not zero). and because premiere in the actual version only accepts bmcc-like cdng-files, i (we?!) had to code this pinkhighlightfix. BUT it works well with 12/16bit maximized pinkhighlight-fixed files.

make the test for yourself, its a huge difference.

(i promise, in the next beta there will be the vertical banding-button, its no1 on todolist)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 23, 2014, 01:15:22 PM
@a1ex is vertical banding an individual static property of every body? if yes, would it be an idea to write a module to calculate it for the body and save it as a block into the mlv?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: a1ex on May 23, 2014, 02:57:42 PM
From the same body, it seems to differ from clip to clip, but inside one clip it stays pretty much the same.

My heuristic (for speed reasons) is to compute it from the first frame only. This works fine most of the time, but if you start recording with lens cap on for example, it will fail. A better approach would be to take say 5 or 10 frames, evenly spread across the footage, and compute from there. Most likely, averaging the coefficients or taking the median value should do the trick.

I don't know exactly how it behaves with long recordings though, but I assumed the current approach works fine, since there were no relevant bugs reported for this use case.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 23, 2014, 03:07:12 PM
thx. it was only my assume, the banding is kind of fingerprint per body-electronics. so it was a simple idea to implement the calculations into the body, calculate it once and saving the array of floatingpoint on the body. but it's not i know now.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on May 27, 2014, 01:48:56 PM
hi chmee, with beta 6, audio sync in resolve works with certain footage and some others are wrongly synced. Same audio synced twice to different footage.

Just another note which i wanted to share is that, i used to optimize the flickering problem in resolve by changing file name codes in raw2cdng. I had to work with .raw files and .mlv files together for a project. A single timeline in resolve having both raw's cdng and mlv's cdng really threatened me regarding flickering issues. However, i went through resolve's manual and now one can overcome the flickering issue by checking the "Output single field when paused" option from "General Options" in resolve. It worked for me (but I also changed file naming in converter). Hope the solution works and has nothing to do with file naming... thanks
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: contactarproyectos on May 29, 2014, 04:21:05 AM
Hi guys,
This could be an old question, but I really need some help:
The only raw2cdng version I can launch are 1.2.0 or olders. With newest: “raw2cdng has stopped working” appear. I´ve tried run as adm, compatibility mode, etc, and nothing.
Do you know why?
OS win7, 64bits

Thanks!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 29, 2014, 09:33:58 AM
Hmm. It seems, my app cant write some data. It writes two Files:

* debug.log - since 1.4.6, into the same path as the app
* raw2cdng.settings.json - since 1.2.1, into Users/[User]/Appdata/roaming

* Dont use programs or windows as path.
* maybe it has something to do with special chars in your username?

* try a start from root or other harddisk.
* under file-properties, try to set admin static.
* wait a moment, it should show a some debug info, there will be some files in a temp-directory. the small xml-file should show the reason.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: contactarproyectos on May 29, 2014, 07:20:39 PM
* No debug.log file
* No raw2cdng.settings.json file

I found this xml files:

-C:\Users\Dario\AppData\Local\ElevatedDiagnostics\733862231\2014052901.000\PCW.0.debugreport.xml:

miércoles, 28 de mayo de 2014 22:19:02true6.1Clientamd641.5Microsoft WindowsHaga que los programas antiguos se ejecuten en esta versión de Windows.03740x0C:\Windows\diagnostics\system\PCW0x00000000006B5C40<Answers Version="1.0"> <Interaction ID="IT_LaunchMethod"> <Value>ContextMenu</Value> </Interaction> <Interaction ID="IT_SelectProgram"> <Value>NotListed</Value> </Interaction> <Interaction ID="IT_BrowseForFile"> <Value>C:\Users\Dario\Desktop\magic lantern\raw2cdng.1.5.0.beta6.exe</Value> </Interaction> </Answers>50380x803C0100


-C:\Users\Dario\AppData\Local\Temp\WER5B1F.tmp.WERInternalMetadata.xml:

<?xml version="1.0" encoding="UTF-16"?>
-<WERReportMetadata>-<OSVersionInformation><WindowsNTVersion>6.1</WindowsNTVersion><Build>7601 Service Pack 1</Build><Product>(0x30): Windows 7 Professional</Product><Edition>Professional</Edition><BuildString>7601.18113.amd64fre.win7sp1_gdr.130318-1533</BuildString><Revision>1130</Revision><Flavor>Multiprocessor Free</Flavor><Architecture>X64</Architecture><LCID>3082</LCID></OSVersionInformation>-<ProblemSignatures><EventType>CLR20r3</EventType><Parameter0>raw2cdng.1.4.9.exe</Parameter0><Parameter1>1.0.0.0</Parameter1><Parameter2>52f7c322</Parameter2><Parameter3>raw2cdng</Parameter3><Parameter4>1.0.0.0</Parameter4><Parameter5>52f7c322</Parameter5><Parameter6>82</Parameter6><Parameter7>c8</Parameter7><Parameter8>System.IO.FileNotFoundException</Parameter8></ProblemSignatures>-<DynamicSignatures><Parameter1>6.1.7601.2.1.0.256.48</Parameter1><Parameter2>11274</Parameter2><Parameter22>0a9e</Parameter22><Parameter23>0a9e372d3b4ad19135b953a78882e789</Parameter23><Parameter24>0a9e</Parameter24><Parameter25>0a9e372d3b4ad19135b953a78882e789</Parameter25></DynamicSignatures>-<SystemInformation><MID>8C63604C-DCF0-4276-B73A-601CDEFC9D85</MID><SystemManufacturer>System manufacturer</SystemManufacturer><SystemProductName>System Product Name</SystemProductName><BIOSVersion>2102</BIOSVersion></SystemInformation></WERReportMetadata>


- and this txt:

<?xml version="1.0" encoding="UTF-16"?>
<DATABASE>
<EXE NAME="raw2cdng.1.4.9.exe" FILTER="CMI_FILTER_PRIVACY">
    <MATCHING_FILE NAME="mlv_dump.exe" SIZE="271511" CHECKSUM="0x89C7DAE7" MODULE_TYPE="WIN32" PE_CHECKSUM="0x48851" LINKER_VERSION="0x10000" LINK_DATE="02/06/2014 20:42:55" UPTO_LINK_DATE="02/06/2014 20:42:55" EXE_WRAPPER="0x0" FILE_ID="000019729fc7e9e575a55adfed708247462ec458211f" PROGRAM_ID="00034d1e3b779acee3608899521a05a279ae0000ffff" />
    <MATCHING_FILE NAME="raw2cdng.1.4.9.exe" SIZE="553472" CHECKSUM="0x276FD127" BIN_FILE_VERSION="1.0.0.0" BIN_PRODUCT_VERSION="1.0.0.0" PRODUCT_VERSION="1.0.0.0" FILE_DESCRIPTION="raw2cdng" PRODUCT_NAME="raw2cdng" FILE_VERSION="1.0.0.0" ORIGINAL_FILENAME="raw2cdng.exe" INTERNAL_NAME="raw2cdng.exe" LEGAL_COPYRIGHT="Copyright ©  2013" VERDATEHI="0x0" VERDATELO="0x0" VERFILEOS="0x4" VERFILETYPE="0x1" MODULE_TYPE="WIN32" PE_CHECKSUM="0x0" LINKER_VERSION="0x0" UPTO_BIN_FILE_VERSION="1.0.0.0" UPTO_BIN_PRODUCT_VERSION="1.0.0.0" LINK_DATE="02/09/2014 18:04:18" UPTO_LINK_DATE="02/09/2014 18:04:18" VER_LANGUAGE="Independiente del idioma [0x0]" EXE_WRAPPER="0x0" FILE_ID="0000e0cfb8a8373eb6a02cd0171346989ea56ad09f07" PROGRAM_ID="000391263d3901eb726be10e0256222aeca100000000" />
</EXE>
<EXE NAME="KERNELBASE.dll" FILTER="CMI_FILTER_THISFILEONLY">
    <MATCHING_FILE NAME="KernelBase.dll" SIZE="274944" CHECKSUM="0xE4148384" BIN_FILE_VERSION="6.1.7601.18015" BIN_PRODUCT_VERSION="6.1.7601.18015" PRODUCT_VERSION="6.1.7601.18015" FILE_DESCRIPTION="Archivo DLL de cliente API BASE de Windows NT" COMPANY_NAME="Microsoft Corporation" PRODUCT_NAME="Sistema operativo Microsoft® Windows®" FILE_VERSION="6.1.7601.18015 (win7sp1_gdr.121129-1432)" ORIGINAL_FILENAME="Kernelbase" INTERNAL_NAME="Kernelbase" LEGAL_COPYRIGHT="© Microsoft Corporation. Reservados todos los derechos." VERDATEHI="0x0" VERDATELO="0x0" VERFILEOS="0x40004" VERFILETYPE="0x2" MODULE_TYPE="WIN32" PE_CHECKSUM="0x4AA27" LINKER_VERSION="0x60001" UPTO_BIN_FILE_VERSION="6.1.7601.18015" UPTO_BIN_PRODUCT_VERSION="6.1.7601.18015" LINK_DATE="11/30/2012 04:56:42" UPTO_LINK_DATE="11/30/2012 04:56:42" EXPORT_NAME="KERNELBASE.dll" VER_LANGUAGE="Español (España, internacional) [0xc0a]" EXE_WRAPPER="0x0" FILE_ID="000090b069cbc93c701394b47d4e830cdd7d384f5d5c" PROGRAM_ID="0000f519feec486de87ed73cb92d3cac802400000000" />
</EXE>
<EXE NAME="kernel32.dll" FILTER="CMI_FILTER_THISFILEONLY">
    <MATCHING_FILE NAME="kernel32.dll" SIZE="1114112" CHECKSUM="0x9725986B" BIN_FILE_VERSION="6.1.7601.18015" BIN_PRODUCT_VERSION="6.1.7601.18015" PRODUCT_VERSION="6.1.7601.18015" FILE_DESCRIPTION="Archivo DLL de cliente API BASE de Windows NT" COMPANY_NAME="Microsoft Corporation" PRODUCT_NAME="Sistema operativo Microsoft® Windows®" FILE_VERSION="6.1.7601.18015 (win7sp1_gdr.121129-1432)" ORIGINAL_FILENAME="kernel32" INTERNAL_NAME="kernel32" LEGAL_COPYRIGHT="© Microsoft Corporation. Reservados todos los derechos." VERDATEHI="0x0" VERDATELO="0x0" VERFILEOS="0x40004" VERFILETYPE="0x2" MODULE_TYPE="WIN32" PE_CHECKSUM="0x11B7A6" LINKER_VERSION="0x60001" UPTO_BIN_FILE_VERSION="6.1.7601.18015" UPTO_BIN_PRODUCT_VERSION="6.1.7601.18015" LINK_DATE="11/30/2012 04:56:41" UPTO_LINK_DATE="11/30/2012 04:56:41" EXPORT_NAME="KERNEL32.dll" VER_LANGUAGE="Español (España, internacional) [0xc0a]" EXE_WRAPPER="0x0" FILE_ID="0000f9ac70c562ba70dd7917e99b00ed6878531de66f" PROGRAM_ID="0000f519feec486de87ed73cb92d3cac802400000000" />
</EXE>
</DATABASE>


* I´ve tried in an old xp and it works!

Any idea?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 29, 2014, 09:28:43 PM
Is .net4.0 installed?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: contactarproyectos on May 30, 2014, 12:21:25 AM
It´s installed.

A while ago I made some changes in the regional language settings for another program compatibility (pro tools), but today I restored to default and nothing happened.
Could be something there, maybe?

Thanks for your time chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 30, 2014, 01:08:20 AM
try to save an empty textfile into the roaming-path i described before, then name it raw2cdng.settings.json or (>=1.5.0) raw2cdng.2.settings.json.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: contactarproyectos on May 30, 2014, 04:49:34 PM
Done, but nothing has changed.

What about updating to netframework 4.5? It´s safe? Vulnerability risk or something?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 30, 2014, 05:49:23 PM
someone's said, it needs 4.5 - didnt found the parts i used 4.5-code.. so. the main thing is, 4.5 is NOT backward compatible to 4.0 - 4.5 is NOT an update from 4.0. if you wanna try and tell me if it works after installing 4.5, would be great. THEN i would analyze my code once again for this dotnet-thing.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on June 07, 2014, 02:54:27 PM
I just wanted to let you know that I used 1.5 Beta 6 and did not have any issues on my Windows 8.1, GeForce GTX 780, and pulling the footage into Premiere CC with audio synced. No pink highlights or green frames. Even when viewing the DNGs in Explorer, the images were scaled correctly and the colors were represented properly. Great work building this from the bottom up! Excited to see more to come and what Adobe does with the rumors of more DNG support.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: sergiocamara93 on June 18, 2014, 09:01:16 PM
Hey chmee, I wanted to offer you some testing results regarding the timecode issue of the WAVs auto-sync in Resolve, I've read that your software uses the datetime of the file to generate the timecode. I've tested this and I found why Resolve it's not syncing some of the files properly. I used two files M03-1753 and M03-1755.

The timecode generated for M03-1713 and its WAV was 20:15:59:00 to 20:16:28:12 (6 more for the WAV)
The timecode generated for M03-1715 was 20:15:28:00 - 20:16:05:18 (5 more for the WAV)

I don't think is a problem of timecode generation. Its consistent for video and audio, and they seem to match but, as you see, the timecodes for the two files are not unique and overlap, therefore, when Resolve syncs them it goes to the first file it can find that has some of that timecode (the first file when shorted by name that has some of that time interval, it defaults to that and ignores the reelname - because audio doesn't catch the reelname from the folder).
The timecode match the date of creation of the file on the hard drive, that means when the .MLV was copied to the hard drive, which obviously makes files overlap because the creation times are usually close.

I believe there can be a quick workaround for this (the proper fix would be emulating the BMCC WAVs) taking the timecode start from the name of the file i.e: "1713". As I mention this would be just a workaround because in the case of having files created at the "same time" in different days the timecode would overlap, also when using different cameras, and, Resolve would go crazy again. But this is solvable. When put in different folders of the Media Pool in Resolve (Day 1, Day 2 / Camera A, Camera B...) And selecting the clips in those folders, then clicking the auto-sync, the two files sync properly and only with the WAVs that have the same timecode in that folder. If timecodes did not overlap in one folder, we should be just fine. It's not perfect, but it works.

Well, in any case, I hope some of this testing offers some insight and helps in solving the problem.

Tech details:
- Camera: 5D Mark III
- raw2cdng 5.0beta6
- MLV / Nightly ML build of February 14th 2014 (I know, it's an old build, but I'm not aware of any changes on the timecode/name/file creation process that can influence the issue reported)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Johannes on June 22, 2014, 10:04:19 PM
Create clean cDNG´s from MLV would be so great! Cant wait!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: actingnurse on June 23, 2014, 02:47:13 AM
Create clean cDNG´s from MLV would be so great! Cant wait!

Raw2CDNG 1.5.0 beta 6 is what I use and it does break MLV files apart into wav and CDNG..
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: actingnurse on June 23, 2014, 02:48:41 AM
I just wanted to let you know that I used 1.5 Beta 6 and did not have any issues on my Windows 8.1, GeForce GTX 780, and pulling the footage into Premiere CC with audio synced. No pink highlights or green frames. Even when viewing the DNGs in Explorer, the images were scaled correctly and the colors were represented properly. Great work building this from the bottom up! Excited to see more to come and what Adobe does with the rumors of more DNG support.

Chad - ditto here. I use 1.5.0 beta 6 with Adobe Premiere CC 2014 on Windows 8.1, GeForce GTX 670M, and have no issues.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on June 23, 2014, 10:24:29 AM
thanks. so, next days i will turn back on coding. i'll try to implement vertical banding and chroma smoothing. as i've seen, the new dng-import-module in premiere 8.0 does pink highlights as well. hummm, and they#re working on a fix. do some tests today. i hoped, i can delete/forget the highlightfix.

@sergiocamara93
thanks for tests. it wonders me, why it behaves this way. its not unusual, that there are more files with nearly the same timecode (multicam).
Quote
..proper fix would be emulating the BMCC WAVs..
by now i tried to do my best to implement all things, that are in bmcc-wav files.
http://www.magiclantern.fm/forum/index.php?topic=11520.0
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on June 24, 2014, 04:13:59 PM
by the way. i told days ago i wrote a diagnostic tool for mlv-files. here it is.

mlvdiag 1.0.0 - download here (http://dl.phreekz.de/raw2cdng/mlvdiag.1.0.0.zip)
whats the idea behind it?
* analytic look into the blockdata saved inside the mlv
* important for me: saving a minimal set of blocks into a new file for app-problems and debugging
-> it writes all metadata, one videoframe and one audiochunk into a new file, found on same place as original.
-> if you have some problems with raw2cdng-crashes this minimal mlv can help me to find those bugs.
* its not perfect, but it works :) ('til now)

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fmlvdiag.jpg&hash=5ffcae6d7b93f494104f3f1a639795d0)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on June 24, 2014, 04:50:50 PM
Oh, Sweet! Thanks, chmee! :3
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on June 24, 2014, 07:43:16 PM
THis looks like a fantastic diagnostic tool to help resolve issues w problematic MLV files. THanks so much! Looking forward to the next version of raw2cdng.
Please let us know how we can use it to either fix files, and\or use it to provide you with useful diagnostic information
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: evanbuzzell on June 28, 2014, 01:10:12 AM
In beta 6, I still cannot scroll through the files without the app crashing
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on June 28, 2014, 03:28:53 AM
Just a thought for a request. I have seen in the MLV to DNG Batch Converter v1.9 there is an option to use an ACR profile when exporting the DNGs. The idea behind it is to apply an ACR preset that is a flat profile or one that you would create for what ever you have in mind. Then, you could either bring those into AE to export to ProRes or what ever codec you want OR bring those right into Premiere or Resolve with that preset applied. That would save a ton of clicks, especially with larger projects with bringing them with a flat preset applied already. However, I do not know if Premiere or Resolve will read that applied preset since cDNG is a different process than just the normal DNG (to my knowledge).
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: sergiocamara93 on June 29, 2014, 11:50:24 PM
However, I do not know if Premiere or Resolve will read that applied preset since cDNG is a different process than just the normal DNG (to my knowledge).
No, they won't. PP uses Speedgrade framework which doesn't read ACR and the same goes for Resolve (where you can apply a flat profile - BMPCC - and then a LUT of your choice to all your footage. Plus it encodes faster).

Forgot to mention that MLV to DNG Batch Converter v.1.9 generates DNG, not CDNG. So PP won't read them at all, for Resolve it's about the same between the two formats right now (reads both equally).
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on June 30, 2014, 02:42:57 AM
@sergiocamara93

Thanks for the info! Never mind about my request then. heh.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on July 01, 2014, 09:11:41 AM
Hi, from my 2TB shooting I have a trouble opening one of my file. The loader bar is still loading, both raw2cdng and mlvdiag.

What i did -- downloaded all the mlv's, deleted those 1 bytes and batch renamed all the mlv's (because of Resolve round-trip method).

The debug info:
Code: [Select]
[drop] ------
[drop] -- file K:\xxx\002_M0336.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in K:\xxx\002_M0336.MLV

My theory is that the end of this file was splitted into those 1byte chunks and sice I deleted them, raw2cdng don't know where to find them? The progress bar is still active, but the disc activity is zero.

EDIT: But I deleted many 1 byte files and from 399 MLV's only with this I am getting error. The file exctracts with MLV Converter 1.9.2 fine. :(
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on July 01, 2014, 10:41:30 AM
@kaco
can you open it with mlvdiag (https://secure.magiclantern.fm/forum/index.php?topic=5618.msg119737#msg119737)? Oh Sorry, see now, you tried. ok. will change the reader module in mlvdiag/raw2cdng to allow open/read "corrupted"/strange blocks.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on July 01, 2014, 11:38:27 AM
@chmee: now tested with mlv convertor 1.9.2 ... the files open, but won't extract... since it's a 80gb of data I think there's something important in it.. :( oh, god...
Do you think there is a chance to get the data from the file?

tested it also with mlv2dng:
Code: [Select]
mlv2dng.exe 002_M0336.MLV
Exception! Header not recognized: ''

Exception! Header not recognized: ''

mlv_dump:
Code: [Select]
mlv_dump.exe 002_M0336.MLV

 MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: '002_M0336.MLV'
   - Verify file structure
File 002_M0336.MLV opened
File 002_M0336.M00 not existing.
Processing...
[ERROR] Missing file header
Processed 0 video frames
Done
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on July 01, 2014, 11:58:47 AM
Do you have a Hex-File-Editor like HXD (http://mh-nexus.de/de/hxd/)? Open the file with it, copy the first 1MB (or 5 or 10) to a new file and send it to me. raw2cdng@phreekz.de (i will code such a button into mlvdiag).. Ahh, please write down the original and ml-firmware-version and body, maybe its a hint for @a1ex on a bug. maybe..
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on July 01, 2014, 12:29:36 PM
as you assumed in your pm, this file is empty, full with zeros.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: pompeiisneaks on July 12, 2014, 07:56:27 PM
Chmee, I've got visual studio 2012 and have been playing with the source a bit, added some exception code due to warnings, etc.  I'd be happy to help out on the code base, I'm trying to get my C# skills up, I've got a class on it this semester, so any tinkering I can do will help me.  Also, I can help try and add some more try/catch blocks for some of the I/O type errors we've been seeing, but wanted to make sure you're needing help with the program.

~Phil
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on July 12, 2014, 09:22:53 PM
@Phil
of course, lets start a workflow with bitbucket.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on July 16, 2014, 02:08:41 AM
hey chmee,

here are a few logs from attempting to decode MLV files from my latest vacation clips. These are all shot on a 7D. Tried numerous times, and it seems some of these files just dont want to decode, they just crash the application. Only several files out of almost a TB of video tho, so not horrible. Just wish I knew what was causing these crashes. Please let me know if I can provide any further info about any specific file.

Smaller log1 error:https://www.dropbox.com/s/3szc3mruyhop361/raw2cdng.2.debug_crash630_alaska.log (https://www.dropbox.com/s/3szc3mruyhop361/raw2cdng.2.debug_crash630_alaska.log)
Log 2 error log: https://www.dropbox.com/s/pschwkg4derpi1e/raw2cdng.2.debug_crash2_630alaska.log (https://www.dropbox.com/s/pschwkg4derpi1e/raw2cdng.2.debug_crash2_630alaska.log)
Log 3 Errors: https://www.dropbox.com/s/ts3z31ejyunnn8q/raw2cdng.2.debug_crash3_630alaska.log (https://www.dropbox.com/s/ts3z31ejyunnn8q/raw2cdng.2.debug_crash3_630alaska.log)
BIG log file: https://www.dropbox.com/s/houu1koi7ff3yw1/raw2cdng.2.debug.log (https://www.dropbox.com/s/houu1koi7ff3yw1/raw2cdng.2.debug.log)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: arturochu on July 23, 2014, 05:53:17 PM
chmee what happened to the development of this program? not a single update in two months? bummer.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on July 23, 2014, 09:00:23 PM
+1
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on July 23, 2014, 11:28:32 PM
guys. sorry, really.. it just has been full o' work, and it is still. next days i'll publish a beta7 with some fixes but *SORRY* still no chroma and vertical banding. on the end of this month i'll start my holiday-time with more time for this project again.

to be honest, i didnt found the trick with chroma and vertical banding - i just spent far too little time to understand the sourcecode - and i think about changing the whole code to byte-arrays (instead of uint16) to be on a better track for that. cant say, if it changes the transcoding time. on the todolist is a c#-dll-library for mlv-things as well, cause i use nearly the same code on raw2cdng and mlvdiag. makes sense :) but positive news: a german guy mailed, he wants to help on raw2cdng, maybe we get all the transcoding-code in c++, hoping it speeds up the thing.
 
regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on July 24, 2014, 07:42:08 PM
Glad to see more things coming and expanding the team! Excited for more developments.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: arturochu on July 25, 2014, 05:30:21 AM
+1
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 02, 2014, 10:58:38 AM
short info on some work: as someone stated i didnt used the WBAL-data from MLV (https://docs.google.com/spreadsheet/ccc?key=0AgQ2MOkAZTFHdHJraTVTOEpmNEIwTVlKd0dHVi1ULUE#gid=0). the values are kind of tricky. why? The values inside WBAL are only usable under some circumstances:

these are (by now) the interesting data-fields in WBAL
Quote
wb_mode - from 0 to 6 and 9
kelvin - only accurate if wb_mode = 9
wbgain_r ,_g and _b - only accurate if wb_mode = 6
If you use wb-Presets (tungsten, shade, cloudy etc), i only see the preset, but neither kelvin nor wb_gain is right (there will be old values from the moment you used them last time). If you use KELVIN (manual setting), only kelvin will be right, the wbgain-values are not refreshed. as well, wb_mode MANUAL leads to accurate wbgain-values, but kelvin wont be refreshed.

(by the way)
Quote
wbs_gm - is the known green-magenta-shift, i assume, its a simple value shift green versus red/blue
wbs_ab - assume its a YPbPr-Shift (Lab)

Why do i write this? it seems, it makes sense to write down all necessary fields/values and save them as lists into the main ml-modules or into the converter. If you look into a cr2-file from your body, you ll find the presets. more: to get the most informations we should save the values from all possible Kelvin-Steps.

(1) i did pictures with all kelvin-steps (5DIII 2.500-10.000).
(2) saved them into a folder
(3) then i used the great exiftool to extract the wbgain-fields.

save this code into a file named wb_manual.fmt into this folder:
Code: [Select]
$ColorTemperature,${WB_RGGBLevelsAsShot;tr/ /,/}
and run this via command-line inside the folder you saved the cr2-files.
Code: [Select]
exiftool -p wb_manual.fmt *.CR2 > list.csv
voila, you ve got a commaseparated list - kelvin to rggb-values. here you see it inside excel:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fwb_5DIII_113.jpg&hash=d0bdf25c4ea52af7842866e7fe4338d9)

then do this format-file (name it wb_presets.fmt) on only one cr2-file to save the presets:
Code: [Select]
Daylight,$ColorTempDaylight,${WB_RGGBLevelsDaylight;tr/ /,/}
Shade,$ColorTempShade,${WB_RGGBLevelsShade;tr/ /,/}
Cloudy,$ColorTempCloudy,${WB_RGGBLevelsCloudy;tr/ /,/}
Tungsten,$ColorTempTungsten,${WB_RGGBLevelsTungsten;tr/ /,/}
Fluorescent,$ColorTempFluorescent,${WB_RGGBLevelsFluorescent;tr/ /,/}
Flash,$ColorTempFlash,${WB_RGGBLevelsFlash;tr/ /,/}
call it:
Code: [Select]
exiftool -p wb_presets.fmt [CR2-FILENAME] > wb_presets.csvafter that we've got the rggb-values for the wb_mode's 0-5. for 5DIII fw 1.1.3 these are the values:
Code: [Select]
Daylight,5200,2052,1024,1024,1678
Shade,7000,2362,1024,1024,1431
Cloudy,6000,2208,1024,1024,1544
Tungsten,3200,1479,1024,1024,2497
Fluorescent,3714,1796,1024,1024,2399
Flash,6320,2284,1024,1024,1520
so, now i've got all values to give the cinemadngs (nearly) accurate rggb-values to simplify the workflow inside the editing tools.

and now? whats the problem? it seems (look inside a cr2) these rggb-values from 5dIII are not accurate for all bodies. i will build this things into the next raw2cdng-version, but maybe its necessary to do these rggb-lists for every modell - and maybe these values change with the firmware. if someone's got some time for that, try to build these lists as well..

@g3gg0 , @baldand and @a1ex was this work somewhere done already?

@jbowdach i didnt forgot you, right now i just have the time to solve actual things. if you've got a "smaller" non-functional file, please send it to me.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on August 02, 2014, 03:09:11 PM
Hey there,

unless I did not make any mistake, the values are different on my 5D mark III's:

1. Older 5D mark III, bought in 2012, SN 013021001***, FW 1.1.3
Code: [Select]
2500,1182,1024,1024,31392600,1226,1024,1024,30222700,1274,1024,1024,29212800,1326,1024,1024,28192900,1369,1024,1024,27523000,1413,1024,1024,26893100,1450,1024,1024,25763200,1489,1024,1024,24673300,1526,1024,1024,23993400,1565,1024,1024,23303500,1606,1024,1024,22603600,1638,1024,1024,22083700,1672,1024,1024,21533800,1708,1024,1024,20973900,1739,1024,1024,20564000,1768,1024,1024,20164100,1802,1024,1024,19784200,1833,1024,1024,19384300,1859,1024,1024,19074400,1886,1024,1024,18724500,1913,1024,1024,18434600,1942,1024,1024,18114700,1967,1024,1024,17804800,1990,1024,1024,17534900,2009,1024,1024,17275000,2028,1024,1024,16995100,2048,1024,1024,16755200,2068,1024,1024,16495300,2089,1024,1024,16315400,2110,1024,1024,16135500,2131,1024,1024,15965600,2153,1024,1024,15775700,2171,1024,1024,15635800,2189,1024,1024,15475900,2208,1024,1024,15336000,2226,1024,1024,15176100,2241,1024,1024,15076200,2255,1024,1024,14946300,2270,1024,1024,14836400,2284,1024,1024,14716500,2300,1024,1024,14586600,2315,1024,1024,14486700,2330,1024,1024,14366800,2346,1024,1024,14276900,2362,1024,1024,14157000,2378,1024,1024,14047100,2394,1024,1024,13967200,2405,1024,1024,13897300,2416,1024,1024,13807400,2427,1024,1024,13727500,2439,1024,1024,13647600,2450,1024,1024,13577700,2461,1024,1024,13487800,2473,1024,1024,13417900,2485,1024,1024,13328000,2497,1024,1024,13268100,2509,1024,1024,13178200,2521,1024,1024,13118300,2533,1024,1024,13038400,2545,1024,1024,12988500,2551,1024,1024,12938600,2558,1024,1024,12878700,2570,1024,1024,12828800,2576,1024,1024,12768900,2583,1024,1024,12719000,2595,1024,1024,12669100,2602,1024,1024,12609200,2608,1024,1024,12569300,2615,1024,1024,12509400,2628,1024,1024,12459500,2635,1024,1024,12419600,2641,1024,1024,12359700,2655,1024,1024,12319800,2661,1024,1024,12259900,2668,1024,1024,122110000,2675,1024,1024,1215
Code: [Select]
Daylight,5200,2068,1024,1024,1649
Shade,7000,2378,1024,1024,1404
Cloudy,6000,2226,1024,1024,1517
Tungsten,3200,1489,1024,1024,2467
Fluorescent,3714,1817,1024,1024,2372
Flash,6320,2305,1024,1024,1492

1. Newer 5D mark III, bought in 2014, SN 268021002***, FW 1.1.3
Code: [Select]
2500,1159,1024,1024,30132600,1201,1024,1024,29052700,1247,1024,1024,28042800,1295,1024,1024,27092900,1334,1024,1024,26483000,1376,1024,1024,25833100,1413,1024,1024,24793200,1450,1024,1024,23783300,1485,1024,1024,23153400,1522,1024,1024,22503500,1558,1024,1024,21853600,1589,1024,1024,21363700,1621,1024,1024,20853800,1654,1024,1024,20323900,1683,1024,1024,19934000,1713,1024,1024,19564100,1745,1024,1024,19204200,1774,1024,1024,18834300,1799,1024,1024,18534400,1824,1024,1024,18244500,1846,1024,1024,17924600,1872,1024,1024,17654700,1896,1024,1024,17364800,1917,1024,1024,17114900,1935,1024,1024,16865000,1956,1024,1024,16595100,1975,1024,1024,16365200,1993,1024,1024,16115300,2013,1024,1024,15945400,2032,1024,1024,15775500,2052,1024,1024,15605600,2072,1024,1024,15425700,2093,1024,1024,15295800,2110,1024,1024,15135900,2127,1024,1024,15006000,2144,1024,1024,14856100,2158,1024,1024,14756200,2171,1024,1024,14646300,2185,1024,1024,14526400,2198,1024,1024,14426500,2212,1024,1024,14316600,2226,1024,1024,14216700,2241,1024,1024,14096800,2255,1024,1024,14006900,2270,1024,1024,13897000,2284,1024,1024,13787100,2300,1024,1024,13717200,2310,1024,1024,13647300,2320,1024,1024,13557400,2330,1024,1024,13487500,2341,1024,1024,13417600,2351,1024,1024,13327700,2367,1024,1024,13267800,2378,1024,1024,13177900,2389,1024,1024,13118000,2399,1024,1024,13048100,2411,1024,1024,12968200,2422,1024,1024,12908300,2433,1024,1024,12828400,2444,1024,1024,12778500,2450,1024,1024,12738600,2461,1024,1024,12668700,2467,1024,1024,12628800,2473,1024,1024,12578900,2485,1024,1024,12519000,2491,1024,1024,12479100,2497,1024,1024,12429200,2509,1024,1024,12379300,2515,1024,1024,12329400,2521,1024,1024,12289500,2533,1024,1024,12229600,2539,1024,1024,12189700,2545,1024,1024,12149800,2558,1024,1024,12089900,2564,1024,1024,120410000,2570,1024,1024,1198
Code: [Select]
Daylight,5200,1993,1024,1024,1611
Shade,7000,2284,1024,1024,1378
Cloudy,6000,2144,1024,1024,1485
Tungsten,3200,1450,1024,1024,2378
Fluorescent,3714,1751,1024,1024,2280
Flash,6320,2217,1024,1024,1460

For instance also the shutter sounds a bit different on these, so maybe there's some little design difference?

I also doublechecked that no WB shift was set on both and that both had same settings, same standard profile, disabled all "enchance" functions, etc.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 02, 2014, 03:57:50 PM
@kaco
thanks a lot. i assumed, that every sensor has got its own indivdual adjustment - maybe once on build or maybe once on switch on or firmware-update.. means, every 5DIII has a slightly other characteristic curve. but for the job as a converter-app with static rggb-values it should be enough - OR - i/we apply this data as external files - i dont think its worth it.

now could be interesting to see, how the curves differ on other models.

thanks and regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on August 02, 2014, 04:02:22 PM
For me it's totally OK to use some "average" values. This values will still be used only when creating proxies and then finetuned in the final color grade.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on August 03, 2014, 08:34:43 AM
Im having sync issues when I autosync in Resolve. Theres a consistent gap at the beginning of the audio. Anyone else having this issue or other audio sync issues?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 06, 2014, 11:01:33 AM
@jbowdach
i just did a small test with ~25 files, recorded (in terms of timecode) in nearly same time. after autosync it went good, no gaps, no real problems. i assume, my timecode-calculations for the audiotrack going wrong if they're broken framerates. did you checked the timecode inside the files?

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fresolve_timecode_sync_audio.jpg&hash=d00f20e1cc19945a688d4b230939d2ad)

just found that: maybe thats (http://www.magiclantern.fm/forum/index.php?topic=11017.msg124784#msg124784) kind of similar?

regards chmee

btw: i implemented the WBAL-usage and resolve, premiere and photoshop behaves quite well. did some more exception-trapping and debug-writing. beta7 is near.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on August 08, 2014, 12:10:47 AM
btw: i implemented the WBAL-usage and resolve, premiere and photoshop behaves quite well. did some more exception-trapping and debug-writing. beta7 is near.

very nice!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 08, 2014, 12:54:32 AM
raw2cdng 1.5.0 beta7 online

downloadlink (http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/)

* 12bit shouldnt crash anymore with big/strange resolutions
* WBAL from MLV or CR2 (presets and manual as well) test!
* debuglog reorganized - readability
* if ffmpeg.exe in same folder - proxy output will be as mpeg2 as well (half resolution) (you see mpeg2 in button if ffmpeg is recognised)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on August 08, 2014, 01:38:59 AM
@chmee I think I figured my issue. I was bringing in all the files in one folder. Didnt realize I need to bring in EACH folder individually, and use the autosync option on the folder for each clip. Im going to try a mass sync tonight and see how it work along w beta7, as Ive had a few crashes w beta6 (mainly related to scrolling when it is reviewing the clips after dragging them).

Many thanks for the update and testing! :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 08, 2014, 09:11:01 AM
you can drag/drop them all first into the mediapool. after that you can select them (all) and use autosync. should work.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 08, 2014, 07:46:37 PM
@jbowdach crash issues mentioned on bitbucket (https://bitbucket.org/chmee/raw2cdng/issue/20/crash-when-attempting-to-drag-in-mlv-files)

first debug: [beta7] failure on DayFive/M01-0925.MLV (Exception listed at the end, couldnt read MLV because of negative fileofffset, zero frames?!)
second debug: [beta7] failure on DaySeven/M03-1132.MLV (fpsnom is zero and found no VIDF-Block)
third debuglog: [beta6] hmm. Day Five\M01-1214.MLV. somewhere while identifying VIDF-Blocks. please try this one with beta7.

mainly, it seems, something disturbes the reading of the vidf-blocks and/or the blocks at all. If you ha a shorter file from your recording you could send me, it would be great. Did you tried to read the files with mlvdiag (https://secure.magiclantern.fm/forum/index.php?topic=5618.msg119737#msg119737)? shorten a file with it and send it to me. or use HxD and copy the first ~50MB of M03-1132 (and/or M01-0925.MLV) into a new file. send this one.

bytheway: Thanks a lot for the help (and patience) to you all!!

edit:

This is raw2cdng 1.5.0 beta7 under win7/win8 with aero and enabled ffmpeg.exe
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fraw2cdng_150b7.jpg&hash=c3ed07451c3488ed9a6b6f6586d92477)
and an hour ago i put mlvdiag into the raw2cdng-zip (download it again if needed). now in version 1.0.1 with a light helper. just drag/drop a file on the rectangle on bottom, and it will be copied into a 55mb big file with the ending ".55mb.mlv". no checks, just to copy the first 55mb of the main file to give me a chance to deal with some of the mysterious files.
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fmlvdiag_101.jpg&hash=5971f35435d602a89fc533d18cba2bce)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on August 08, 2014, 11:02:58 PM
Will do with the problem files so you can take a look at them. No clue why these are recording with errors, as they were all recorded at the same time with the same build on the same cards. What's strange as I had 4 previous days FULL of MLVs that decoded without any crashes......(confused). Not sure why it started having issues on Day 5 and 7 (didnt shoot on day 6)

No worries, thank you for your help in diagnosing the issues and debug logs.

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on August 09, 2014, 05:19:08 AM
@chmee

Please find the links below to the files you requested.

https://dl.dropboxusercontent.com/u/787062/raw2cdng/M01-0925.MLV.55mb.mlv (https://dl.dropboxusercontent.com/u/787062/raw2cdng/M01-0925.MLV.55mb.mlv)
https://dl.dropboxusercontent.com/u/787062/raw2cdng/M01-1214.MLV.55mb.mlv (https://dl.dropboxusercontent.com/u/787062/raw2cdng/M01-1214.MLV.55mb.mlv)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 09, 2014, 08:44:04 AM
please give me DaySeven/M03-1132.MLV as well. on a short test i could use both shortened mlv's without problems.
Are your problematic files seeable in mlvdiag (dragdrop into list)?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on August 11, 2014, 06:40:31 AM
The file appears to be damaged, this is the file as copied off the card. All files came up fine in mkvdiag.
https://dl.dropboxusercontent.com/u/787062/raw2cdng/M03-1132.MLV (https://dl.dropboxusercontent.com/u/787062/raw2cdng/M03-1132.MLV)

On another note, I all of a sudden cant launch raw2cdng. Something happened w my system. ever seen anything like the following?


Problem signature:
  Problem Event Name:   CLR20r3
  Problem Signature 01:   raw2cdng.1.5.0.beta7.exe
  Problem Signature 02:   1.5.0.0
  Problem Signature 03:   53e3fb45
  Problem Signature 04:   System.Web.Extensions
  Problem Signature 05:   4.0.30319.18408
  Problem Signature 06:   523111a8
  Problem Signature 07:   1a2
  Problem Signature 08:   ed
  Problem Signature 09:   System.Windows.Markup.XamlParse
  OS Version:   6.1.7601.2.1.0.256.4
  Locale ID:   1033
  Additional Information 1:   0a9e
  Additional Information 2:   0a9e372d3b4ad19135b953a78882e789
  Additional Information 3:   0a9e
  Additional Information 4:   0a9e372d3b4ad19135b953a78882e789

EDIT: Not sure what caused it, uninstalled\reinstalled .net and still didnt work. Ended up restoring an image backup to a few days ago (lucky me!)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: pakal on August 11, 2014, 11:30:28 PM
Hi, first of all thanks for that neat piece of software!
I use it to get mp4 proxies but I have an issue about the color though, the jpegs are off, lifted blacks, wrong gamma.

I copied ffmpeg in the folder and used the proxy option.
Is there a way to get jpegs with the same look as the dng please?

I searched the thread but couldn't find any workaround for this, I apologize if this has been answered before.

Thanks!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 12, 2014, 12:11:02 AM
(answering from a smartphone) the proxy-option was/is only for cutting purposes. but i will optimize it in the next Version..
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: pakal on August 12, 2014, 09:22:16 AM
(answering from a smartphone) the proxy-option was/is only for cutting purposes. but i will optimize it in the next Version..

thanks for the quick answer and thanks for the work, it's the only program that has worked with MLVs for me so far
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on August 12, 2014, 10:38:18 PM
@chmee: in beta7 there is still present the bug that the app will crash when scrolling all loaded MLV files during convert
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on August 12, 2014, 11:41:25 PM
+ it would be perfect if folder drop will be working. Now I use 3 cameras -- I have to copy the mlvs into folders "card1, card2,.." because 2 cameras can create same named files. Then I convert MLVs to cdngs with namespace "[D2]_[F]_[P]_". But I have to manually drop every folder, which is time consuming if I have 40 cards and so.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on August 12, 2014, 11:46:01 PM
And very nice future feature would be progress bar in the taskbar button. You know like when you copy files with explorer, the icon in the taskbar will start to fill its background with green. So you know the app is working and you'll see once it's done :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on August 12, 2014, 11:53:45 PM
I have a problem converting one file. It's the last file on the card, so maybe it's corrupted, but extracts with MLV Converter 1.9.2 fine.

Here's debug:
Code: [Select]
------------- 1.5.0.BETA7 started at 14.08.12 23:49:48 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M00
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M01
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M02
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M03
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M04
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M05
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M06
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M07
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M08
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.MLV
[drop] ------
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M00 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M01 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M02 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M03 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M04 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M05 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M06 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M07 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M08 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (10)
[createMLVBlocklist] indexing Blocks in O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.MLV
[createMLVBlocklist] --- exception(!) ---
[createMLVBlocklist] blockTag|offset|fileNo : &???|-199613494|0
Message = Non-negative number required.
Parameter name: value
ActualValue =
ParamName = value
Data = System.Collections.ListDictionaryInternal
InnerException =
TargetSite = Void set_Position(Int64)
StackTrace =    at System.IO.FileStream.set_Position(Int64 value)
   at raw2cdng_v2.io.createMLVBlockList(String filename, raw raw)
HelpLink =
Source = mscorlib
HResult = -2146233086

Also won't open with mlv diag, probably the same problem. The progress is stuck on loading.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: edge11 on August 13, 2014, 11:37:32 AM
Is there any way to integrate pink dot removal or chromasoothing for us t4i users?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 13, 2014, 07:40:12 PM
@kaco
thanks for ideas.
folderimport is coded (useable with P-Tag, as you've shown)
progress on taskicon, i will see, minor importance.

your file, thats crashes the app, seems to deliver wrong offset-data straight on first block. thats strange and could be a hint on a undeleted value somewhere. will do some longer recording to test it. could be equal to @jbowdachs problems.

@edge11
sorry, i didnt found the time to implement it.. its on the list and if this is done i'd change the versionnumber to some more :) if someone could help me, would be great.

by now i'm on holidays, for the next 10 days. i try to bugfix and implement as much as possible.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: zacshome on August 14, 2014, 11:51:29 AM
chmee is there anyway to mass import the jpg sequences into premiere? it seems i have to do them all individually.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on August 14, 2014, 03:10:30 PM
I don't know if debugging and impletementing your program is a holiday for you or not but, thanks for being a team player! I know ML users will be happy!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on August 14, 2014, 04:36:28 PM
chmee is there anyway to mass import the jpg sequences into premiere? it seems i have to do them all individually.

in premiere open the import windows, select the first file of the sequence and in the window there should be checkbox "numbered stills". Then you import jpeg sequence as a "video" and you can change properties and set correct FPS.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on August 14, 2014, 10:13:10 PM
I don't know if debugging and impletementing your program is a holiday for you or not but, thanks for being a team player! I know ML users will be happy!

Another massive thanks for a happy ML \ Cdng user. You've created the best solution for MLV to cdng for windows so far, otherwise we'd still be stuck in DNG sequence land
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on August 15, 2014, 04:57:41 PM
very similar problem as the last one:
Code: [Select]
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M00 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M01 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M02 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M03 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M04 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M05 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M06 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M07 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M08 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (10)
[createMLVBlocklist] indexing Blocks in O:\2014-06-23 Freedomky MLV\M23-1239.MLV
[createMLVBlocklist] --- exception(!) ---
[createMLVBlocklist] blockTag|offset|fileNo :
x6?|-1118282230|0
Message = Non-negative number required.
Parameter name: value
ActualValue =
ParamName = value
Data = System.Collections.ListDictionaryInternal
InnerException =
TargetSite = Void set_Position(Int64)
StackTrace =    at System.IO.FileStream.set_Position(Int64 value)
   at raw2cdng_v2.io.createMLVBlockList(String filename, raw raw)
HelpLink =
Source = mscorlib
HResult = -2146233086
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 17, 2014, 04:10:54 PM
sorry guys. debugging on bigger files does not work here where i am, instead of this i coded a neat extra and optimized the proxy output.

raw2cdng 1.5.0 beta8 (http://dl.phreekz.de/raw2cdng/raw2cdng.1.5.0.beta8.zip) is online.
* mlvplayer embedded - glue the file extension MLV to this app and lets play it :) RAW not tested yet. (no audio)
* whitebalance optimized (hopefully) on jpg/mpeg2 output
* no rec709-log/lut but gamma 2 on proxyoutput.
* fixed glitch on mpeg2-output (length was taken by audio, now it should be ok)
* folder dragdropable. Use P-tag.

for sure the playback is not accurate, but massively helps to overview recordings. and it looks better than ffplay (sorry, this was my first idea)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jbowdach on August 18, 2014, 06:16:00 AM
sorry guys. debugging on bigger files does not work here where i am, instead of this i coded a neat extra and optimized the proxy output.

raw2cdng 1.5.0 beta8 (http://dl.phreekz.de/raw2cdng/raw2cdng.1.5.0.beta8.zip) is online.
* mlvplayer embedded - glue the file extension MLV to this app and lets play it :) RAW not tested yet. (no audio)
* whitebalance optimized (hopefully) on jpg/mpeg2 output
* no rec709-log/lut but gamma 2 on proxyoutput.
* fixed glitch on mpeg2-output (length was taken by audio, now it should be ok)
* folder dragdropable. Use P-tag.

for sure the playback is not accurate, but massively helps to overview recordings. and it looks better than ffplay (sorry, this was my first idea)

regards chmee

Many thanks for the folder option, what do you mean by using the p-tag? For what reason? Seems to work just by dragging a folder
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 18, 2014, 08:36:49 AM
not a Must, but a hint for the filename-generator.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: zacshome on August 20, 2014, 04:47:01 AM
Hey Chmee, trying to get the mpeg proxies to work, I got the exe for ffmpeg in where raw2cndg is and it has the mpeg option next to the jpg proxi text, i click this hoping to make the mpeg proxies but it still just creates jpg's.

Any help with this? Essentially do they also have the same file name, so if i edl//xml the proxies out of premiere into resolve, they will link strait up with the dng files?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 20, 2014, 09:10:37 AM
Look into the debug. The ffmpeg error should stay there. The mpgs should be indeed in the same folder as the dngs and jpgs.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: mrd777 on August 22, 2014, 06:14:09 AM
Chmee,

All my timecode are wrong. I use Resolve and when I import the CDNG sequence it starts at like 17th hour time code. I just want the timecode to be zero. Because when I export into my editor and use After Effects, it starts on 170000+ starting frames!

Let me know what I can do?

Thanks,
Mr D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 22, 2014, 10:15:58 AM
@mrd77 are the tc-values really wrong or do you just dont need them? Cant you disable or manually reset tc in any app?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: mrd777 on August 22, 2014, 05:32:55 PM
timecodes are correct. However, in After Effects, it makes all clips start on frame 170000 or more. It's really high up there and hard to deal with when matching things up between VFX programs, I find. I just found out how to make TC back to zero in Resolve, so I guess I'm good now!  8)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 22, 2014, 05:51:57 PM
Good To hear.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jonas18z on August 25, 2014, 09:16:18 PM
I have problems to run the 1.50 beta8 and older versions under a virtual machine (Windows7) tested on Parallels and VMvare ESXI two different machines and same problem. I can add a file and press convert but I don't se any progress bar or anything else.


Doesn't the software work under VM?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 25, 2014, 10:07:31 PM
cant tell you, never tried that. you need .net4.0 to use it.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jonas18z on August 26, 2014, 12:14:25 AM
I got .net 4.5. Windows 7 all uppdates on Winodws.
Machine #1
Mac pro  mid 2010, 16RAM, nvidia quadra 4000. 10.9.4
Parallells Desktop 9 (updated)
Windows 7 (updated)
 
Machine #2
Asus P9D-I
Intel Xeon E3-1265Lv3 2,5GHz
16GB ECC RAM
ESXi 5.5 + Windows 7


The software starts correctly on boths machines but when I press convert nothings more happens. Waited >1h for a 250MB MLV-file.

Posted the debuginfo. Did you add some Anti-Virtual Machine and Anti-Sandbox in the software?
Do you need any special system requirements, cpu speed, cores, ram, hdd, GPU?


Code: [Select]
------------- 1.5.0 BETA8 started at 14.08.25 21:12:50 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108.MLV
[drop] ------
[drop] -- file C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108.MLV.
[drop] -- file C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
 ***** [getMLVAttributes] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
 *****
 ***** [getMLVAttributes] metaData Object:
xResolution=1920
yResolution=804
frames=246
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrix=System.Byte[]
lostFrames=0
fpsNom=25000
fpsDen=1000
dropFrame=False
fpsString=25,00
stripByteCount=2701440
stripByteCountReal=2701440
modell=Canon EOS 5D Mark III
camId=MAGIC737d375
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
jpgConvertR=0,540084388185654
jpgConvertG=1
jpgConvertB=0,547008547008547
whiteBalance=4500
whiteBalanceMode=0
whiteBalanceGM=4400
whiteBalanceBA=4400
blackLevelOld=2046
blackLevelNew=2046
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5,05913231434306
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.5.0 BETA8
versionString=System.Byte[]
propertiesString=
 *****
 ***** [getMLVAttributes] fileData Object:
convertIt=False
fileName=C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108.MLV
fileNameOnly=M20-2108
fileNameShort=M202108F
fileNameNum=202108
tempPath=C:\Users\jonte\AppData\Local\Temp\
sourcePath=C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=2014-08-25 21:12:40
modificationTime=2014-08-20 21:07:56
VIDFBlock=
RAWBlock=
 *****
 ***** [getMLVAttributes] lensData Object:
lens=EF50mm f/1.4 USM:
focalLength=50
aperture=160
isoValue=1600
shutter=1/50s
 *****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 648
[drop] hasAudio. AUDF-List created. Blocks: 50
[drop] VIDF-List created. Blocks: 244
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[readVIDFBlockData] done: [100][101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124]
[readVIDFBlockData] done: [125][126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143][144][145][146][147][148][149]
[readVIDFBlockData] done: [150][151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174]
[readVIDFBlockData] done: [175][176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199]
[readVIDFBlockData] done: [200][201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224]
[drop] VIDF-Blockdata read and created.
[drop] adding M20-2108 to batchList
[drop] ** Item M20-2108 imported | MLV
[drop] * res 1920x804px | 25,00fps | 244 frames | BL2046 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 244 | has audio
[doWork] started
[doWork] all in all there are 244 frames to convert
[doWork] 1 Cores used
[doWork] -> converting item M20-2108
[doWork] settings format 0 with BL2046 WL15000 with+out maximizingvalue 5,05905511811024
[doWork] prefix - filename generator [F]
[doWork] destinationPath -> M20-2108
[doWork] outputFilename  -> M20-2108
[doWork] Directory C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork][for] read/convert MLV VIDF: [0]
[doWork][for] read/convert MLV VIDF: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]
[doWork][for] read/convert MLV VIDF: [26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50]
[doWork][for] read/convert MLV VIDF: [51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75]
[doWork][for] read/convert MLV VIDF: [76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99][100]
[doWork][for] read/convert MLV VIDF: [101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124][125]
[doWork][for] read/convert MLV VIDF: [126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143][144][145][146][147][148][149][150]
[doWork][for] read/convert MLV VIDF: [151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174][175]
[doWork][for] read/convert MLV VIDF: [176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199][200]
[doWork][for] read/convert MLV VIDF: [201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224][225]
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 26, 2014, 07:47:47 AM
it starts and converts. so the .net-framework is working properly. the debug shows an output into folder
Code: [Select]
C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108but it does not output audio. there are some strange things i have to check in my source. please try with a sample file from here (http://www.magiclantern.fm/forum/index.php?topic=11899.0).

(if the progressbars dont move, seems like a problem with (wpf/silverlight inside) the vm's)
converting 250mb is a no-brainer, should be done in 4-5 seconds.

Quote
..Did you add some Anti-Virtual Machine and Anti-Sandbox in the software?..
why should i? its a free software without special ambitions (indirect adware only working on native win-systems? :D now i like the idea, if intel or microsoft would pay me "a lot" for that)
 
regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: zacshome on August 26, 2014, 10:50:12 AM
Hey Chmee, Awesome, got the mpg's to work!! only thing is, it takes for ever to import them all into premiere because i have to go into each folder individually which wont work if im shooting a proper gig with 300 clips etc.

Secondly, once i export the xml out of premiere, i cant get the mpgs to link with the dngs in resolve. Any answers to these?

otherwise, love your buddy.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 26, 2014, 11:01:40 AM
by now i do not work with proxies, so i cant tell you a proper way to work with them in terms of edl/xml. (edit: please tell, how you fixed the ffmpeg-thing, maybe someone will be thankful for this info)

But: Importing: Open an File-Explorer, do a search on *.mpg and dragdrop the files into premiere-bin. should be the fastest way.

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jonas18z on August 26, 2014, 11:20:59 AM
it starts and converts. so the .net-framework is working properly. the debug shows an output into folder
Code: [Select]
C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108but it does not output audio. there are some strange things i have to check in my source. please try with a sample file from here (http://www.magiclantern.fm/forum/index.php?topic=11899.0).

(if the progressbars dont move, seems like a problem with the vm's)
converting 250mb is a no-brainer, should be done in 4-5 seconds.
why should i? its a free software without special ambitions (indirect adware only working on native win-systems? :D)
 
regards chmee


Tyyed with all four MLV file in second post and same problem, nothing happens when i press convert.  Tryed Two diffrent machines parallells and and Vmware ESXi.
Same problem and no problem with a real Windows 7 machine.



Code: [Select]
------------- 1.5.0 BETA8 started at 14.08.26 11:17:11 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON.MLV
[drop] ------
[drop] -- file C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON.MLV.
[drop] -- file C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
[getMLVAttrib] [!] NO WAVI Block
 ***** [getMLVAttributes] metaData Object:
xResolution=1600
yResolution=900
frames=15
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrix=System.Byte[]
lostFrames=0
fpsNom=23976
fpsDen=1000
dropFrame=False
fpsString=23,98
stripByteCount=2520000
stripByteCountReal=2520000
modell=Canon EOS 5D Mark III
camId=MAGIC6e4781b
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
jpgConvertR=0,540084388185654
jpgConvertG=1
jpgConvertB=0,547008547008547
whiteBalance=4500
whiteBalanceMode=0
whiteBalanceGM=3200
whiteBalanceBA=3200
blackLevelOld=2048
blackLevelNew=2048
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5,05991352686844
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.5.0 BETA8
versionString=System.Byte[]
propertiesString=
 *****
 ***** [getMLVAttributes] fileData Object:
convertIt=False
fileName=C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON.MLV
fileNameOnly=5DIII_113_14apr13_blfixON
fileNameShort=5DIII113
fileNameNum=51131413
tempPath=C:\Users\jonte\AppData\Local\Temp\
sourcePath=C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=2014-08-26 11:02:18
modificationTime=2014-08-26 11:03:25
VIDFBlock=
RAWBlock=
 *****
 ***** [getMLVAttributes] lensData Object:
lens=120-300mmI:
focalLength=235
aperture=280
isoValue=100
shutter=1/33s
 *****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 47
[drop] VIDF-List created. Blocks: 15
[readVIDFBlockData] started
[drop] VIDF-Blockdata read and created.
[drop] adding 5DIII_113_14apr13_blfixON to batchList
[drop] ** Item 5DIII_113_14apr13_blfixON imported | MLV
[drop] * res 1600x900px | 23,98fps | 15 frames | BL2048 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 15 | has no audio
[doWork] started
[doWork] all in all there are 15 frames to convert
[doWork] 1 Cores used
[doWork] -> converting item 5DIII_113_14apr13_blfixON
[doWork] settings format 0 with BL2048 WL15000 with+out maximizingvalue 5,05983631871526
[doWork] prefix - filename generator [F]
[doWork] destinationPath -> 5DIII_113_14apr13_blfixON
[doWork] outputFilename  -> 5DIII_113_14apr13_blfixON
[doWork] Directory C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork][for] read/convert MLV VIDF: [0]
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 26, 2014, 11:49:14 AM
@jonas18z
sorry, cant help you and i dont have the time to test under vm's as well. if someone got some hints, fine, will try to solve it.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jonas18z on August 26, 2014, 01:17:07 PM
Understand! Anyone else got the same problem under VM?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jonas18z on August 27, 2014, 02:43:59 AM
Compiled the software and it seems that it stucks on:

Code: [Select]
Line 572 in MainWindows.xaml.cs

  cde.Wait();   

Removed that line.
  // cde.Wait();

And now the software produce all frames .DNG and .WAV files even if I add more then one file. But the progress bar stucks on 95% or something like that but no problems to add more file after that :)






Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 27, 2014, 08:14:54 AM
are you able to give your vm more than one cpu? then try again with cde.wait(). its waiting, 'til all workerthreads ended..

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jonas18z on August 27, 2014, 10:57:54 AM
Gave the VM 2 cpu and now it works prefectly!

so 1 CPU wil fail >1 CPU works....


 
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 27, 2014, 12:58:50 PM
thankful for exploring this behaviour.

thank you. regards chmee

edit: thanks to @beauchampy, sent me some q7-samples, DNG-Data (from dng_validate)
Code: [Select]
Validating "q7_2k.dng"...

Uses little-endian byte order
Magic number = 42

IFD 0: Offset = 8, Entries = 42

NewSubFileType: Main Image
ImageWidth: 2064
ImageLength: 1088
BitsPerSample: 12
Compression: Uncompressed
PhotometricInterpretation: CFA
FillOrder: 1
Make: "ODYSSEY 7Q RECORDER"
StripOffsets: Offset = 4096
Orientation: 1 - 0th row is top, 0th column is left
SamplesPerPixel: 1
RowsPerStrip: 1088
StripByteCounts: Count = 3368448
PlanarConfiguration: 1
Software: "2.10.141"
CFARepeatPatternDim: Rows = 2, Cols = 2
CFAPattern:
    Red      Green
    Green    Blue
DateTimeOriginal: 0000:00:00 00:00:00
DNGVersion: 1.1.0.0
DNGBackwardVersion: 1.0.0.0
UniqueCameraModel: "FS700"
BlackLevelRepeatDim: Rows = 1, Cols = 1
BlackLevel: 256.00
WhiteLevel: 4095
DefaultScale: H = 1.0000 V = 1.0000
DefaultCropOrigin: H = 8.00 V = 4.00
DefaultCropSize: H = 2048.00 V = 1080.00
ColorMatrix1:
      0.6290  -0.1537  -0.0334
     -0.3453   1.1168   0.2371
     -0.1034   0.1852   0.5259
CameraCalibration1:
      1.0000   0.0000   0.0000
      0.0000   1.0000   0.0000
      0.0000   0.0000   1.0000
AsShotNeutral: 0.4186 1.0000 0.5604
BaselineExposure: +0.00
BayerGreenSplit: 0
CameraSerialNumber: "SERIAL"
CalibrationIlluminant1: 5400K
BestQualityScale: 1.0000
ActiveArea: T = 0 L = 0 B = 1088 R = 2064
ProfileEmbedPolicy: Allow copying
ProfileCopyright: "Sony FS700"
*Tag51043: Byte, size = 8
    22 05 20 06 00 00 00 00                            ". .....       
*Tag51044: SRational = 50/1
*Tag51081: "Reel:R001 Sc:SCENE1  "
*Tag51105: "Cam:A Day:0 1"
NextIFD = 0

Validation complete
it seems, its 12bit linear without a linearizationtable
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: zacshome on September 01, 2014, 02:11:37 AM
The way I fixed ffmpg was purely by removing mlviewer.. Not sure if that is a legit fix....but I thought it may be messing with the code somewhere... so i gave it a go and it worked.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kgv5 on September 06, 2014, 01:42:43 PM
I dont know if this is a known bug but the app (1.5.0 beta 6, tried beta 7 and it seems that it has the same problem) crashes very often when i am trying to import more than 10-15 files. Just saying something like "program stop working". Because of that i  need to put files in small portions, its annoying because leaving app working overnight with dozens of file is impossible. I have also noticed than it crashes every time im trying to import 0,5 fps clips (made with fps override). I am on windows 7 64bit. Is there any log file or something i could post so you could see what is going on?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on September 06, 2014, 01:53:33 PM
Thanks for info. There is a debug-button on top. Click, close and open once again.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kgv5 on September 06, 2014, 02:33:59 PM
OK, here is log, tried to put more than 20-25 files and crashed, after that i tried to put mixed fps files 23,976 and 0,5 fps - crashed when importing 0,5fps files and in the third attempt i put only 0,5fps files and crashed immidiately.

https://www.dropbox.com/s/h4xdl1nmprlrrcc/problems%20with%20raw2cdng.2.debug.log?dl=0
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on September 06, 2014, 02:40:50 PM
Please send me a fps-overridden file, that was shortened with mlvdiag (inside the beta6 or 7) Ah.. Open the taskmanager and look on the memory usage while you import this amount oft files. Does it reach 1.5gb? Could be a hint for a memory leak..
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kgv5 on September 07, 2014, 09:12:17 AM
Here is fps overriden mlv file (0,5 fps). I tried and it crashed the app so it should be good for checking.

https://www.dropbox.com/s/kf06lba745aq80d/M07-0857.MLV?dl=0
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on September 08, 2014, 11:02:38 PM
short before v1.6.0.. i need some videos having verticalbanding. the best would be to provide it to the whole coder-comunity, we could test our algorithms with it. put it into the samplefiles-thread (http://www.magiclantern.fm/forum/index.php?topic=11899.0). thanks a lot.

@kgv5
yeah found the bug, i truncated framerates for the timecode. therefore it was a "division by zero". fixed. framerates below 1 will be now 1. shouldnt be a problem for any workflow. (v1.6.0)

@zacshome
strange, but if it helped. ok.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on September 09, 2014, 10:39:51 PM
raw2cdng 1.6.0 online
* bugfixes
* GUI slightly changed
* with thumbnail and histogram
* mlvplayer
* rec709 on proxies (jpg, mpeg2 and h264 with ffmpeg)

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fraw2cdng.1.6.0.jpg&hash=39c5c39b781853f2abc78df268352ef7)

download
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

(i need files with verticalbanding!!!) regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on September 10, 2014, 09:34:10 PM
I have a 5D3 that has vertical banding. Any particalur things you want me to shoot for you? Lens cap on, blown out, underexposed, time limit? Just let me know and I could send you some.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kgv5 on September 10, 2014, 11:43:33 PM
@chmee

wow, i have just imported 46 mlv files at once and seems that my previous problem with importing many files is gone :) I need to make some further testing though but looks good now. Love the new UI, those previews are great and work so nicely:) Do you think it is possible to implement skipping through preview (with mouse clicking) without a need to watch all thing from the beginning to the end?  Thank you for your work :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on September 11, 2014, 12:33:51 PM
@ChadMuffin
nothing especial, my 5DIII does a good job and the diferences between the channels are quite small. to check if my code is working i need some files i can compare with a1ex's code results. send it to raw2cdng@phreekz.de. thanks a lot. please take footage that i can put into the samplefiles-thread, so other coders can participate without injuring your privacy.

@kgv5
cheers. idea is good, realisable
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on September 12, 2014, 05:26:28 PM
@chmee - Sent you a Google Drive link.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: tob on September 18, 2014, 02:18:23 PM
Why do I get error with audio when I try to replace a proxy .mov file with the cDNG files  in Premiere pro cc 2014.0.1  osx.
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2F112.imagebam.com%2Fdownload%2F6gtr4QnmQj7_blm3MrTo4A%2F35209%2F352080622%2Faudioproblem.png&hash=8de3827b69cd302c499e3ab513aa4509) (http://www.imagebam.com/image/15a066352080622)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on September 18, 2014, 03:08:28 PM
@chadmuffin
already downloaded, thanks. will give an info next days.

@tob
seems, like proxy handling doesnt like mediafile (video with embedded audio) versus splitted video and audio-files. will do a try with an audioless proxyfile. this ones should take automatically the wav-file.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on September 19, 2014, 11:04:33 PM
thanks @chadmuffin. vertical Banding will be in 1.6.1 (next days) i assume its not that perfect as a1ex algorithm, but it works :)
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fraw2cdng_verticalbanding.jpg&hash=170a21c78760209a2923ab8cd6ab9ce8)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on September 20, 2014, 06:23:08 AM
Glad to help and appreciate all your hard work! Looking forward to the next release!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: a1ex on September 20, 2014, 08:19:42 AM
@chmee,

Here's a sample with vertical banding, that couldn't be fixed with my algorithm: http://www.magiclantern.fm/forum/index.php?topic=13291

Maybe your algorithm has better luck here?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: allemyr on September 21, 2014, 11:37:38 AM
I love RAW2CDNG! Always using it with my MLV and RAW files. The audio syncing in Davinci Resolve from the MLV files with your app couldn't be better, thanks for that!

/ Karl Allemyr

http://www.allemyr.com/
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on September 21, 2014, 06:27:28 PM
raw2cdng 1.6.1 is online
* vertical banding (really simple algorithm, but seems to work)
* WPF re-coding by rd4eva - was necessary - thus happend some changes to the ui, will be re-optimized next version

Download raw2cdng 1.6.1 (http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: allemyr on September 25, 2014, 02:23:24 PM
Hi!

Thank you for a really great app. If its possible to unsqueeze 60 FPS 1920x672 to 1920x1080 inside RAW2CDNG? Would be helpfull. Having a a kind of problem inside Davinci Resolve right now, mixing 23,976fps and 60fps.

Thanks in advance!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on September 26, 2014, 09:48:58 AM
@allemyr
this problem is quite special and it was my intention to leave the raw-data as original as possible. aproportional rescaling means i'd recalculate the raw-data, therefore the quality would change depending on algorithm. no, too hot, sorry. dng-specification has no "pixel aspect ratio" tag - this would be a solution for me.

but: if read correctly, there's a "edit PAR value" property in Resolve.
resolve 11 - manual (http://software.blackmagicdesign.com/DaVinciResolve/docs/DaVinci_Resolve_11_Manual_June_2014.pdf) page 644
Quote
..Input Sizing Adjustments.. additional MODIFY PAR button.. change the pixel aspect ratio of a clip

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: N/A on September 28, 2014, 06:53:40 AM
Looks like we need a Mac version since the-app-we-shall-not-speak-of is no longer supported on this forum.

 :D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on October 02, 2014, 03:27:47 PM
The vertical banding fix seems to be working great for me. However, I tried 12 bit max, 16 bit and 16 bit max with no luck of importing them into Premiere Pro CC. It would state the files are damaged or corrupt. Possible user error? When I gave it files in one folder it worked fine but, I had files in 5 different folders and it placed them in the batchlist, it loaded them fine but when I hit convert, it crashed. I love the new GUI and the simplicity. You're doing what a company hires a team for! Great work!

Update: There were about 50 MLVs between the 5 folders. And, I had vertical banding fix checked with the coversion before attempting to use them in Premiere Pro CC.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 02, 2014, 04:45:39 PM
thanks for the info, will investigate next days. (i did tests with 3 folders and about 25 mlv's and raw's, and it worked, so, send me or copy/paste (http://pastebin.com/) a debuglog)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on October 06, 2014, 03:34:40 AM
Hi, Chmee. Apologies if this been answered somewhere else, I couldn't seem to find the answer to these:

Does raw2cdng extract wav out of MLV with sound? I'm not getting wav on the output for some reason.

and

Is it possible to put all the .dng output in one folder instead of separate folder for each clip?

Thanks! Great work on that vertical banding! It fixed a lot of the banding I can't seem to get rid of with the other stuff.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 06, 2014, 07:51:26 AM
(a) yes, wav should be extracted as well. does raw2cdng show ( with audio) behind the filename? look here (http://dl.phreekz.de/raw2cdng/raw2cdng.1.6.0.jpg)
(b) hmm. dont see any benefit beside the fact, it fastens the importing into premiere (but leaves a chaotic harddisk) :) you could manage that with a search *.dng in windows and copy them into one folder.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on October 06, 2014, 02:22:11 PM
(a) yes, wav should be extracted as well. does raw2cdng show ( with audio) behind the filename? look here (http://dl.phreekz.de/raw2cdng/raw2cdng.1.6.0.jpg)

Ah! It doesn't say! No wonder. Thanks!

(b) hmm. dont see any benefit beside the fact, it fastens the importing into premiere (but leaves a chaotic harddisk) :) you could manage that with a search *.dng in windows and copy them into one folder.

Yeah, that's what I've been doing. Searching *.dng and moving them into one folder to speed up importing to premiere. Sure, it makes it one big lump but premiere sorts them out just fine. Would've been great if that can be done at the output level though. :D It takes a while for my machine to find all the *.dng and move them out.

Thanks again! Great app!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 06, 2014, 02:56:21 PM
If you dont copy but move, it will work much faster.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on October 07, 2014, 03:16:35 AM
Yeah, I've been moving them instead of copying, but still. Finding then moving tens of thousands of small files takes quite a lot of time.

By outputting to a single folder for all the clips instead of each folders for all the clips will save a lot of time. Especially since premiere will automatically arrange them all into each clips with proper framerates. (I use the media browser to import clips)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: skotch on October 09, 2014, 12:01:22 PM
Hi Chmee !

I love your tool and each version has a lot of improvement. I just download the last version but i can't resolve one thing : i'm trying to make video proxies with sound from MLVs.
Using raw2cdng, i had a folder with DNG and WAV. Worked great.
Reading the forum, i see that i can also do video proxies if ffmpeg is present in the same folder. I just did it and mpg2 and mpeg04 did appear in the menu.
But it seems not to work. When i'm extracting the files, i still get the DNG+WAV but no video proxy at all. If i select jpg proxies, it works well.

Do i miss something ? And is it possible to make video proxies this way ?
Thanks !
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 09, 2014, 05:23:22 PM
the video-proxies should work :) please turn on the debug (upper right button) and do a test. inside the debug.log should stay the output of the ffmpeg.exe process. right in the end.

regards chmee

btw : 16bit tif (demosaiced), therefore prores and other codecs on its way, still coding and deciding what demosaicing-algorithm i should take. vng4? dcb? ahd? (atmoment on vng4). @a1ex maybe you a statement?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on October 13, 2014, 12:00:56 AM
chmee, dose the mlvdiag.1.0.1 take a while to analyze files ? Its kept running with not results.
I stop it after 5min.
I have a problem with mlv+audio file+7 spanning , in total 30GB from 5D2
I tried the new raw2cdng(v1.6.1) could not read it & MlRawViewer 1.2.3 would preview the first 24% with metadata
but could not export anything ,  MLVBrowseSharp did export 14 out of 30 GB with audio to Raw legacy with preview turn off
before the program crashed. I can preview that Raw legacy with audio file in raw2cdng (v1.6.1) & MlRawViewer 1.2.3.
So this is what happen to cause this problem. :-[
On a shot, the door of the cam on the CF card was open by mistake after the
cam stop recording but before the cam was turn off the card was not removed thou.
So I turned the cam of remove the card , dump card to laptop and kept on filming with out any more issue .
I also tried mlv_dump by command line no go , so all this leads me to believe that after the 2nd spanning file it
loose its path.
So I was hoping that  mlvdiag could point to a problem I could repair. So far I saved 4700 frames out of 10,000 frames
about 3min out of 8min . Any advice is appreciated thanks  :D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 13, 2014, 12:39:35 AM
it seems, the problem is more on the following split-files. maybe a debugfile from raw2cdng can help on solving.. (pastebin please)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on October 13, 2014, 01:02:02 AM
OK, thanks
I think see the problem
"NO RAWI Block"
http://pastebin.com/embed_iframe.php?i=6B1zPAGs
No sure how to embed in post
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Audionut on October 13, 2014, 02:46:20 AM
No sure how to embed in post

For text you can use the code tags.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 13, 2014, 06:48:31 AM
@reddeercity:
use mlvdiag 1.01 and drag/drop your file straight on the 55mb rectangle on bottom. make it available for us as a download. cause its really strange it does not find a RAWI-Block - tell us the firmware-version, build-date and changeset, if possible.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on October 14, 2014, 12:07:57 AM
@ chmee
Here you go , dropbox link
http://www.dropbox.com/s/cayxrw4m61ncfus/M30-1952.MLV.55mb.mlv?dl=0

I copied the info from the "autoexec.bin" file

Magic Lantern v2.3.NEXT.2014Feb16.5D2212
Mercurial changeset: 536fbadfcc31 (unified) tip
Built on 2014-02-15 23:11:03 by jenkins@magiclantern.fm

Camera  : 5D2
Firmware: 212-e -e

Is this the info you want ?

Can I repair the spanning files , or can I extract the spanning files individually some how ?

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on October 16, 2014, 05:11:05 AM
No suggestions then on how to fix the missing "RAWI-Block" or how to export the spanning file to cdng's ?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 16, 2014, 07:41:54 AM
sorry, hadnt had the time, will look into it today.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on October 19, 2014, 04:33:09 PM
No help ??? I Guess on my own to try to fix this.
Very sad state ML is in when you can't get a little info!
But I guess every one is more concern with
The OTHER camera  >:(
Look like ML is in for a slow death . As no one cares anymore
Here! 
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 19, 2014, 10:48:45 PM
@reddeercity
sorry, it was weekend, there's more than internet. i could import your file into mlvdiag and it shows a rawi-tag. i was able to use it via raw2cdng as well. so. this shortened version works. you said, its a splitted file, but in your debug.log theres counted only one file. so, yes without doubt i believe you theres a problem, but i cant get it. you said, another converters are having problems also?!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on October 20, 2014, 01:46:40 AM
Sorry from before just frustrated.

@reddeercity
 you said, another converters are having problems also?!
Yes , I can extract about 4700 frame to raw legacy with MLVBrowseSharp out of 10,000 frames then crashes (just stops responding & with no preview enabled )
MlRawViewer 1.2.3. will load the spanning 30 GB file but with a still image and stops responding at 25% on the progress bar.

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on October 20, 2014, 02:10:53 AM
Is there a way for checking to see which spanning file is missing the "RAWI Block"
And can a RAWI Block be copy from a different spanning file with change or modified data to work?
I think its at the 2-3 spit file but not sure.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: dmilligan on October 20, 2014, 02:46:45 AM
Spanned files don't have to have a rawi block. There just needs to be at least one somewhere.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on October 20, 2014, 02:52:37 AM
Strange  :o
I Can extract with old version of mlv2dng from August, 28th 2013, with header error audio & missing 804 or 10,320 frames
as After Effect reports , doing a quick render to see what happens.
but at least I have 90% of it in dng's
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on October 20, 2014, 02:57:31 AM
Spanned files don't have to have a rawi block. There just needs to be at least one somewhere.
So would that be a damage header , also is that where the spanning info is ?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: dmilligan on October 20, 2014, 03:08:58 AM
The MLV file format is a "block" based format. There isn't really just a header, there are lots of blocks each with their own headers. There isn't really spanning information anywhere, spanned files have to be auto-detected by the converter. You should try mlv_dump with the -v option (this will list all the blocks).
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on October 20, 2014, 04:23:15 AM
Ok  I'll give that a try
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on October 20, 2014, 11:00:56 PM
I am having issues importing the cDNGs from my 5D3 into Premiere Pro CC.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FORZ22ma.jpg&hash=b8cbe54ee8100618d747bc79b381c9d6)

Here is some info:
raw2cdng 1.6.1 - 12bit max, highlight fix, vertical banding

5D3 1.1.3 ML Oct 7, 14
50mm 1.4
KB 64GB 1000x (exfat)

MLV module
1920x818
Files > 4GB: ON
Buffer fill method: 4
CF-only buffers: 1
Reserve card space: ON
(if not listed, off or default)

Premiere Pro CC 2014.1
Windows 8.1 64bit
i7 990x
16GB RAM
GTX 780
.NET Framework 4.5.1

Debug:
Code: [Select]
------------- 1.6.1 started at 14.10.20 15:04:02 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file C:\Users\Chad\Desktop\Test cDNG\M20-1456.MLV.
[drop] ------
[drop] -- file C:\Users\Chad\Desktop\Test cDNG\M20-1456.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in C:\Users\Chad\Desktop\Test cDNG\M20-1456.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
 ***** [getMLVAttributes] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
 *****
 ***** [getMLVAttributes] metaData Object:
xResolution=1920
yResolution=818
frames=117
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrix=System.Byte[]
lostFrames=0
fpsNom=23976
fpsDen=1000
duration=00:04:01
dropFrame=False
fpsString=23.98
stripByteCount=2748480
stripByteCountReal=2748480
modell=Canon EOS 5D Mark III
camId=MAGIC9bc9385
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0.629766297662977
wb_G=1
wb_B=0.457960644007156
whiteBalance=3600
whiteBalanceMode=9
whiteBalanceGM=3600
whiteBalanceBA=3600
blackLevelOld=2047
blackLevelNew=2047
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5.05952289045009
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.1
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
 *****
 ***** [getMLVAttributes] fileData Object:
fileName=C:\Users\Chad\Desktop\Test cDNG\M20-1456.MLV
fileNameOnly=M20-1456
fileNameShort=M201456K
fileNameNum=201456
tempPath=C:\Users\Chad\AppData\Local\Temp\
sourcePath=C:\Users\Chad\Desktop\Test cDNG
parentSourcePath=Test cDNG
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=10/20/2014 3:03:58 PM
modificationTime=10/20/2014 2:56:46 PM
VIDFBlock=
RAWBlock=
 *****
 ***** [getMLVAttributes] lensData Object:
lens=EF50mm f/1.4 USM:
focalLength=50
aperture=220
isoValue=200
shutter=1/64s
 *****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 394
[drop] hasAudio. AUDF-List created. Blocks: 26
[drop] VIDF-List created. Blocks: 115
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[drop] VIDF-Blockdata read and created.
[drop] adding M20-1456 to batchList
[showPicture] started
[drop] verticalBanding enabled. decision: do it
[drop] verticalBanding CoEffs: 1 0.996742462104726 1.00024283298405 1.00935884271917 0.998710116109811 0.997916433956018 0.99985583655609 0.985100399264895 1
[drop] ** Item M20-1456 imported | MLV
[drop] * res 1920x818px | 23.98fps | 115 frames | BL2047 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 115 | has audio
[batchList_Click] showPicture
[showPicture] started
[batchList_Click] read Data from M20-1456 frame 0
[batchList_Click] * M20-1456
[previewTimer_Tick] show previewframe 1 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 2 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 3 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 4 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 5 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 6 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 7 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 8 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 9 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 10 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 11 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 12 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 13 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 14 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 15 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 16 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 17 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 18 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 19 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 20 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 21 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 22 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 23 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 24 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 25 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 26 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 27 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 28 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 29 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 30 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 31 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 32 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 33 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 34 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 35 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 36 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 37 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 38 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 39 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 40 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 41 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 42 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 43 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 44 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 45 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 46 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 47 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 48 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 49 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 50 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 51 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 52 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 53 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 54 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 55 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 56 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 57 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 58 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 59 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 60 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 61 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 62 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 63 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 64 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 65 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 66 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 67 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 68 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 69 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 70 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 71 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 72 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 73 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 74 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 75 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 76 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 77 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 78 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 79 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 80 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 81 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 82 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 83 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 84 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 85 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 86 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 87 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 88 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 89 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 90 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 91 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 92 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 93 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 94 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 95 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 96 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 97 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 98 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 99 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 100 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 101 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 102 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 103 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 104 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 105 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 106 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 107 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 108 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 109 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 110 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 111 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 112 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 113 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 114 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 0 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 0 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 1 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 2 from M20-1456
[showPicture] started
[batchList_Click] showPicture
[showPicture] started
[batchList_Click] read Data from M20-1456 frame 2
[batchList_Click] * M20-1456
[convert_Click] all in all there are 115 frames to convert
[doWork] started
[doWork] 12 Cores used
[doWork] -> converting item M20-1456
[doWork] settings format 4 with BL0 WL4095 with+ maximizingvalue 5
[doWork] prefix - filename generator Test12m_[D]_[C]
[doWork] destinationPath -> Test12m_141020_201456
[doWork] outputFilename  -> Test12m_141020_201456
[doWork] Directory C:\Users\Chad\Desktop\Test cDNG\Test12m_141020_201456 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork][for] read/convert MLV VIDF: [0]
[doWork][for] read/convert MLV VIDF: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]
[doWork][for] read/convert MLV VIDF: [26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50]
[doWork][for] read/convert MLV VIDF: [51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75]
[doWork][for] read/convert MLV VIDF: [76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99][100]
[doWork] convert Done
[doWork] hasAudio file -> Test12m_141020_201456.wav
[saveAudio] started
 ***** [saveAudio] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
 *****
[saveAudio] bextName: Canon EOS 5D Mark III
[saveAudio] UDI: DECanMAGIC9BC9385145646302567450
[saveAudio] datetime: 2014:10:2014:56:46
[saveAudio] timeRefMultiplier: 1.001001001001
[saveAudio] time2frame: 54292.2922922923
[saveAudio] timeRef (samplingRate*time2frame): 2606030030
[saveAudio] fpsString: 23976/1000
[saveAudio] DF: NDF
[saveAudio] subChunkSize :998400
[saveAudio] success

MLV Download:
https://drive.google.com/file/d/0Bwd3r5HwGFltOUZHT0RJenpvUDg/view?usp=sharing
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 20, 2014, 11:23:23 PM
@chadmuffin
did a test. Test12m_[D]_[C] leads to very long sequencingnumbers (you: Test12m_141020_20145600001)
try with an underline in parenthesis on end. Test12m_[D]_[C](_)

edit: (ah sorry, i tested first with 16bit maximized/highlight fix). but yes, with 12bit it works as well.
[F]_[D]_001_C0000(_) (for example)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on October 21, 2014, 02:57:49 AM
@chee - That did the trick! Working great now.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: reddeercity on October 21, 2014, 05:26:54 AM
I think I found the problem, Open all the spanning files with a hex editor and found that .M02
has "0" data from Offset (6C98CD00) to (FF6EA030)

Code: [Select]
***first part is the beginning of the file***

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000000  4D 4C 56 49 34 00 00 00 76 32 2E 30 00 00 00 00  MLVI4...v2.0....
00000010  8C 9B 9F 10 1A 70 2F AC 03 00 01 00 01 00 00 00  Œ›Ÿ..p/¬........
00000020  01 00 01 00 73 05 00 00 8E 04 00 00 A8 5D 00 00  ....s...Ž...¨]..
00000030  E8 03 00 00 56 49 44 46 E0 D9 2E 00 EF 61 62 0A  è...VIDFàÙ..ïab.
00000040  00 00 00 00 50 10 00 00 A0 00 B0 00 A0 00 AF 00  ....P... .°. .¯.
00000050  E0 0F 00 00 E0 D9 2E 00 52 54 43 49 2C 00 00 00  à...àÙ..RTCI,...
00000060  2E 33 FC 09 00 00 00 00 39 00 36 00 13 00 1E 00  .3ü.....9.6.....
00000070  08 00 72 00 02 00 00 00 00 00 00 00 00 00 00 00  ..r.............
00000080  00 00 00 00 56 49 44 46 90 D9 2E 00 77 60 F9 09  ....VIDF.Ù..w`ù.
00000090  00 00 00 00 AB 0F 00 00 A0 00 B0 00 A0 00 AF 00  ....«... .°. .¯.
000000A0  90 0F 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000B0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000C0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000D0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000000E0  57 42 41 4C 2C 00 00 00 A2 6E 94 03 00 00 00 00  WBAL,...¢n”.....
000000F0  09 00 00 00 E4 0C 00 00 CD 01 00 00 00 04 00 00  ....ä...Í.......
00000100  4E 02 00 00 00 00 00 00 00 00 00 00 56 49 44 46  N...........VIDF
00000110  08 D9 2E 00 89 C6 8E 03 00 00 00 00 96 05 00 00  .Ù..‰ÆŽ.....–...
00000120  A0 00 B0 00 A0 00 AF 00 08 0F 00 00 10 00 08 80   .°. .¯........€
00000130  FF F7 E7 F5 40 58 62 00 FF FB FF FF 00 00 00 40  ÿ÷çõ@Xb.ÿûÿÿ...@
00000140  7B FD FF FE 00 00 00 10 FF FF FF FF 00 00 00 20  {ýÿþ....ÿÿÿÿ...
00000150  FF FF FF BF 00 40 00 42 DF FF FF 7F 00 02 30 00  ÿÿÿ¿.@.Bßÿÿ...0.
00000160  7F DF 7F FF 00 00 20 00 FF EF FF FF 01 20 41 81  .ß.ÿ.. .ÿïÿÿ. A.
00000170  FF EF FF FF 06 12 00 08 DF DF FF FF 80 02 00 02  ÿïÿÿ....ßßÿÿ€...
00000180  7F FF FF FE 00 80 00 04 FF F5 FF F7 00 01 04 00  .ÿÿþ.€..ÿõÿ÷....
00000190  B7 F7 FF FF 80 10 00 00 FF FF 7E FF 00 00 00 00  ·÷ÿÿ€...ÿÿ~ÿ....
000001A0  BF 5A FF FE 00 20 00 00 FF FF FF EF 00 01 00 00  ¿Zÿþ. ..ÿÿÿï....
000001B0  ED FF FD FF C2 08 50 12 FE FF FF FE 02 48 02 10  íÿýÿÂ.P.þÿÿþ.H..
000001C0  FF FF FE FF 01 00 11 80 FF FF BF E7 00 84 40 80  ÿÿþÿ...€ÿÿ¿ç.„@€
000001D0  F7 E5 AF F5 10 00 00 0A FF FF 7F FF 00 10 00 12  ÷å¯õ....ÿÿ.ÿ....

***this part of the file no data***

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

6C98CD00  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CD10  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CD20  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CD30  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CD40  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CD50  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CD60  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CD70  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CD80  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CD90  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CDA0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CDB0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CDC0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CDD0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CDE0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CDF0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
6C98CE00  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................

***end of the file***

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

FF6E9F90  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
FF6E9FA0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
FF6E9FB0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
FF6E9FC0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
FF6E9FD0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
FF6E9FE0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
FF6E9FF0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
FF6EA000  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
FF6EA010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
FF6EA020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
FF6EA030  00 00 00 00                                      ....
Look to me as to be no info there . So the only way I can extract at this point is with mlv2dng with 800 frames missing of 10,320.
In which I have done but can't get the audio thou.
Better then nothing  ::)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 21, 2014, 10:35:10 AM
@reddeercity
because mlv is a block-based format it should work, you delete empty stuff and partly destroyed blocks. then merge part a and b. do not forget to make a backup..
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fs14.directupload.net%2Fimages%2F141021%2F4ye53udn.jpg&hash=9bc11aa036ea50be2fe4680ca26ff1b9) (http://www.directupload.net)

and yes, thats a good hint for mlvdiag, to cope with those errors.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 24, 2014, 10:08:48 PM
short info: doin and testing chroma smooth - still not 100% satisfied - but on the way.. (1.6.2)

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fraw2cdng1.6.2_chromasmooth.jpg&hash=ce97873757583a699a4775a2df9e7d62)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on October 25, 2014, 08:55:17 AM
wow!thats gonna b great! i hv stopped shooting raw after buying 650D as it has so many problems like pink dots n audio which i cant turn off agc. after releasing raw2cdn version 1.6.2, i sould definitely start shooting raw again. raw2cdng is the best converter out there. many people dont know the right converter so they r having problems recording raw but for me recording n editing raw is as easy as  recording n editing h264
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: edge11 on October 27, 2014, 06:14:33 AM
Thanks Chmee for implementing the chromasoothing!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: dougja on October 31, 2014, 01:53:49 PM
What's peoples' opinions on the best workflow using CDNG? Do you run it through Davinci first or drop it straight into PPro CC?

With the latter, my prem pro is struggling to play the files smoothly, it plays about 1-2 seconds before going jumpy/stopping. It's the same with the former method when compressed to MPEG Quicktime and dropped into PPro.

What settings do people use for sequence Preview File Format, if that makes a difference? thanks

16gb ram, i7-4770 @ 3.5ghz, gtx 660, gpu acceleration
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 31, 2014, 01:56:36 PM
what kind of harddisk(s) do you've got? my opinion: i tried to get a straight workflow inside adobe world. so, yes, its possible without DaVinci. i've got for that 2 ssd's (crucial mx100 512gb), they do at least 120mb/s on very small files and go up til 500mb/s (https://twitter.com/phreekz/status/525596432372625408).
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: dougja on October 31, 2014, 02:37:15 PM
Premiere's running off a SSD but the footage is stored on a sata 7200rpm drive, do you think that could be it then?

Although when I play the original MLV file using an mlv player, it plays around 18-19fps, much better than the jumpy premiere pro

thanks
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 31, 2014, 02:50:08 PM
test it yourself and copy some dng's-folders to the ssd. does it change? dng-sequences are not small, as you know. yes, i assume, the sata-hdd limits..
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: dougja on October 31, 2014, 03:04:08 PM
Didn't make a difference, however I tried converting to 12bit instead of 16bit and now it runs smoothly. Am I losing any data by doing this? Sorry if this has been answered before, did have a quick search.

thanks for a great tool by the way!

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on October 31, 2014, 03:12:10 PM
mathematically yes, practically no. the advantages over 8bit are still huge. (and 12bit means as well, lowering the datarate by ~25% compared to 16bit)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on November 03, 2014, 10:54:15 PM
short sign of life.

(*) i embedded the chroma smoothing, as i said. with the first sample i had some problems, because i couldnt eliminate all false-pixels. then i understood, the chroma smoothing is only able to fix red and blue pixels. the green ones blur a little and rise. i'd recommend to use the badpixel-elimination inside the canon body (http://www.bluehoursite.com/articles/how-get-rid-stuckdead-pixels-canon-dslr-camera) before working with ml. the version with cs will be here next days.
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2FM07-2201_CS_analyse.jpg&hash=fc5e5021a53a10573e6cef85ab1f38bd)
M07-2201 from samplefiles (ML-RAW June 2013--EOS 550D @glubber) (http://www.magiclantern.fm/forum/index.php?topic=11899.msg130484#msg130484) - a good point to start with a green hot/stucked pixel algorithm.

(*) in general i'll code a demosaicing-algorithm to be able transcoding into tif, then prores/dnxhd or other codecs (maybe i can realize the arriraw-output)

BUT

(*) my thoughts are, its time to end this tool. it works well and all effort should be put now into the mlvfs-apps. this is a clever timesaving-idea and if something's problematic with cdng i think i can help there to even make it better :)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: swinxx on November 03, 2014, 11:34:51 PM
Hello Chmee, thx for all your effort!
I use mlvfs (as you probably know from the mlvfs thread) i have also some pixel problems with chroma smoothing. I decided to disable it, but perhaps it could be improved to handle the dark areas right. Mlvfs is indeed the easiest and fastest way to get generated dngs.
Greets swinxx
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on November 08, 2014, 03:15:49 PM
thanks to @dmilligan

* cdngs and audiofile behave under resolve as wanted :) autosync as a mediafile now, not only autosync-functional.
* found a 16->12bit bug. fixed.
so, give me some more days and a new version will arrive..

@swinxx
jepp, but i will have marked this tool as "complete" - and im still learning with every bug i find.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on November 21, 2014, 04:16:33 PM
raw2cdng 1.6.3 beta is online

* chroma smoothing
* corrected length of wav-audio, thus it works now in resolve. (autosynced media-bins with according timecode)
* preview now scrubbing instead of playing.

http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on November 22, 2014, 05:33:09 AM
thank q so much for implementing chroma smoothing. but i can test it only next month as im out of my town
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on November 24, 2014, 07:37:59 AM
OMG Thank you for chroma smoothing!

Btw, does anyone getting different K of white balance when using v1.6.3 vs v.1.6.1?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on November 24, 2014, 09:10:21 AM
@feureau
could you show me a screenshot or write some more infos to it? i have rewritten much of the code, its possible i forgot something
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on November 24, 2014, 11:04:02 AM
Screenshot: http://i.imgur.com/PMljcpD.png

Basically, they're the same MLVs loaded on two versions of raw2cdng. The resulting DNGs seem to come out correctly though. And it seems the 1.6.1 didn't preview WB correctly.

Also: I've getting this error:

Code: [Select]
Problem signature:
  Problem Event Name: CLR20r3
  Problem Signature 01: raw2cdng.1.6.3.beta.exe
  Problem Signature 02: 1.5.0.8
  Problem Signature 03: 546f5484
  Problem Signature 04: mscorlib
  Problem Signature 05: 4.0.30319.18444
  Problem Signature 06: 52717edc
  Problem Signature 07: 442a
  Problem Signature 08: 0
  Problem Signature 09: System.IO.IOException
  OS Version: 6.1.7601.2.1.0.256.1
  Locale ID: 1033
  Additional Information 1: 0a9e
  Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
  Additional Information 3: 0a9e
  Additional Information 4: 0a9e372d3b4ad19135b953a78882e789

Also: That chroma smoothing really slow conversions down... But I'm not complaining. :3
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on November 24, 2014, 11:11:09 AM
(*) thanks. assume, i know, where's the bug.
(*) there s an io-exception. are they used/opened in an app in background? it seems, it couldnt erase/rewrite them. fi with premiere?
(*) will look, if i can optimize, but yes, chromasmoothing slows down the conversion massivelly. before that, there only have been bit-shiftings and multiplications of values..

regards chmee
 
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on November 24, 2014, 12:56:10 PM
Can you tell me the difference between 16 bit and 16 bit Maximized?


ps. this is my new Go To tool for conversion.


Thank you so much Chmee!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on November 24, 2014, 01:13:42 PM
as you maybe know, the canon-rawdata has values between blacklevel (fi 2047) and whitelevel (15000). it was necessary to maximize the valuerange for premiere. subtracting the blacklevel and scaling the values to the maximum value (16bit = 65535). i'd say, in general there's no drawback, but for workflow with acr (photoshop/ae) and lightroom i assume, its good to have these sparedata, you can reconstruct more highlights and optimize the noise-recognition.

btw: i wrote this pinkhighlight-fix for premiere, but it seems, its not needed anymore, adobe-coders did something against it. prove me wrong :)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on November 24, 2014, 02:09:04 PM
(*) thanks. assume, i know, where's the bug.
(*) will look, if i can optimize, but yes, chromasmoothing slows down the conversion massivelly. before that, there only have been bit-shiftings and multiplications of values..

regards chmee
 

You're welcome, and thank you!! :D

(*) there s an io-exception. are they used/opened in an app in background? it seems, it couldnt erase/rewrite them. fi with premiere?

I don't have anything else running at the time except regular background stuff like chrome background etc.


as you maybe know, the canon-rawdata has values between blacklevel (fi 2047) and whitelevel (15000). it was necessary to maximize the valuerange for premiere. subtracting the blacklevel and scaling the values to the maximum value (16bit = 65535). i'd say, in general there's no drawback, but for workflow with acr (photoshop/ae) and lightroom i assume, its good to have these sparedata, you can reconstruct more highlights and optimize the noise-recognition.

btw: i wrote this pinkhighlight-fix for premiere, but it seems, its not needed anymore, adobe-coders did something against it. prove me wrong :)

regards chmee

So, to verify: For ACR/lightroom/Resolve - best to convert without Maximized?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on November 24, 2014, 02:27:14 PM
Quote
So, to verify: For ACR/lightroom/Resolve - best to convert without Maximized?
has to be proven as well.

@feureau
sample-pictures regarding wb above. you recorded outside, i'd say ~6000°K, and set the wb manually to 3100°K. i think, the newer picture is colder because its the wrong wb. do a test - dragdrop the file on raw2cdng-icon (not into the opened app), it should open the player. with arrows left/right, P(resets) and O(riginal) you can look on wb-changes. is it maybe fine, if you change it to something like 6000°k?
 
regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on November 25, 2014, 02:38:24 AM
has to be proven as well.

@feureau
sample-pictures regarding wb above. you recorded outside, i'd say ~6000°K, and set the wb manually to 3100°K. i think, the newer picture is colder because its the wrong wb. do a test - dragdrop the file on raw2cdng-icon (not into the opened app), it should open the player. with arrows left/right, P(resets) and O(riginal) you can look on wb-changes. is it maybe fine, if you change it to something like 6000°k?
 

Using 1.6.3 - Looks about neutral with AWB preset at 4500K, filedata is at 3100K Manual WB
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on November 25, 2014, 08:07:13 AM
BTW, I was just converting some DNGs, and found that the latter part of the batch turned up like this:

http://i.imgur.com/gokLxhZ.png

It's not wb issue, there's nothing in the Resolve raw settings I can play with that would fix it. I tried converting the MLV with mlrawviewer and it come out just fine. I tried converting other MLVs with raw2cdng and it turns out this way all the time. I even tried re-downloading the app and it still come out this way. Any ideas?

Furthermore, about half of the DNG that came out right can't be synced with the wav in resolve. Anyone having this issue too?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on November 25, 2014, 08:56:43 AM
@feureau
there's a debug button on top, enable, close, open. it will be written into the same folder as the app. send me a non-functional dng (or mlv) file to raw2cdng@phreekz.de and show me that debug.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on November 28, 2014, 10:47:27 AM
@feureau
thanks. my tool doesnt change black- and whitelevel in this beta. thanks for hint. will be solved next hours. if you know exiftool, you can change blacklevel to 0 and whitelevel to 60.000.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on December 01, 2014, 02:12:00 PM
Thank you, too!

Can't wait to use the fixed version :D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on December 06, 2014, 06:56:49 AM
Hi, chmee. I now kept getting entirely black DNGs. Tested this from 1.5.0b6 up to 1.6.3b. I tried generating proxy, and resulted in this: http://i.imgur.com/sMEAyHv.png Otherwise, the dng is just a black frame.

Also, raw2cdng 1.6.3b seems to crash when I feed it 1 kb MLVs:

https://mega.co.nz/#!P9EklSra!bKh4OjnL8YuEdv4QA78JgPATRRV6d8gDViOqEHcbmYs

https://mega.co.nz/#!KhFFkSYL!0XwzGSNn-AGAaf8iPYoMacie-q_Wain7GZHaihm4P44

They were created when I hit record then quickly stop the recording. (usually pushing the wrong button)

How's that update coming along btw?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 06, 2014, 05:20:21 PM
:) those files are for sure invalid. so i could do a simple request, if a file is at least 1mb. sorry for the delay with next version, i have to work a lot. try it next days.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on December 08, 2014, 08:48:54 AM
I couldn't get one close to 1mb, but anything above that doesn't crash raw2cdng.

Still getting black DNGs though. Is this something to do with .net framework? Does raw2cdng write settings somewhere? It seems to remember certain options when I switch between version. Maybe I need to reset some settings..?

Thanks for this btw. Looking to try the next version in a few days. :D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on December 10, 2014, 08:26:52 PM
Just ran a test with raw2cdng 1.6.3. I was unable to import the cDNGs into AE. I recieved this error.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FnLGkxQp.jpg&hash=f81c0c4e4d3ab34f3b630e0e549873a3)

With 1.6.3 I used 12bit max with vertical banding on and chroma on and off. Normally, I can see the .dng's in Windows Explorer on Windows 8.1 but, they did not load and I was unable to see them in Windows Photo Viewer.

raw2cdng 1.6.1 worked just fine.

Here is the debug from raw2cdng 1.6.3

Code: [Select]
------------- 1.6.3.beta started at 14.12.10 11:45:38 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file C:\Users\Chad\Desktop\M10-1231.MLV.
[drop] ------
[drop] -- file C:\Users\Chad\Desktop\M10-1231.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in C:\Users\Chad\Desktop\M10-1231.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
 ***** [getMLVAttributes] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
 *****
 ***** [getMLVAttributes] metaData Object:
xResolution=1920
yResolution=818
frames=116
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrix=System.Byte[]
lostFrames=0
fpsNom=23976
fpsDen=1000
duration=00:04:00
dropFrame=False
fpsString=23.98
stripByteCount=2748480
stripByteCountReal=2748480
modell=Canon EOS 5D Mark III
camId=MAGICf24bd98
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0.69235970250169
wb_G=1
wb_B=0.410092110532639
whiteBalance=3200
whiteBalanceMode=3
whiteBalanceGM=4900
whiteBalanceBA=4900
blackLevelOld=2047
blackLevelNew=2047
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5.05952289045009
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.3.beta
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
deadSensel=System.Collections.Generic.List`1[raw2cdng_v2.point]
 *****
 ***** [getMLVAttributes] fileData Object:
fileName=C:\Users\Chad\Desktop\M10-1231.MLV
fileNameOnly=M10-1231
fileNameShort=M101231W
fileNameNum=101231
tempPath=C:\Users\Chad\AppData\Local\Temp\
sourcePath=C:\Users\Chad\Desktop
parentSourcePath=Desktop
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=12/10/2014 11:39:48 AM
modificationTime=12/10/2014 12:31:31 PM
VIDFBlock=
RAWBlock=
 *****
 ***** [getMLVAttributes] lensData Object:
lens=24-70mmP4I:
focalLength=70
aperture=0
isoValue=3200
shutter=1/64s
 *****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 393
[drop] hasAudio. AUDF-List created. Blocks: 25
[drop] VIDF-List created. Blocks: 114
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[drop] VIDF-Blockdata read and created.
[drop] adding M10-1231 to batchList
[showPicture] started
[drop] calculating verticalBandingCoeefs and searching hot/dead-pixels..
[drop] NO dead sensels found
[drop] verticalBanding enabled. decision: do it
[drop] verticalBanding CoEffs: 1 1.0008630905897 1.00138402455486 1.01540475086815 1.00000057378494 1.00183129887873 1.00058909931684 0.987164737274471 1
[drop] ** Item M10-1231 imported | MLV
[drop] * res 1920x818px | 23.98fps | 114 frames | BL2047 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 114 | has audio
[convert_Click] all in all there are 114 frames to convert
[doWork] started
[doWork] 12 Cores used
[doWork] -> converting item M10-1231
[doWork] settings format 4 with BL0 WL4095 with+ maximizingvalue 5
[doWork] prefix - filename generator 1.6.3 Riggs
[doWork] destinationPath -> 1.6.3 Riggs
[doWork] outputFilename  -> 1.6.3 Riggs
[doWork] Directory C:\Users\Chad\Desktop\Video Test\1.6.3 Riggs created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork][for] read/convert MLV VIDF: [0]
[doWork][for] read/convert MLV VIDF: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]
[doWork][for] read/convert MLV VIDF: [26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50]
[doWork][for] read/convert MLV VIDF: [51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75]
[doWork][for] read/convert MLV VIDF: [76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99][100]
[doWork] convert Done
[doWork] hasAudio file -> 1.6.3 Riggs.wav
[saveAudio] started
 ***** [saveAudio] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
 *****
[saveAudio] bextName: Canon EOS 5D Mark III
[saveAudio] UDI: DECanMAGICF24BD98123131484946241
[saveAudio] datetime: 2014:12:1012:31:31
[saveAudio] timeRefMultiplier: 1.001001001001
[saveAudio] time2frame: 42030.03003003
[saveAudio] timeRef (samplingRate*time2frame): 2017441441
[saveAudio] fpsString: 23976/1000
[saveAudio] DF: NDF
[saveAudio] subChunkSize :951652
[saveAudio] success

I can send you the MLV if you would like.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 11, 2014, 12:25:02 AM
i'm pretty sure, its about some wrong black/whitelevel. please be patient and use 1.6.1 by now.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on December 11, 2014, 02:48:48 AM
No problem! Thanks for all the effort you have put into the product.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on December 11, 2014, 08:19:32 PM
I was working in 1.6.1 and recieved black frames as well. This was after I used 1.6.3. I deleted all of the files for raw2cdng that I downloaded and redownloaded 1.6.1 and had the same issue. I also noticed all of my settings, such as the filename, were the same even after deleting the file. Does this product save files else where for settings? Should they be deleted to correct this issue? If you need me to run some test, or send you some files, I'll be glad to help.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 12, 2014, 01:33:06 AM
there is the settings-file inside User\[you]\appdata\roaming named raw2cdng.settings.json

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on December 12, 2014, 06:22:26 PM
So that's where that is...

Might be a good idea to keep these things inside the same folder as the main program files. I mean, it's kinda hard to find...
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on December 14, 2014, 07:19:09 PM
That did the trick! Thanks!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on December 15, 2014, 07:58:55 PM
Hey Chmee.

I was converting files and this is the first time I try Vertical Banding fix. Not something usually pay attention to in my footage, but everyone is complaining about it so I thought I better "fix" the issue with my CDNG's.

Problem is that some of the shots now have extreme vertical banding and some dont.. Even though all were exported in one run.

Here is an example: https://www.dropbox.com/s/3ilasxv87den79i/Austria%201_M12-0712_140819_12071200000.dng?dl=0 You can see it especially in shadows. This cdng has VisionLog on it.

Could there be some setting that I am doing wrong or something just went wrong with the conversion?


Ver: 1.6.3 beta

16 bit (not maximized)

Vertical Banding on (Green)

no chroma smoothing

No proxy

No highlight fix



Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 15, 2014, 08:14:42 PM
sometimes the estimation of vertical differences fails, so yes, depending on footage it can deliver worse data. i wouldnt enable vertical banding by default.

[please be still patient solving bugs, i've got birthday today, and the next days i will party as well :)]

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on December 15, 2014, 09:12:55 PM
Happy birthday!! Hope you have a great one :)

 and thanks for the quick reply!

Yeah I will stick to my usual conversion without vertical fix and if I see vertical banding, I will try vertical fix.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: DeafEyeJedi on December 15, 2014, 10:27:45 PM
right on @chmee and Happy Birthday, mister!

Be safe.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on December 17, 2014, 01:40:04 PM
happy birthday chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on December 18, 2014, 08:40:16 AM
Happy Birthday, chmee! :D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on December 18, 2014, 11:08:35 AM
raw2cdng newer versions like 1.6.1 and 1.6.3 has problem while importing xml from premiere pro cc 2014 to resolve lite. older versions are ok. i also tried shootin@50fps and use version 1.6.3 but there was no problem but those which were shot @23.97 fps has problem. resolve keep on crashing while importing xml. i m unable to figure out why this is happenin. older version like 1.4.9.3 works fine
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Ottoga on December 18, 2014, 01:37:00 PM
Happy Birthday Chmee. Hope your party rocks/ed

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 19, 2014, 10:33:45 AM
thanks. (today is the main party, so it didnt ended yet :)) so, i ll address bugs asap. one by one. important is the first one with the wrong black/whitelevel, then i will look for the problems with resolve/premiere roundtripping..

@andy_kh
i think its a problem with dropped-framerates. have to re-check/test my code for none-integer fps. did you tested with/without sound?

cheers chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on December 19, 2014, 03:38:28 PM
yeah i hv done test with sound and without sound @chmee
BTW enjoy your party
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on December 20, 2014, 11:22:22 PM
hi chmee, happy birthday  :) :)... I am using 1.6.1 meanwhile.. but,

1) I get black frames and the tool closes showing "raw2cdng stopped working" even after deleting the .json file in c/user/appdata folder
2) I drop six to seven files (around 1 to 3GB), the tool closes showing the above error. This also occurs when I drop seven files and uncheck five of them to extract only two.
3) The tool shows "converting" even after the files have been extracted to cdng... If I load despite that, the tool closes showing error.

The above all occurs in 1.6.1.... thanks.. :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on December 21, 2014, 01:38:20 AM
Did you check if you are actually missing frames?

Cant help you with the other questions, sorry.


Why not try 1.6.3 ?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on December 21, 2014, 03:29:09 PM
hi kharak, there are no missing frames.
I used 1.6.3 beta after facing above problems in 1.6.1, but 1.6.3 beta also crashes upon loading even two files at a time, each with frames greater than 1000.
since then, I am uploading and extracting single file at a time (total includes 100s and 100s of files for serious video purpose), very laborious :(... but seriously, glad to have this tool :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on December 21, 2014, 08:20:16 PM
CClean and Re-install, probably some registry still interferring from older version. Just a guess, I know this worked for me on Mlrawviewer.

 Good luck!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 21, 2014, 09:43:53 PM
@anandkamal if you show a debug, i/we can assume, on what point it crashes - btw. the crash on closing is known but imho not essential for anything (!)and: my tool doesnt write any data into the registry. someone had installed mlvrawviewer and my tool crashes, ater deleting mlvrawviewer mine worked, so sometimes its only magic :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: allemyr on December 22, 2014, 02:01:12 PM
Hi Chmee! Happy birthday some days late :)

I see that you can't download the 1.6.3 version anymore. I found a bug in the audio from that release, only the first 10% of the audio was there in Davinci Resolve. i use the 1.6.1 for now, and the problem isn't there as it wasn't with previous versions as well.
Thanks again for a great tool. Really love it togheter with Davinci Resolve.

Kind Regards

Karl Allemyr
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on December 25, 2014, 03:24:33 AM
Hi chmee and friends, happy christmas ;)
 
the footage that is shot when a CF card is almost full, popping up a "card full" message, at times crashes the tool. WIll send a debug next time... btw, curious to knw how the update is coming up  :D :D ;D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 25, 2014, 10:51:06 AM
merry christmas. my december is kind of "training day" in party'ing. i did some tests with my footage, it works flawless. i already coded the new colormatrices. because i didnt implemented much exception-trapping/handling, most of these problems while recording will lead to a crash. by now i m still interested in the black dng's.. audiowise, i will put some effort into proper audiolength-handling.

finally i hope, i'll give you a new version this year.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on December 25, 2014, 12:34:07 PM
wow this year!

Looking forward to it!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on December 25, 2014, 12:49:21 PM
eagerly waiting....
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on December 25, 2014, 05:34:05 PM
vow.. u r awesome :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 26, 2014, 01:11:34 AM
put a 1.6.5 on my server (sourcecode in bitbucket refreshed as well)
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

(+) timeRef adjusted for proper sync-handling in resolve (seems to work now)
(+) new color-matrices as stated by andy600 (dmilligan uses them in mlvfs already (https://bitbucket.org/dmilligan/mlvfs/src/ae3c4696191da1a1e4b142780d6b0bee6e64a7df/mlvfs/dng.c?at=master))

but:

(-) i couldnt find any showstopper-problems with my mlv-files. @feureau @anandkamal @andy_kh @chadmuffin please give me some more informations (body, mlv-file) again, maybe write them into bitbucket as issue (https://bitbucket.org/chmee/raw2cdng/issues)

@feureau regarding wb-levels-problem, use mlvdiag1.01 from my site and shorten the file to ~55mb. send it to me. under what circumstances does this happen? (output-properties? bitdepth? proxy? debugfile?)

@chadmuffin not recognised by adobe-apps. yeah, had that as well, seems to be gone now. please test and tell.

did the blackframes went away after deleting settings-file?

@andy_kh i will do more tests now with xml-roundtripping, maybe its the same bug i fixed now (timeRef in audiofile) with two files just tested it worked. no crashes.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on December 26, 2014, 06:34:37 AM
Thank you for 1.6.5 \o/

I'm gonna have to try shoot some stuff for that wb thing and see if it's still / how it's happening. Doesn't seem to be a problem so far though.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 26, 2014, 02:47:49 PM
btw. i did this chart for beginners to clarify some questions.
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fcdng_mlv_raw_ml_bitdepth.gif&hash=7b86f3710affe394c46802ab472bf738)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on December 26, 2014, 03:14:59 PM
thank q for the new version. i wil try once i get back home
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on December 26, 2014, 04:18:02 PM
Thanks for all your hard work! I'll run tests after the holiday season.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on December 26, 2014, 04:34:58 PM
i am not able to drag and drop mlv file in the new version. i also tried the same mlv file in version 1.6.1 but no problem
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 26, 2014, 05:12:10 PM
@andy_kh
i know, windows sometimes disables dragdrop an apps, but works after time. if its not a systemwide problem, try to delete and/or reuse from another place.. or a simple shutdown/restart..

links:
http://www.sevenforums.com/general-discussion/299845-my-drag-drop-function-not-working.html
http://forum.thewindowsclub.com/windows-7-management-support/32562-drag-drop-not-working-windows-7-explorer.html

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on December 26, 2014, 05:28:38 PM
after shutdown/restart i cant stil drag n drop mlv files in the new version 1.6.5 whereas i dont have any problem with version 1.6.3 and 1.6.1
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on December 27, 2014, 01:07:56 AM
You may need to delete some old settings file. On windows: User\[you]\appdata\roaming named raw2cdng.settings.json. It helped me when using 1.6.3 with black frames, might help with you. Also, removed your old raw2cdngs and download the lastest stable one for you. You may need to run it as an administrator in Windows. If you are using a different OS, don't know...
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on December 27, 2014, 01:29:43 AM
Btw, chmee, is that black/white level dng supposed to be fixed in 1.6.5? I'm still getting that green cast. (using CDNG 16bit and vertical banding and chroma smoothing on)

http://i.imgur.com/EwfICfY.jpg

I deleted the .json too btw.

EDIT: For those looking to fix DNGs with similar issues using exiftool and kept failing to get black/whitelevel to change:

BlackLevel is written to SubIFD by default.  Nothing will happen if the DNG doesn't contain a SubIFD.

Use this command to figure out where BlackLevel is stored:

exiftool -G1 -blacklevel image.dng

then write it to the same place:

exiftool -GROUP:BlackLevel=0 image.dng

In my case the GROUP is IFD0The complete command is thusly:

exiftool -IFD0:BlackLevel=0 *.dng

Same with whitelevel
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 27, 2014, 09:38:19 AM
@feureau
please send me a short mlv with that problem. (pleaseplease)
Quote
..I'm still getting that green cast. (using CDNG 16bit and vertical banding and chroma smoothing on)..
maybe thats a hint i have to re-check. seems, its not re-setting bl/wl if helpers are enabled. thx.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on December 28, 2014, 05:42:50 AM
You're welcome :D Looking forward for the fix.

Btw, any plans for hot pixel detection/removal thingy? :P
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on December 28, 2014, 06:38:40 PM
I ran some tests and here are my results:

5D3 with raw2cdng 1.6.5 in Adobe products on Windows 8.1.

I used vertical banding and chroma smoothing 2x2 in raw2cdng (after I deleted settings.json from previous builds). raw2cdng worked great! White balance was from the camera correctly, it correctly knew my 23.976 fps, 5 fps and 48 fps. There were no black frames or green tint. It did not know my aperture from my lens though. Came back as f/0, which could be because I use Tamron lenses? Not hindering the workflow though so not really an issue. I was able to successfully view them in Windows Explorer and the native photo viewer.

After Effects read the files just fine. I had to manually set the fps from 30 fps which seems to be typical. The .wav file started at the correct point and it went a few frames after but, that is easy to just cut that part out with exporting the clip. I used a clapper board over a span of 20 sec and did not see it the waveform off beat or visual see/hear it off beat.

I pulled it into Premiere Pro and brought in the .cdngs in through the media browser. The audio was on time, did not go past the video. However, it did not know my custom fps. It is an easy fix by adjusting the speed in Premiere.

I did not use Resolve.

Everything seems to be working great on my end! If you need more info or tests, just let me know.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on December 28, 2014, 06:39:26 PM
Chmee, I think I narrowed it down a little bit more. It seems if vertical banding is triggered, the white/black level would be exif-clipped. If I select vertical banding and chroma smoothing first, then go change the CDNG bit level, it would not be exif-clipped.

Also: Enabling vertical banding on a 7D clip causes vertical stripes.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on December 28, 2014, 07:47:11 PM
@ChadMuffin
thx for response. good to hear. aperture is just a string. i think its not really worth investigating on it. but if you#ve got a shortened mlv for me i'll try to look into this string.

@feureau
thanks a lot. i'll test this behaviour. as i said before, vertical banding can deliver worse data than before. if you want, send it as well, i'll try to narrow down, if its a coding or a logical problem.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on December 29, 2014, 12:02:01 AM
I did a test with all of my lenses to see if it was specific lenses or all of them. I only have a small handful of lenses to test to include Canon 15mm Fisheye f/2.8, Tamron 24-70mm f/2.8 VC, Canon 50mm f/1.4, Tamron 70-200mm f/2.8 VC. I also have the Tamron SP AF Tele-Converter. The only 2 lenses that had "f/0" were the Canon 15mm and the Tamron 24-70mm. Also, these lenses had typos in their names. "EF15mm f/2.8 Fishee" and "24-70mmP4I" The other lenses were named properly. I also tried the tele-converter on my lenses just to see and the same result happened on the same lenses. The tele-converter did not change the results. I was able to recreate this test 3 times with the same results. Your software seems to be consistent, it appears to be with specific lenses. I have included an MLV of about 3 seconds from each of my lenses and one with the tele-converter for you to check out.


https://drive.google.com/file/d/0Bwd3r5HwGFltQ1dzMUluckU3TXM/view?usp=sharing
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on December 30, 2014, 06:19:56 PM
I'm still testing some footage, especially with the new color matrix. This is the same Adobe Standard color profile as in the Adobe Raw converter thingy, right? Do you have the Canon Standard profile too? Would be awesome to have that and the Portrait profile for awesome skin tones.

A slice of MLV: https://www.dropbox.com/s/bwbm0jbo0qqi992/M03-1438.MLV.55mb.mlv?dl=0
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jose_ugs on January 02, 2015, 02:20:29 PM
Hi all, i though its about time i try raw2cdng and Davinci Resolve. Now this here is off of 1.6.5 and there's something totally wrong in what i'm doing, could you please shoot me :)


Five different exports:
1st (far left) - 12bit maximized
2nd - 16bit with chroma smoothing ON
3rd - 16bit with vertical banding ON
4th - 16bit with highlights fix ON
5th(far right) - 16bit (all fixes are OFF)

Those 5 were imported in Resolve 11.1 with just Color space changed to BMD Film and Highlight recovery On (which didn't do much)... pretty much nothing else!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on January 02, 2015, 02:55:02 PM
not so much wrong :) it seems, if the buttons are clicked in the wrong order, black- and whitelevel are wrong set. i'm off to detroit next 10 days, i'll try to find the main problem there. pardon for this bug and the patience u need now.

(simple fix: delete the raw2cdng.settings-file in /users/[user]/appdata/roaming and then try only with 16bit without nothing else) - ah and look if you've got blacklevel-fix in body en- or disabled.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jose_ugs on January 02, 2015, 07:55:35 PM
hmmm, will try that and report back :) and for the black level in cam, i'm pretty sure it is ON since whenever g3gg0 implemented it, will double check though

P.S. the simple fix did it!!! a thing of beauty :) thanks a ton, and btw i'll be using your tool a lot, so if you need material off a 5dm3, i'll be around
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on January 04, 2015, 10:41:16 AM
ah and look if you've got blacklevel-fix in body en- or disabled.

Btw, I have this turned on in my 7D. Without it the darks will have a weird green cast, especially in high ISO.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jose_ugs on January 04, 2015, 12:51:45 PM
Yeah, this one is a must! I thought it's a setting just for the 5DM3 & M2 ( as well as 50D ), but it's good to know it's also valid for the 7D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on January 06, 2015, 08:38:23 AM
Hi.. before downloading 1.6.5, I had to do a video with 5d3 mlv extracted through earlier version (don't remember the version of raw2cdng). The procedure is premiere-resolve roundtrip. Roundtrip was successful. Every effects and cuts were maintained. Client was happy. Resolve (full), premiere cc 2014.

I now extracted new mlv files from 5d3 through 1.6.5 raw2cdng. I imported files to resolve after setting "master project settings" "assist reel name.." etc.,. I exported dnxhd 36 for proxy (as usual) and in premiere I did offline editing and exported finalcutpro XML for resolve roundtrip. This time I did not get the "FCP translation result.." notepad along with the xml (unlike in previous project). When i imported the XML to resolve 11. Resolve could not link the timeline to media pool, also nothing is seen under "reel name" section in resolve. Roundtrip failed and i had to manually link files.

Am i doing something wrong? any new tutorial on this roundtrip?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on January 07, 2015, 02:27:51 PM
I think that was a resolve-premiere roundtripping issue. Did you uncheck the import material thingy when going back to resolve?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on January 08, 2015, 09:42:54 AM
yes feureau.. i did uncheck the "automatically import..." The reel name is blank when i import the xml from premiere.... Is premeire-resolve roundtrip with mlv's extracted via 1.6.5 working for someone pls?? It worked a few days ago with old mlv files. Can u guide pls?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on January 08, 2015, 10:54:11 AM
i found a strange problem with cdng files. im unable to import xml in resolve  exported from premiere pro. i thought the problem may be with raw2cdng converter so i tried converting in mlvRawViewer  and found same problem. resolve keep crashing. i also test shoting at different framrates. only 23.97 has the problem. shot at other framerate like 25 and 50 works fine. unable to figure out where is the problem. i test some old mlv shot wit 550D and they all work fine again
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on January 08, 2015, 05:56:26 PM
@andy_kh

23,97fps is defined as 24000/1001, whereas ml writes 23976/1000. in earlier versions i readjusted manually broken frames from fi 59940/1000 to 60000/1001. maybe thats a solution.

regards chmee

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on January 08, 2015, 08:33:09 PM
yeah i hope so. thanks for ur reply
regards andy kh
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on January 09, 2015, 01:20:47 PM
"23,97fps is defined as 24000/1001, whereas ml writes 23976/1000. in earlier versions i readjusted manually broken frames from fi 59940/1000 to 60000/1001. maybe thats a solution."

chmee.. sorry i don't understand. By earlier versions, do u mean raw2cdng tool?

How do i do that (manually adjusting broken frames)? thanks nd sorry for not knowing :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on January 09, 2015, 02:41:17 PM
fps are defined as fractions in cdng.
 
25/1 is 25fps, 25000/1000 also..

dropped frames look like

23.976fps = 23976/1000 or 24000/1001 (the first is the value ml saves, the second is the official definition)

i assume you can change them with exiftools (tag 51044 or C764.H), just for a test. its quite the same like changing black- or whitelevels. if this works, leave me an info, i will do back the handling for known broken framerates.


regards chmee

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on January 10, 2015, 05:24:13 AM
hi chmee...

may i know the preferred settings in raw2cdng 1.6.5 for direct cdng editing in premiere pro cc (2014), for DCP/theatrical projection output?

the info is all spread out in posts and a compiled answer could save many beginners like me :) :)

thanks
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jotys on January 11, 2015, 05:55:30 PM
Problem raw2cdng 1.6.5 won't allow me to load any mlv files. I am on windows7 64bit.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on January 11, 2015, 06:28:37 PM
Problem raw2cdng 1.6.5 won't allow me to load any mlv files. I am on windows7 64bit.


same problem with me
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on January 11, 2015, 08:49:07 PM
does it show anything in the debug-file?

@anandkamal
my last tests were with 12bit, 16bit and 16bit maximized WITHOUT highlightfix. Verticalbinding IF needed, Chromasmoothing IF needed. all these cdng's are usable in premiere - but i have to say, i didnt had done short- or musicvideos a long time - so i cant say, what are the optimum settings.

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Walter Schulz on January 11, 2015, 09:04:28 PM
Opened RAW2CDNG and dropped a MLV file.

Code: [Select]
------------- 1.6.5 started at 15.01.11 20:54:16 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file E:\RAW2CDNG\M11-1112.MLV.
[drop] ------
[drop] -- file E:\RAW2CDNG\M11-1112.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] filesize counted (1348MB)
[createMLVBlocklist] indexing Blocks in E:\RAW2CDNG\M11-1112.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
[getMLVAttrib] [!] NO WAVI Block
 ***** [getMLVAttributes] metaData Object:
xResolution=1344
yResolution=756
frames=792
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=
lostFrames=0
fpsNom=23973
fpsDen=1000
duration=00:33:09
dropFrame=False
fpsString=23,97
stripByteCount=1778112
stripByteCountReal=1778112
modell=ERR:1 md:0x       0 ml:0r
camId=MAGIC21edb63
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0,730385164051355
wb_G=1
wb_B=0,376886271623114
whiteBalance=3000
whiteBalanceMode=9
whiteBalanceGM=3000
whiteBalanceBA=3000
blackLevelOld=2047
blackLevelNew=2047
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5,05952289045009
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.5
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
deadSensel=System.Collections.Generic.List`1[raw2cdng_v2.point]
 *****
 ***** [getMLVAttributes] fileData Object:
fileName=E:\RAW2CDNG\M11-1112.MLV
fileNameOnly=M11-1112
fileNameShort=M111112V
fileNameNum=111112
tempPath=C:\Users\<Username>\AppData\Local\Temp\
sourcePath=E:\RAW2CDNG
parentSourcePath=RAW2CDNG
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=11.01.2015 18:34:27
modificationTime=11.01.2015 11:13:10
fileSize=1348
VIDFBlock=
RAWBlock=
 *****
 ***** [getMLVAttributes] lensData Object:
lens=18-125mms9:
focalLength=18
aperture=0
isoValue=200
shutter=1/38s
 *****

Left box looked like this:
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fpicload.org%2Fimage%2Fcccaopw%2Fraw2cdng_err.jpg&hash=8aeff8e441ab1371b8ffb6359b157114)

Grey rectangle is there after drop. Red oval was added by me.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on January 12, 2015, 12:25:19 AM
thanks. so:
Quote
modell=ERR:1 md:0x       0 ml:0r
is the problem. i assumed, the modellname will be written with no hickups (but im wrong). if the modell-name is not accordingly, it cant set colormatrix and forwardmatrix - this leads to another things. so, i will simulate that and fix it in the coming version.

(the same file opens fine in versions before 1.6.5?!)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Walter Schulz on January 12, 2015, 12:31:47 AM
(the same file opens fine in versions before 1.6.5?!)

Tested with 1.6.1: Working fine.

Code: [Select]
------------- 1.6.1 started at 15.01.12 00:37:03 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file E:\RAW2CDNG\M11-1112.MLV.
[drop] ------
[drop] -- file E:\RAW2CDNG\M11-1112.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in E:\RAW2CDNG\M11-1112.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
[getMLVAttrib] [!] NO WAVI Block
 ***** [getMLVAttributes] metaData Object:
xResolution=1344
yResolution=756
frames=792
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrix=System.Byte[]
lostFrames=0
fpsNom=23973
fpsDen=1000
duration=00:33:09
dropFrame=False
fpsString=23,97
stripByteCount=1778112
stripByteCountReal=1778112
modell=ERR:1 md:0x       0 ml:0r
camId=MAGIC5e57b61
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0,730385164051355
wb_G=1
wb_B=0,376886271623114
whiteBalance=3000
whiteBalanceMode=9
whiteBalanceGM=3000
whiteBalanceBA=3000
blackLevelOld=2047
blackLevelNew=2047
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5,05952289045009
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.1
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
 *****
 ***** [getMLVAttributes] fileData Object:
fileName=E:\RAW2CDNG\M11-1112.MLV
fileNameOnly=M11-1112
fileNameShort=M111112V
fileNameNum=111112
tempPath=C:\Users\<username>\AppData\Local\Temp\
sourcePath=E:\RAW2CDNG
parentSourcePath=RAW2CDNG
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=11.01.2015 18:34:27
modificationTime=11.01.2015 11:13:10
VIDFBlock=
RAWBlock=
 *****
 ***** [getMLVAttributes] lensData Object:
lens=18-125mms9:
focalLength=18
aperture=0
isoValue=200
shutter=1/38s
 *****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 1617
[drop] VIDF-List created. Blocks: 792
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[readVIDFBlockData] done: [100][101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124]
[readVIDFBlockData] done: [125][126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143][144][145][146][147][148][149]
[readVIDFBlockData] done: [150][151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174]
[readVIDFBlockData] done: [175][176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199]
[readVIDFBlockData] done: [200][201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224]
[readVIDFBlockData] done: [225][226][227][228][229][230][231][232][233][234][235][236][237][238][239][240][241][242][243][244][245][246][247][248][249]
[readVIDFBlockData] done: [250][251][252][253][254][255][256][257][258][259][260][261][262][263][264][265][266][267][268][269][270][271][272][273][274]
[readVIDFBlockData] done: [275][276][277][278][279][280][281][282][283][284][285][286][287][288][289][290][291][292][293][294][295][296][297][298][299]
[readVIDFBlockData] done: [300][301][302][303][304][305][306][307][308][309][310][311][312][313][314][315][316][317][318][319][320][321][322][323][324]
[readVIDFBlockData] done: [325][326][327][328][329][330][331][332][333][334][335][336][337][338][339][340][341][342][343][344][345][346][347][348][349]
[readVIDFBlockData] done: [350][351][352][353][354][355][356][357][358][359][360][361][362][363][364][365][366][367][368][369][370][371][372][373][374]
[readVIDFBlockData] done: [375][376][377][378][379][380][381][382][383][384][385][386][387][388][389][390][391][392][393][394][395][396][397][398][399]
[readVIDFBlockData] done: [400][401][402][403][404][405][406][407][408][409][410][411][412][413][414][415][416][417][418][419][420][421][422][423][424]
[readVIDFBlockData] done: [425][426][427][428][429][430][431][432][433][434][435][436][437][438][439][440][441][442][443][444][445][446][447][448][449]
[readVIDFBlockData] done: [450][451][452][453][454][455][456][457][458][459][460][461][462][463][464][465][466][467][468][469][470][471][472][473][474]
[readVIDFBlockData] done: [475][476][477][478][479][480][481][482][483][484][485][486][487][488][489][490][491][492][493][494][495][496][497][498][499]
[readVIDFBlockData] done: [500][501][502][503][504][505][506][507][508][509][510][511][512][513][514][515][516][517][518][519][520][521][522][523][524]
[readVIDFBlockData] done: [525][526][527][528][529][530][531][532][533][534][535][536][537][538][539][540][541][542][543][544][545][546][547][548][549]
[readVIDFBlockData] done: [550][551][552][553][554][555][556][557][558][559][560][561][562][563][564][565][566][567][568][569][570][571][572][573][574]
[readVIDFBlockData] done: [575][576][577][578][579][580][581][582][583][584][585][586][587][588][589][590][591][592][593][594][595][596][597][598][599]
[readVIDFBlockData] done: [600][601][602][603][604][605][606][607][608][609][610][611][612][613][614][615][616][617][618][619][620][621][622][623][624]
[readVIDFBlockData] done: [625][626][627][628][629][630][631][632][633][634][635][636][637][638][639][640][641][642][643][644][645][646][647][648][649]
[readVIDFBlockData] done: [650][651][652][653][654][655][656][657][658][659][660][661][662][663][664][665][666][667][668][669][670][671][672][673][674]
[readVIDFBlockData] done: [675][676][677][678][679][680][681][682][683][684][685][686][687][688][689][690][691][692][693][694][695][696][697][698][699]
[readVIDFBlockData] done: [700][701][702][703][704][705][706][707][708][709][710][711][712][713][714][715][716][717][718][719][720][721][722][723][724]
[readVIDFBlockData] done: [725][726][727][728][729][730][731][732][733][734][735][736][737][738][739][740][741][742][743][744][745][746][747][748][749]
[readVIDFBlockData] done: [750][751][752][753][754][755][756][757][758][759][760][761][762][763][764][765][766][767][768][769][770][771][772][773][774]
[drop] VIDF-Blockdata read and created.
[drop] adding M11-1112 to batchList
[showPicture] started
[drop] verticalBanding enabled. decision: not needed
[drop] verticalBanding CoEffs: 1 1,00241055350834 0,998533097079163 0,999124084358047 0,997837972587185 0,999753836807715 0,995905262997038 0,998138322767508 0
[drop] ** Item M11-1112 imported | MLV
[drop] * res 1344x756px | 23,97fps | 792 frames | BL2047 | WL15000
[drop] * modell ERR:1 md:0x       0 ml:0r | vidfblocks 792 | has no audio
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on January 12, 2015, 12:48:44 AM
ok. so i have to do a check against a list of models, if its not in, i have to set to one. (or recheck against the first value of the colormatrix inside the raw/mlv) thats not really perfect, but gives an output. (notice for me: giving an info to the user for those errors/bugs inside the gui).

what camera/ml-firmware do you've got?
i assume, 1.6.5 works ok, if you use a raw instead of a mlv-file :) (for raw-files i use the model-assumption on color-matrix-values)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Walter Schulz on January 12, 2015, 01:04:36 AM
7D and 650D available here. Testing different ML versions is not the problem. Some cards here, too.
File logged was created with 650D and ML version 2015Jan03.

EDIT: RAW seems to work fine.
EDITEDIT: I may revise that ...
Code: [Select]
[drop] * modell Canon EOS 700D | rawblocks 240 | has noaudio
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on January 12, 2015, 01:16:15 AM
hmm.. maybe its only the newest ml-version - i hope. can be a discussion thing for cameraspecific-thread or as general bug in newest ml-fw.

EDIT: yeah, the raw-module recognises the body out of the colormatrix (with raw the only way to define it). as i assumed, the model-writing inside the body seems faulty. from my sourcecode:
Code: [Select]
case 6602: // first value of colormatrix
                    rData.data.metaData.modell = "Canon EOS 700D"; // thats for 650D, 100D, EOSM as well
(bitbucket io.cs line ~770 (https://bitbucket.org/chmee/raw2cdng/src/e62afbaf5fc524f884b813e1f2d77510b38c4b39/csharp2010_code/raw2cdng_v2/io.cs?at=master))
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Walter Schulz on January 12, 2015, 01:26:27 AM
650D, ML build 2014Oct07 and MLV recording: No, same error.
And MLV created with ML version 2014Apr29 won't do, either.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: artyg on January 20, 2015, 10:08:26 PM
Quote
I imported files to resolve after setting "master project settings" "assist reel name.." etc.,. I exported dnxhd 36 for proxy (as usual) and in premiere I did offline editing and exported finalcutpro XML for resolve roundtrip. This time I did not get the "FCP translation result.." notepad along with the xml (unlike in previous project). When i imported the XML to resolve 11. Resolve could not link the timeline to media pool, also nothing is seen under "reel name" section in resolve. Roundtrip failed and i had to manually link files.

Am i doing something wrong? any new tutorial on this roundtrip?

I test new workflow with raw2cdng 1.6.5 and AP CC 2014 (Davinci->AP->Davinci) and all work correct for me. If this topic is of interest to someone I could write a short lesson about it.

But one question for @anandkamal - in Resolve after AP you edit cdng or you render some kind of Log?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on January 20, 2015, 11:18:50 PM
thanks Artyg.. in resolve i just go on with coloring after linking the offline-edited timeline to cdng mediapool. I don't render anything for resolve. But after I grade, I would render dnxhd 220 for final import into AP.

I am currently editing cdngs directly in premiere cc. Even if I get pink highlights or other issues, I don't have to worry as I will import the xml to resolve checking on "automatically import files from media pool." Resolve never shows issues with cdng files (like pink highlights etc.,.). After coloring, I render high quality footage back to premiere for final mastering.

It would be great to have a lesson on the resolve-premiere-resolve-premiere roundtrip, as things are updated now :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: artyg on January 21, 2015, 01:17:53 AM
I did a quick test with cdng (not log renders) and you are right - Resolve crash when I try import xml from AP. Only EDL I was able to import to Resolve. But content not linked. Only edit (cut) positions. Some problem with reel names. I will try tomorrow more detail test. BTW I recomend you to use Cinelog-C to render high quality log source and work with it, not cdng.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on January 21, 2015, 11:03:36 AM
hi artyg..thanks and how do i do that? rendering cinelog-c..?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: artyg on January 21, 2015, 03:30:50 PM
This is is a paid lut bank. Visit official site http://www.cinelogdcp.com/ (http://www.cinelogdcp.com/) and forum thread http://www.magiclantern.fm/forum/index.php?topic=10151.0 about additional info. BTW I will try to record my whole roundtrip Resolve-AP and publish it.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on January 24, 2015, 01:16:37 AM
hi.. there seems to be a problem.. Few days ago, when I imported my .raw files to raw2cdng 1.6.5, it showed me AWB (autowhite balance?) and it set the temperature to 4500 for all .raw files. I carried on with that along with my .mlv files, which got extracted fine (showed the temperature that i shot with).

Now, my .mlv files too show in the tool that the temperature is 4500 AWB.. This was not the temperature i set while shooting.

Code: [Select]
------------- 1.6.5 started at 15.01.24 05:47:40 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file L:\NSamanpdu\Taxi Green\sel14\M26-0441.MLV.
[drop] ------
[drop] -- file L:\NSamanpdu\Taxi Green\sel14\M26-0441.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] filesize counted (1545MB)
[createMLVBlocklist] indexing Blocks in L:\NSamanpdu\Taxi Green\sel14\M26-0441.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
 ***** [getMLVAttributes] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
 *****
 ***** [getMLVAttributes] metaData Object:
xResolution=1920
yResolution=1080
frames=446
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=
lostFrames=0
fpsNom=23976
fpsDen=1000
duration=00:18:08
dropFrame=False
fpsString=23.98
stripByteCount=3628800
stripByteCountReal=3628800
modell=Canon EOS 5D Mark III
camId=MAGICa8ff4d4
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0.540084388185654
wb_G=1
wb_B=0.547008547008547
whiteBalance=4500
whiteBalanceMode=9
whiteBalanceGM=4500
whiteBalanceBA=4500
blackLevelOld=2048
blackLevelNew=2048
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5.05991352686844
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.5
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
deadSensel=System.Collections.Generic.List`1[raw2cdng_v2.point]
 *****
 ***** [getMLVAttributes] fileData Object:
fileName=L:\NSamanpdu\Taxi Green\sel14\M26-0441.MLV
fileNameOnly=M26-0441
fileNameShort=M260441J
fileNameNum=260441
tempPath=C:\Users\Kamali\AppData\Local\Temp\
sourcePath=L:\NSamanpdu\Taxi Green\sel14
parentSourcePath=sel14
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=7/26/2014 3:49:42 PM
modificationTime=7/26/2014 4:41:04 AM
fileSize=1545
VIDFBlock=
RAWBlock=
 *****
 ***** [getMLVAttributes] lensData Object:
lens=EF50mm f/1.8 II:
focalLength=50
aperture=280
isoValue=3200
shutter=1/50s
 *****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 1193
[drop] hasAudio. AUDF-List created. Blocks: 94
[drop] VIDF-List created. Blocks: 443
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[readVIDFBlockData] done: [100][101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124]
[readVIDFBlockData] done: [125][126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143][144][145][146][147][148][149]
[readVIDFBlockData] done: [150][151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174]
[readVIDFBlockData] done: [175][176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199]
[readVIDFBlockData] done: [200][201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224]
[readVIDFBlockData] done: [225][226][227][228][229][230][231][232][233][234][235][236][237][238][239][240][241][242][243][244][245][246][247][248][249]
[readVIDFBlockData] done: [250][251][252][253][254][255][256][257][258][259][260][261][262][263][264][265][266][267][268][269][270][271][272][273][274]
[readVIDFBlockData] done: [275][276][277][278][279][280][281][282][283][284][285][286][287][288][289][290][291][292][293][294][295][296][297][298][299]
[readVIDFBlockData] done: [300][301][302][303][304][305][306][307][308][309][310][311][312][313][314][315][316][317][318][319][320][321][322][323][324]
[readVIDFBlockData] done: [325][326][327][328][329][330][331][332][333][334][335][336][337][338][339][340][341][342][343][344][345][346][347][348][349]
[readVIDFBlockData] done: [350][351][352][353][354][355][356][357][358][359][360][361][362][363][364][365][366][367][368][369][370][371][372][373][374]
[readVIDFBlockData] done: [375][376][377][378][379][380][381][382][383][384][385][386][387][388][389][390][391][392][393][394][395][396][397][398][399]
[readVIDFBlockData] done: [400][401][402][403][404][405][406][407][408][409][410][411][412][413][414][415][416][417][418][419][420][421][422][423][424]
[drop] VIDF-Blockdata read and created.
[drop] adding M26-0441 to batchList
[showPicture] started
[drop] calculating verticalBandingCoeefs and searching hot/dead-pixels..
[drop] NO dead sensels found
[drop] verticalBanding enabled. decision: do it
[drop] verticalBanding CoEffs: 1 0.983723055487067 0.996604155729983 0.989969260901431 0.999345336570097 0.982056042319198 0.990180069471225 0.970404062519835 1
[drop] ** Item M26-0441 imported | MLV
[drop] * res 1920x1080px | 23.98fps | 443 frames | BL2048 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 443 | has audio
[convert_Click] all in all there are 443 frames to convert
[doWork] started
[doWork] 8 Cores used
[doWork] -> converting item M26-0441
[doWork] settings format 0 with BL2048 WL15000 with+out maximizingvalue 5.05983631871526
[doWork] prefix - filename generator [F]_[D]_[T](_)
[doWork] destinationPath -> M26-0441_140726_154942
[doWork] outputFilename  -> M26-0441_140726_154942_
[doWork] Directory F:\CDNGrepairs\M26-0441_140726_154942 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork][for] read/convert MLV VIDF: [0]
[doWork][for] read/convert MLV VIDF: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]
[doWork][for] read/convert MLV VIDF: [26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50]
[doWork][for] read/convert MLV VIDF: [51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75]
[doWork][for] read/convert MLV VIDF: [76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99][100]
[doWork][for] read/convert MLV VIDF: [101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124][125]
[doWork][for] read/convert MLV VIDF: [126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143][144][145][146][147][148][149][150]
[doWork][for] read/convert MLV VIDF: [151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174][175]
[doWork][for] read/convert MLV VIDF: [176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199][200]
[doWork][for] read/convert MLV VIDF: [201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224][225]
[doWork][for] read/convert MLV VIDF: [226][227][228][229][230][231][232][233][234][235][236][237][238][239][240][241][242][243][244][245][246][247][248][249][250]
[doWork][for] read/convert MLV VIDF: [251][252][253][254][255][256][257][258][259][260][261][262][263][264][265][266][267][268][269][270][271][272][273][274][275]
[doWork][for] read/convert MLV VIDF: [276][277][278][279][280][281][282][283][284][285][286][287][288][289][290][291][292][293][294][295][296][297][298][299][300]
[doWork][for] read/convert MLV VIDF: [301][302][303][304][305][306][307][308][309][310][311][312][313][314][315][316][317][318][319][320][321][322][323][324][325]
[doWork][for] read/convert MLV VIDF: [326][327][328][329][330][331][332][333][334][335][336][337][338][339][340][341][342][343][344][345][346][347][348][349][350]
[doWork][for] read/convert MLV VIDF: [351][352][353][354][355][356][357][358][359][360][361][362][363][364][365][366][367][368][369][370][371][372][373][374][375]
[doWork][for] read/convert MLV VIDF: [376][377][378][379][380][381][382][383][384][385][386][387][388][389][390][391][392][393][394][395][396][397][398][399][400]
[doWork][for] read/convert MLV VIDF: [401][402][403][404][405][406][407][408][409][410][411][412][413][414][415][416][417][418][419][420][421][422][423][424][425]
[doWork] convert Done
[doWork] hasAudio file -> M26-0441_140726_154942.wav
[saveAudio] started
 ***** [saveAudio] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
 *****
[saveAudio] bextName: Canon EOS 5D Mark III
[saveAudio] UDI: DECanMAGICA8FF4D4044104285728854
[saveAudio] datetime: 2014:07:2604:41:04
[saveAudio] timeRefMultiplier: 1.001001001001
[saveAudio] time2frame: 16880.8808808809
[saveAudio] timeRef (samplingRate*time2frame): 810282282
[saveAudio] fpsString: 23976/1000
[saveAudio] DF: NDF
[saveAudio] subChunkSize :3698084
[saveAudio] success

other .mlv files in other folders are each opening with some temperatures like 5800, 5000 etc.,. Each folder/shooting location has its own temperature shown for all the files contained in it. Is this how it will be? or am i doing something wrong?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on January 24, 2015, 04:43:41 PM
You are converting to CinemaDNG RAW. I really don't think it matters what white balance you set it to in Camera or how they are converted out, you can always change the WB before you actually start working on footage.

For in camera WB, its nice to set it somewhere close to what you believe is the right WB when shooting, but only for shooting preferences, the beauty of RAW is that you don't have to care about WB, you can set it to 12000K or 2000K, it doesn't affect the final image.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ruber on January 24, 2015, 08:19:14 PM
Hi all,

chmee, thank you for your efforts and results. I'd like to put raw2cdng in my workflow and there is just an element I'd like to understand:

I am using 1.6.5 version and trying to understand how to solve some vertical banding issues: in order to reduce vertical banding, should the "vertical banding" button be green or white?

I am asking because when the button is white I get absolutely better results than when it is green. When the button is green, vertical banding is very strong.

I am also comparing the results with MLRawViewer's output and I tink that banding in this MLRV is almost absent (with banding reduction on).

I'd really like to pass to raw2cdng since I find it more stable and I see you are still developing it.

Thank you. Cheers.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2F&hash=e328456e3a59589f384b3e257867a432)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on January 24, 2015, 10:03:24 PM
would be fine you send me an example of yours to inspect it. (use mlvdiag from my site to extract first 55mb from this example and send it to raw2cdng@phreekz.de) - you re not the first one and it seems its really necessary to change some code ;)

by the way, as every version, please some patience, i'm recoding the file-read-function to get faster transcoding straight from cf-card.. thanks

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ruber on January 25, 2015, 01:31:22 PM
wetransfer sent  ;)

Thank you. Regards.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on January 25, 2015, 01:33:09 PM
:) thx. will do my best.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on January 26, 2015, 01:23:35 PM
ok. here s a 1.6.9.ALPHA DO NOT USE FOR PRODUCTION PURPOSES!
Download 1.6.9.alpha (http://dl.phreekz.de/raw2cdng/raw2cdng.1.6.9.ALPHA.zip)

(*) by now it only can transcode single MLV - not stripped files. (!) no raw(!)

-- Fixes/Patches --
(*) modelname, if not found/wrong in mlv, will be recreated from colormatrix-values @Walter_Schulz
(*) fps-tag - dropped frames are now well-formed (25000/1001 instead of 24975/1000)
(*) massive speed up on transcoding straight from cf-card/hdd

-- cf-card(usb3.0) to ssd - 2GB file (1.6.5 ~3min, 1.6.9 ~30seconds) 6x faster

-- hdd to hdd - 2GB file (1.6.5 ~34.9sec, 1.6.9 ~9.5sec) more than 3x faster
-- same hdd read write (1.6.5 ~32.4sec, 1.6.9 ~10.8 seconds) 3x faster
-- hdd to ssd - 2GB file (1.6.5 ~23.5sec, 1.6.9 ~11.5sec) 2x faster

-- ssd(ntfs) to ssd(exfat) - same 2GB file (1.6.5 ~10.1sec, 1.6.9 ~10.9sec) 8% slower (look beneath!)
-- ssd(exfat) to ssd(ntfs) - same 2GB file (1.6.5 ~6.6sec, 1.6.9 ~6.2sec) 6% faster (look beneath!)

-- same ssd (exFat) read/write (1.6.5 ~12.5sec, 1.6.9 10.8sec) 15% faster
-- same ssd (ntfs) read/write (1.6.5 ~9.5sec, 1.6.9 ~6.9sec) 37% faster

(exFat vs ntfs - cant explain that.. seems, write-rates on exfat here are slow(er) than expected..)

-- todo for 1.7.0
(*) verification of you, if fixes above do well - rewriting for all file-variations.
(*) verticalbanding refinement
(*) some GUI infos (warnings and errors on files that are making problems while dropping)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on January 26, 2015, 02:07:03 PM
wish I had a computer to test your new build.

6x performance.. Wow! how did you pull that off?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: anandkamal on January 26, 2015, 07:39:52 PM
Quote
"You are converting to CinemaDNG RAW. I really don't think it matters what white balance you set it to in Camera or how they are converted out, you can always change the WB before you actually start working on footage.

For in camera WB, its nice to set it somewhere close to what you believe is the right WB when shooting, but only for shooting preferences, the beauty of RAW is that you don't have to care about WB, you can set it to 12000K or 2000K, it doesn't affect the final image."

yes ..thanks kharak
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on January 27, 2015, 03:57:23 AM
so eager to test it bt im away from home
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: ChadMuffin on January 29, 2015, 06:42:08 PM
I did a test for you. 5D3 with 1.2.3.

Appeared to work fine in raw2cdng 1.6.9 Alpha but when I pulled it into AE, error on the 50th frame, as well as in Premiere Pro. I had vertical banding and chroma smoothing on limited to 12bit.

Error in AE when importing
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FF28I0To.jpg&hash=c4efe370268a163308d0c3ccef7be7f2)

The 50th Frame
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FYEjNGnz.jpg&hash=424e26d726c56b89c9a07c355e7a47d5)

Importing into Premiere Pro did not give me an error through the Media Browser but, this is the 50th Frame
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FaNZ5tcm.jpg&hash=88429538375eb9433e32be333dc9b62b)

Here is the debug from raw2cdng 1.6.9 Alpha
Code: [Select]
------------- 1.6.9 started at 15.01.29 11:31:40 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file F:\DCIM\100EOS5D\M26-2005.MLV.
[drop] ------
[drop] -- file F:\DCIM\100EOS5D\M26-2005.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] filesize counted (426MB)
[createMLVBlocklist] indexing Blocks in F:\DCIM\100EOS5D\M26-2005.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
 ***** [getMLVAttributes] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
 *****
 ***** [getMLVAttributes] metaData Object:
xResolution=1920
yResolution=1080
frames=123
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=
lostFrames=0
fpsNom=24000
fpsDen=1001
duration=00:05:08
dropFrame=True
fpsString=23.98
stripByteCount=3628800
stripByteCountReal=3628800
modell=Canon EOS 5D Mark III
camId=MAGIC131b4df
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0.69235970250169
wb_G=1
wb_B=0.410092110532639
whiteBalance=3200
whiteBalanceMode=3
whiteBalanceGM=4800
whiteBalanceBA=4800
blackLevelOld=2048
blackLevelNew=2048
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5.05991352686844
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.9
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
deadSensel=System.Collections.Generic.List`1[raw2cdng_v2.point]
 *****
 ***** [getMLVAttributes] fileData Object:
fileName=F:\DCIM\100EOS5D\M26-2005.MLV
fileNameOnly=M26-2005
fileNameShort=M262005H
fileNameNum=262005
tempPath=C:\Users\Chad\AppData\Local\Temp\
sourcePath=F:\DCIM\100EOS5D
parentSourcePath=100EOS5D
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=1/26/2015 8:04:56 PM
modificationTime=1/26/2015 8:05:04 PM
fileSize=426
VIDFBlock=
RAWBlock=
 *****
 ***** [getMLVAttributes] lensData Object:
lens=24-70mmP4I:
focalLength=24
aperture=0
isoValue=6400
shutter=1/128s
 *****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 447
[drop] hasAudio. AUDF-List created. Blocks: 27
[drop] VIDF-List created. Blocks: 121
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[drop] VIDF-Blockdata read and created.
[drop] adding M26-2005 to batchList
[showPicture] started
[drop] calculating verticalBandingCoeefs and searching hot/dead-pixels..
[drop] NO dead sensels found
[drop] verticalBanding enabled. decision: do it
[drop] verticalBanding CoEffs: 1 0.998021254016585 0.999951270393377 1.01021137184845 0.996759966744552 0.997290595487447 0.994221241876036 0.982397019246451 1
[drop] ** Item M26-2005 imported | MLV
[drop] * res 1920x1080px | 23.98fps | 121 frames | BL2048 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 121 | has audio
[batchList_Click] showPicture
[showPicture] started
[batchList_Click] read Data from M26-2005 frame 0
[batchList_Click] * M26-2005
[previewTimer_Tick] show previewframe 15 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 29 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 26 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 29 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 33 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 25 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 18 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 16 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 18 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 19 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 23 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 25 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 28 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 34 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 41 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 45 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 50 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 53 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 56 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 60 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 66 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 70 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 74 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 76 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 77 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 78 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 79 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 80 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 82 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 84 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 86 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 87 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 90 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 92 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 94 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 96 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 98 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 100 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 101 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 102 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 103 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 104 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 106 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 107 from M26-2005
[showPicture] started
[convert_Click] all in all there are 121 frames to convert
[doWork] started
[doWork] 12 Cores used
[doWork] -> converting item M26-2005
[doWork] settings format 4 with BL0 WL4095 with+ maximizingvalue 5
[doWork] prefix - filename generator [F]
[doWork] destinationPath -> M26-2005
[doWork] outputFilename  -> M26-2005
[doWork] Directory C:\Users\Chad\Desktop\M26-2005 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork] frameconvert Done
[saveAudio] started
 ***** [saveAudio] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
 *****
[saveAudio] bextName: Canon EOS 5D Mark III
[saveAudio] UDI: DECanMAGIC131B4DF200504714205397
[saveAudio] datetime: 2015:01:2620:05:04
[saveAudio] timeRefMultiplier: 1
[saveAudio] time2frame: 72304
[saveAudio] timeRef (samplingRate*time2frame): 3470592000
[saveAudio] fpsString: 24000/1001
[saveAudio] DF: DF
[saveAudio] subChunkSize :1010084
[saveAudio] success
[doWork] audioconvert Done -> M26-2005.wav
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on January 29, 2015, 08:42:49 PM
@ChadMuffin
thanks. havent tested yet, if they re ok inside the apps :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: feureau on January 31, 2015, 09:18:17 AM
Hi, chmee great job on the superfastification! :D They work great.

Crash reports: I have this two 4GB+ MLVs that kept crashing. I tried slicing them down to 55mb to send to you but they don't crash. Size is exactly 4,293,845,600 bytes on both. It's one of them multiple files MLV.

Also, on the thumbnail preview: it seems the highlights are clipped. I exposed it to the right and almost maxed out the histogram and the thumbnails are almost entirely white. (these are videos of the sky with clouds etc).
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: KelvinK on February 03, 2015, 01:12:30 PM
hi chmee

great tool!
didn't find your answer on question - can we get bad/hot pixel fix any time soon? :)


thanx

ps: with 6d dng's and vertical banding on with 1.6.5 i was getting some weird super "oversharp"  on the edges with some (not all) sequences, so decided to turn it off. do you need these files to check what's wrong?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on February 04, 2015, 08:00:21 PM
Converted footage with 165 and 169

Can't say how much.. But 169 is MUCH faster!

First time I enjoy watching MLV being converted ;)

Thanks Chmee.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on February 10, 2015, 01:53:42 PM
@ChadMuffin
it seems, theres simply no frame 50.
(1) is this the only one that lacks? or lacks every 50s?
(2) does it happen on every mlv?

while im rewriting the read-functions i just found more potential pitfalls, where the chronology could be wrong, the offset could change.. so please be patient.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on February 10, 2015, 09:36:58 PM
Had a file I converted with raw2cdng 1.6.9. It had a missing frame at around count 100 and one right before the shot ends.

I thought it was the MLV file but as I saw you mentioning it might be the conversion, I tried converting the same file with mlvmystic and the frames were not missing.

Its not at an interval. The shot is 4000+ frames and its only frame 101 and 4000(something).
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on February 10, 2015, 10:29:03 PM
actually i'm implementing some kind of diagnostics on rightclick, because i ve got as well one file it seems to lose one frame somewhere in the middle..

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jose_ugs on February 16, 2015, 04:40:21 AM
Hey chmee, please, don't leave out spanned MLVs, when(ever) possible :)))
Cool new update though!!!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on February 16, 2015, 08:09:34 AM
its already implemented :) actually i finish the diagnostics and a small bug in raw-spanned files.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on February 16, 2015, 10:29:34 AM
Hi Chmee. What is your solution for handling spanned raw files. Curious.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on February 16, 2015, 12:18:33 PM
spanned files (raw and mlv) are supported a long time. i just found a bug in my new implementation. i dont know, how you mean that question.

(a) mlv is easy. generate list of VIDFs with file-offset, frame-number and spanfile-Number. frames are complete in one file.
(b) raw is quite the same while generating this list. caution: frames can be splitted. thus theres a delta-offset in spanned files.

i hope, today i will throw a new source on bitbucket.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on February 16, 2015, 03:20:03 PM
Yea, I,m on mac so havn,t seen how extracting spanned files is done. Mlrawviewever and another app for mac here. Would you have any clue how to get it done in unix perhaps. Long shot I know but you seem to know your stuff.
WOuld you also know how white balance is recorded to MLV files? I tried setting K(kelvin) on my cam(5d mark 3) But when viewing in exiftool I get Asshotneutral. Sorry if this is a little of topic.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on February 17, 2015, 09:08:27 AM
(a) as i know mlrawviewer exists for linux as well.
(b) colortemperatur/whitebalance isnt recorded in raw, theres a static entry for around 5600°K.
(c) whitebalance-data in mlv are recorded only as presetnumber or -if manual- as kelvin-value, but there arent whitebalance data. so raw2cdng has the ability to reconctruct wb-data from this fields. (see here (https://bitbucket.org/chmee/raw2cdng/src/e62afbaf5fc524f884b813e1f2d77510b38c4b39/csharp2010_code/raw2cdng_v2/dng.cs?at=master))

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on February 17, 2015, 09:38:48 AM
Thanks. Found the mlv metadata, noticed the kelvin and rgb numbers. Having a hard time applying wb multipliers to dcraw to get correct wb from mlv, now i see why.
Impressive work on your windows application by the way.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on February 28, 2015, 02:48:29 PM
raw2cdng 1.7.0 is out - download (http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/) - sourcecode (https://bitbucket.org/chmee/raw2cdng)

* treat it as a beta :)

* massive speed up on read is working on raw and mlv, single and multifile
Quote
(tested with 1.6.9.alpha)
-- cf-card(usb3.0) to ssd - 2GB file (1.6.5 ~3min, 1.6.9 ~30seconds) 6x faster

-- hdd to hdd - 2GB file (1.6.5 ~34.9sec, 1.6.9 ~9.5sec) more than 3x faster
-- same hdd read write (1.6.5 ~32.4sec, 1.6.9 ~10.8 seconds) 3x faster
-- hdd to ssd - 2GB file (1.6.5 ~23.5sec, 1.6.9 ~11.5sec) 2x faster

-- ssd(ntfs) to ssd(exfat) - same 2GB file (1.6.5 ~10.1sec, 1.6.9 ~10.9sec) 8% slower (look beneath!)
-- ssd(exfat) to ssd(ntfs) - same 2GB file (1.6.5 ~6.6sec, 1.6.9 ~6.2sec) 6% faster (look beneath!)

-- same ssd (exFat) read/write (1.6.5 ~12.5sec, 1.6.9 10.8sec) 15% faster
-- same ssd (ntfs) read/write (1.6.5 ~9.5sec, 1.6.9 ~6.9sec) 37% faster

(exFat vs ntfs - cant explain that.. seems, write-rates on exfat here are slow(er) than expected..)
* verticalbanding only worked well with 16bit non maximized - now 12bit and 16bit maximized too.
* modelname, if not found/wrong in mlv, will be recreated from colormatrix-values @Walter_Schulz
* fps-tag - dropped frames are now well-formed (25000/1001 instead of 24975/1000)

* please test and tell

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jose_ugs on February 28, 2015, 11:56:38 PM
Testing it right now and it's as good as it gets!!!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on March 01, 2015, 07:12:32 AM
so eager to test but i wont be able to test until next month since im away from home
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on March 01, 2015, 11:32:05 AM
cool stuff.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: rbrune on March 09, 2015, 09:23:09 AM
I tried converting a mlv from a 500D - looking at your code I would assume a colormatrix for the 500D is missing.

Excerpts from the log:
Code: [Select]
------------- 1.7.0 started at 15.03.09 08:18:57 --
[init][settings] -- ffmpeg Exists: false

[createMLVBlocklist] indexing Blocks in C:\Users\Peter Banane\Desktop\video\janarashtest\M09-0943.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] [!] modelName unknown - estimating from colormtrix
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
[getMLVAttrib] [!] NO WAVI Block
 ***** [getMLVAttributes] metaData Object:
xResolution=1536
yResolution=864
frames=65
bitsperSample=14

colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=

modell=Canon EOS 500D
camId=MAGIC29cc4ec
apiVersion=0

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on March 09, 2015, 10:47:08 AM
thank you for info. you could send me a super short (max 100mb) mlv-file to raw2cdng@phreekz.de
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: rbrune on March 09, 2015, 05:29:55 PM
Mail with a short sample is on the way.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: evanbuzzell on March 18, 2015, 02:05:48 AM
BUG: 1.7.0
The 12th frame of the video being exported drops on 1.7.0. I tried deleting the settings file and it does not work. Reverting back to 1.6.1 corrects the issue and I can confirm there is a non-blank frame in the position. I tried deleting the file again after switching versions and it is still skipping the 12th frame on every different video I export.

Going to try 1.6.5 and see how that works since it seems like many have used it and liked it. The 1.6.1 has just been so rock solid for me!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on March 18, 2015, 09:38:58 AM
ok, thanks. will re-check framenumbering and sort-order of raw-sequences.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: KelvinK on March 18, 2015, 10:36:31 AM
ok, thanks. will re-check framenumbering and sort-order of raw-sequences.

I've same issue with 1.7, frames drop. Was fine with 1.6.5.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on March 18, 2015, 11:56:25 AM
i just had 2 mentions before about that. in 1.7.0 i read and convert in chunks of 50 frames, thus it could be logical it's got some hickups on beginning or on end of that data-chunk, but 12 is quite strange for me. nonetheless, give me some days to analyze and patch it.

as long as you've got those issues, please use older versions you count as stable.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: KelvinK on March 18, 2015, 03:17:10 PM
i just had 2 mentions before about that. in 1.7.0 i read and convert in chunks of 50 frames, thus it could be logical it's got some hickups on beginning or on end of that data-chunk, but 12 is quite strange for me. nonetheless, give me some days to analyze and patch it.

as long as you've got those issues, please use older versions you count as stable.

for me it was quite random was 24 and 65 in different MLVs, didndt try more coz of this bug.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: N/A on March 18, 2015, 10:37:27 PM
Pulled out the old pc so I could try out this app, great work chmee! I definitely like the jpg proxy feature. Now I just need to get Windows XP to read these damn dng files.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 03, 2015, 10:21:58 AM
i hav premiere resolve round triping problem if the footage is shot in 23.97. resolve keep crashing while trying to import xml. everything works fine if i shoot in 25 fps
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 03, 2015, 10:30:58 AM
thanks for info. will investigate. damn dropped frames :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 04, 2015, 03:49:40 PM
v1.7.1 online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* fixed 500D support
* inputfield above the preview. write a frame-no into it, it will show you the data to the VIDF-chunk. (if debug is enabled, it will be saved into the debug-file)
* possibly framelost fixed

* still open:
* framelost fix not guaranteed, therefore this frame analyzer.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: glubber on April 04, 2015, 08:01:05 PM
Hey chmee,

i'll give you some feedback... err i have some problem:

I made a FRSP-Timelapse with an outdated built which recorded 1 MLV File (4 GB) and 99 single picture files (.M00 to .M99), which was state of the art back then.
When i open the mlv in raw2cdng it sees 150 frames in 22 mlv-files resulting in dng-files counting from 0 - 128, 133 - 149 and 249 - 252. (see pictures below)

(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fdl.dropboxusercontent.com%2Fu%2F90483500%2FMAGICLANTERN%2FM99files.jpg&hash=348883e292003c4a35fbd67eeae6181d)

(https://www.magiclantern.fm/forum/proxy.php?request=https%3A%2F%2Fdl.dropboxusercontent.com%2Fu%2F90483500%2FMAGICLANTERN%2FTL-cdngs.jpg&hash=1d7587ecaa2e38909424e08f47eae7fd)

debug.log:
https://dl.dropboxusercontent.com/u/90483500/MAGICLANTERN/raw2cdng.2.debug.log (https://dl.dropboxusercontent.com/u/90483500/MAGICLANTERN/raw2cdng.2.debug.log)

Do you see a chance to get this working?

Thanks in advance for your generous help and keep up the good work. :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 04, 2015, 08:10:43 PM
@Glubber
thanx a lot for this insight. please look with 1.7.1 inside the frame-metadata 128, 129, 130. There should be a jump of framenumbers.. Reason: For MLV i take the framenumbers, that are written into the metadata and i dont renumber them while converting - contrary to RAW where i use an own counting.

this could be the key for "missing" frames.

edit: this 4GB- MLV should have a lot more frames, isnt it? move the files m01-m99 temporarily into another folder and do a second test.
edit2: until m99? fk. i restricted to somewhere m20. patch will come.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on April 04, 2015, 08:39:00 PM
I might add that I tried converting FRSP MLV too, it had one MLV and three m.00 files.


But 1.7.1 crashes almost immediately. Also when only trying with the MLV file alone which contains around 100 pics. 1.6.5 was able to convert only the MLV file once, but if I put all m00 files together it also crashes immediately.

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on April 06, 2015, 10:09:38 AM
stil no luck with round triping
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: glubber on April 06, 2015, 11:11:37 AM
Thank you for your quick response chmee!

I'm afraid i don't understand " look with 1.7.1 inside the frame-metadata 128, 129, 130."

I just get:
[previewTimer_Tick] show previewframe 130 from 23550017
[showPicture] started


Clicking "show meta blocks" results in the log-file i posted.
When looking at the log-file i see no inconsistend numbering outside 0 to 150.

If i test the big mlv-file alone it shows 129 frames. -> 129*31 mb = 3999  ~ 4GB. So the math seem right.

Again many thanks for your efforts!
i really appreciate it.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 06, 2015, 11:28:04 AM
above the preview-window, on the right side, there is an inputfield (filled with 12345?!). write the frames around the "glitches" into this field. test some frames, that seems not to exist.(129-133 or 150-248). but there will be another patch for longer strips until m99, by now i assume, some of these pictures are inside the splitfiles beyond m20.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2Fraw2cdng_171_VIDFdata.jpg&hash=06685cd65d688107253452444241423d)

edit:
Quote
If i test the big mlv-file alone it shows 129 frames. -> 129*31 mb = 3999  ~ 4GB. So the math seem right
yeah. i forgot, its frsp.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: glubber on April 06, 2015, 11:32:00 AM
Hello chmee,

another issue that's puzzling me is the metadata of mlv-files (regular video, not FRSP):

In the raw2cdng app i see all the metadata like shutterspeed, fps, focal-length, iso etc.
But in both Lightroom and exiftoolgui i see nothing except cameramodel, framesize, colormatrix and white- and blacklevel.

Is there no way to write the first mentioned metadata inside the dng or am i missing something?

Thx again
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 06, 2015, 11:33:20 AM
never thought about it, 'cause i assumed its not relevant.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 06, 2015, 04:44:38 PM
chmee, I'm trying to convert quite a large amount of MLV's (626 files, 1.4TB), but the app keeps crashing when I drop the files and the app starts to load them. I enabled the debug, but it sometimes crashes after loading approx 190 files, the next time after 196, etc. Everytime on another file. Is there anything I can do about it?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 06, 2015, 04:49:01 PM
I tried to load less files (approx 180), it loads OK but after I clicked on convert, it crashed:
Code: [Select]
[convert_Click] all in all there are 76216 frames to convert
[doWork] started
[doWork] 12 Cores used
[doWork] -> converting item M04-1024_k01
[doWork] settings format 1 with BL2048 WL15000 with+out maximizingvalue 5,05983631871526
[doWork] prefix - filename generator [F]
[doWork] destinationPath -> M04-1024_k01
[doWork] outputFilename  -> M04-1024_k01
[doWork] Directory F:\RAW VIDEO\2015-04-04 Sevara Bobur\cdng\M04-1024_k01 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 06, 2015, 04:57:54 PM
btw, I tested older releases -- raw2cdng.1.5.0.beta7 loads all files ok, from the 1.6.0 version it starts to crash. maybe it is something related to the thumbnail generator?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: glubber on April 06, 2015, 06:19:11 PM
above the preview-window, on the right side, there is an inputfield (filled with 12345?!). write the frames around the "glitches" into this field. test some frames, that seems not to exist.(129-133 or 150-248). but there will be another patch for longer strips until m99, by now i assume, some of these pictures are inside the splitfiles beyond m20.

Aaah now i understand :)

Ok the results are:

nanual frame    file nbr    frame nbr
    128                  0               128
    129                  5               133 
    130                  6               134

    133                  9               137
    130                 10              138

    147                  2               250
    148                  3               251
    149                  4               252

so indeed it's skipping some files.

About the metadata:
You're right, as a filmmaker you're ought to know your FPS  ;)

I'm still a rookie, so i try to analize my f-stop and shutterspeed when i overexposed etc.
In my case its also helpful for timelapses with exposure-jumps.
Dual-ISO raw video would need the base ISO for deinterlacing.

Thank you once again.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 06, 2015, 06:57:43 PM
@kaco @all
open while using raw2cdng the taskmanager (or the ressourcemanager) and look on ram. i assume it exceds 1,5GB and then it crashes - .net is(was?) not able to allocate more than this. and of course the thumbnails/histogramm-pictures allocate ram as well. this is the reason as well, why FRSP crashes. it needs now significantly more ram per frame - so already coded something to minimize the ability to crash :) will put that online in the german evening :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on April 06, 2015, 08:46:31 PM
Glad I am currently in Austria then ;)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 06, 2015, 10:07:30 PM
v1.7.2 is online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* did some tests with FSRP, worked for me, more checks needed. (if it crashes, try first to convert to 16bit)
* 70D color matrix added
* splitfiles can be more than only 20 (now until .m99 .r99)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: wyrlyn on April 07, 2015, 11:02:31 AM
I still get framedrops, how can I fix that?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 07, 2015, 11:08:38 AM
@chmee, you were right, the last value I saw in task manager was 1556 Mb, then it crashed.
Tried the 1.7.2. -- still not able to load all the files. I loaded less files, allocated approx 1GB of ram, but after I hit convert (16 bit) it crashed.

So for now, I have to stay with the 1.5.x version, since I work with tons of data many times a year.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on April 07, 2015, 12:53:20 PM
Tried same FRSP sequence that didn't work on 1.7.1 and now it works.

1 MLV file and 3 m00.

Haven't come to make any longer timelapses than that at the moment, but I will do some later today and see how it handles more.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 07, 2015, 06:28:19 PM
@kaco

1.7.3 is online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* more ram usage should be possible now.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on April 07, 2015, 06:41:12 PM
You are an updating machine, Chmee.

Downloading.. :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kaco on April 07, 2015, 07:17:42 PM
@chmee, getting better results, but still crashing after filling more than approx 2000 Mb of RAM
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: X-STATE on April 08, 2015, 08:25:45 AM
raw2cdng.1.7.3 and raw2cdng.1.7.2 Not work with RAW from 70D.
Crash then start convert.
raw2cdng.1.6.5 was OK.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 08, 2015, 10:58:56 AM
need a short RAW (<50MB) from 500D and 70D to fix it.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: X-STATE on April 08, 2015, 08:12:49 PM
need a short RAW (<50MB) from 500D and 70D to fix it.

regards chmee
https://yadi.sk/d/_HIfLxdZfrd6Z
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 08, 2015, 08:41:06 PM
v1.7.4 online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* :) the crash came generally on raw - my fault, thanks for the hint. fixed.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jankrueck on April 10, 2015, 11:18:59 AM
Hey chmee,

I was wondering if there is an easy way of adding some grade/LUT to the Program.
As I'm filming most of my stuff on a 5Dmk2 and purchased VisionLOG's Osiris a while ago.

At the Moment I have to batch convert all of those beautiful cDNG's your Programm provides,
in ACR with "VisonLOG" straight to TIF. (while doing this, I loose raw benefits in Resolve)
http://www.vision-color.com/wp-content/uploads/2015/01/VisionLOG-for-Adobe-Camera-RAW.zip

Maybe we can find a better workflow, beside using BMD Film LUT's.
Feel free to add me on Skype, wir sprechen ja die gleiche Sprache ;)
//trashnart

Cheers, Jan

ps: I just realized, that I follow ML raw for like 2 years now, and didn't have a account yet...
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 10, 2015, 11:23:19 AM
idea is not bad :) before that i have to solve the actual bug of leaving out some frames while converting.

@jankrueck
welcome :) geht auch whatsapp? oder threema?

@all
is this problem persistent? if you convert the file more times, are there every time the same frames, that are missing? i have to find out if its an multi-threading error or wrong filenumber-readout.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on April 10, 2015, 02:45:46 PM
Hi Chmee! I, m curious if you, re app is adding specific color matrices for RAW files and if so how do you find out information about what camera has been used? I can, t see any camera related data with exiftool in the extracted dng files?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Ottoga on April 10, 2015, 03:07:04 PM
@chmee

Thanks for your efforts in making your app compatible with FRSP mlv's.  Today I have tested v1.7.4 with single and multi-spanned files.

All processed without error and without dropping any frames. A few things:

Is it possible to change the number of threads used into a user definable quantity or is there an ini file that can be modified? I'm not in a position to upgrade my notebook and it works quite hard with the default 4 threads.

MLV_Dump has the ability to change the frame rate via the -A parameter. Adding this feature to raw2cdng would eliminate the need to do this as a separate workflow step when working with FRSP or any timelapse MLVs. Just a feature suggestion.

Lastly, the following images are the same image and frame using both raw2cdng and MLV_Dump apps. On the image extracted via raw2cdng there is a black border an the left and top of the image. This does not occur in the MLV_Dump extractions.

A close visual inspection of both images suggests that the image is there in its entirety within the raw2cng image. Ideally, it would be better to have the DNG created without the black borders and thereby eliminate the need to crop and possibly stretch each frame in post.

Using raw2cdng
https://www.dropbox.com/s/gdrizkeyqtk7y87/5489000000005.dng?dl=0

Using MLV_Dump
https://www.dropbox.com/s/1eab8emt0c73c3x/5489_000005.dng?dl=0

Just some feedback and thoughts.  Thanks again for your ongoing support and development of this app.

Cheers.... Otto
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 11, 2015, 10:10:13 AM
@Danne
- MLV, according to modelname it writes the colormatrix1/2 and forwardmatrix1/2.
- on RAW its possible to define the body with the colormatrix provided inside RAW - i didnt implemented all bodies, but it works ( search for setModellOnColorMatrix in io.cs (https://bitbucket.org/chmee/raw2cdng/src/ef55e16df51fc66249f2215d0fffe269f0807766/csharp2013_code/raw2cdng_v2/io.cs?at=master))

@Ottoga
(*) threads - ok, will do something :) but there have to be at least 2.
(*) framerate - doable as well - will think about handy GUI-implementation
(*) black border on frsp - ok, not that difficult as well, but have to inspect, if real picturesize is written/implemented in MLV.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on April 11, 2015, 10:58:14 AM
Nice! Will check out. Got matrices implemented in mlv already so this will be really good for raw.
Thanks
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on April 11, 2015, 01:04:48 PM
Thanks! setModell opens up a lot!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jankrueck on April 11, 2015, 02:45:46 PM
hey chmee,
Hab keine Messenger aufm dem Handy^^
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Ottoga on April 12, 2015, 11:09:36 AM
@chmee.... Thanks
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: KelvinK on April 15, 2015, 04:10:18 PM
hi, chmee... tried 1.7.4 - keeps randomly drop frames, usually between 17 and 22 frames (1 frame drops).
same mlv with 1.6.5 works fine
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 16, 2015, 03:22:22 PM
will be back home ~21.04. (actually in shanghai)

@KelvinK
"randomly" means -> same MLV, more converting trials - every time another dropped frames?

@all
and please tell me, if verticalbanding-fix is working accurate in 1.7.4 or not - cause i read in another thread, 1.6.5 was the best until now..

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: KelvinK on April 16, 2015, 04:03:57 PM
chmee, no. randomly i mean: different mlvs drop random frmaes, like mlv1 - its 21 frame, mlv2 - 19, mlv3 has no drops
and since 1.7 conversion with chroma and banding on works slower, even progress bars act different
like if put 1 mlv file in 1.6.5 you can see both bars going simultaneously, in 1.7.4 lower bar goes slow when upper bar passing twice (with one mlv file) till lower (overall)  bar reaches end
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 16, 2015, 04:33:00 PM
@KelvinK
please do me a favor, convert the same "problematic" file more than two times - just to inspect, if these dropped frames vary.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: KelvinK on April 17, 2015, 08:50:59 AM
Ok, chmee, will do it. Actaully, I can upload (but next week only) some problematic MLVs for you, if you need to inspect it.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 18, 2015, 04:03:48 PM
:) yes please.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on April 23, 2015, 01:57:50 PM
Does anyone here have any experience with CDNG in Speedgrade?

 CinemaDNG has a strong green cast in Speedgrade. This might have something to do with blacklevel, but I am not sure.. Looks a little like the ffmpg player when it importing ML Raw. And what I read in some threads on adobe, it is a known thing.. And I read that I should "counter colour grade the green".. Sounds very unintuitive to me.. But I have tried, making LogPrimaries and what not.. Basically just trying out different keywords that I read about here and there. But I can't get it to Normally normal.. Not at all as ACR does.

Would be nice if someone had a good pointer to getting the CDNG looking normal in Speedgrade, I would really like to try out some CDNG grading in Speedgrade.

The CDNG's I used were 12 bit maximized. MK III.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Andy600 on April 23, 2015, 02:03:44 PM
@Kharak - Try adding the inverse of the D65 5D3 ColorMatrix1 on a daylight shot then white balance it. This should sort out the green problem (it will require different matrices for different color temps). I'm working on a Speedgrade solution.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on April 23, 2015, 02:29:17 PM
Ok thank you Andy, I will try that.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 23, 2015, 03:32:42 PM
many entries before (nov2013 (http://www.magiclantern.fm/forum/index.php?topic=5618.msg89880#msg89880)) i talked a bit about speedgrade and handling colormatrices..

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on April 23, 2015, 06:43:51 PM
@Chmee.

I copied the code and made a cube file.

But frankly I don't know how to import luts into Speedgrade.


Tried your way too Andy, but I can't find the D65 inverse of the 5D MK III etc.. or what I am actually supposed to do.

I added fxInversion, but I don't think this is what you mean.. Mind you, I am a newcomer to Speedgrade, this is my 5th attempt at this software. I don't know what has changed in CC compared to CS6 but I do like it better now. But I still find it troublesome to navigate..

Need to find some tutorial to get me through the essentials.

So far I got the best results adding Gamma 2.2.ilut to the footage, still has a green cast then but the luminance seems to be at a normal level, but yeah, it ain't good.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Andy600 on April 23, 2015, 07:43:17 PM
@Kharak - I'm not a daily SpeedGrade user (actually never colored with it) so I think I over simplified things earlier regarding the matrix inversion. This might be part of a solution but not the full solution. I'll work out how to get MLV to display color and gamma correctly in SpeedGrade and build a profile for the 5D3. Fortunately, the Iridas format makes it easy to get technical :)

For reference, if you want to invert a color matrix you will need to find the D65 colormatrix in one of your DNG files using raw digger or something else that can read metadata. Then invert it - a good online matrix tool is http://www.bluebit.gr/matrix-calculator/ (http://www.bluebit.gr/matrix-calculator/)

The 5D Mark III D65 color matrix ML uses is:

Code: [Select]
0.672200 -0.063500 -0.096300
-0.428700  1.246000  0.202800
-0.090800  0.216200  0.566800


so the inverse of this would be:

Code: [Select]
1.543930  0.035363  0.249663
 0.523447  0.867682 -0.221521
 0.047670 -0.325303  1.888783

If my understanding of SpeedGrade is correct, this would give you the 5D3's linear raw color but would still require white balance and a color correction matrix + gamma. It needs a proper calibration profile - I'm working on a test profile for 5D3 MLV in SG and will make the rec709 version available for everyone.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 23, 2015, 09:31:52 PM
here's a picture from the past, where you see the lut used inside sg - you have to put this file into a specific folder. wait.

C:\Program Files\Adobe\Adobe SpeedGrade CC 2014\settings\looks\CinemaDNG

ah, and they're named -> [name].look

try the values from andy600 - my values were just tests long ago..

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fdl.phreekz.de%2Fraw2cdng%2F5DIII_Greencast_Example.jpg&hash=0bf73f5170410aa164e334e7caa96ed2)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on April 23, 2015, 10:25:46 PM
renamed it to look and it sees it, but it says "invalid preset" before the name.

Maybe it has something to do with under format defaults tab (On your screenshot, that would be FormatStandards), I chose Alexa at one point, it was standard on CinemaDNG, I could choose Alexa and R3D. But now I can't choose CinemaDNG again, I can choose R3D or Alexa and it is standard on Alexa.

Same under Clip and Format Options..

Doesn't matter what footage I import, it keeps saying Alexa now.

Not sure if I wanna continue with Speedgrade at this point... Its not fun anymore :) hehe
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 24, 2015, 08:08:27 AM
@Kharak
in my old Versions 1.4.0 and 1.4.8 there are .look files (http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/) as well. try these.

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on April 24, 2015, 09:56:48 AM
Thanks, these .look files work.

With the look file from 1.4.8. The colours looked quite normal, but the image gets heavily overexposed.

Adjusting exposure in Colour Space Defaults and Blue channel gets it close to the way Windows shows my DNG's.

But its not quite normal to grade, to me it seems.. Perhaps it's the program that is that way, but seems like highlights and shadows act up very strange.. Their levels also seem to be slightly off all the time.

Definitely not the same as grading with After Effects.

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 24, 2015, 10:35:43 AM
@Kharak
@DFM wrote some good points about sg and the rest (2013) (http://www.magiclantern.fm/forum/index.php?topic=6021.msg76056#msg76056). Do not forget, these .look-files are from 2013, first trials to get working inside speedgrade, yes there's a lot of space to optimize them.

got a pm by @DAP , will discuss it here
Quote
..just wondering what happened to the highlights fix button in 1.7.4  I am still getting pink highlights in premiere 8 cc with 16bit maximized output from raw2cdng via 5D3
i freezed the highlightfix, because it changes the rawdata, and i want to change less as possible on the sensordata - there's a fix inside of premiere (https://forums.adobe.com/thread/1500841). this problem affects other cameras/recorder as well.

regards chmee
Title: wrong topic
Post by: Lysander on April 26, 2015, 10:02:01 PM
sorry, wrong topic
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: extremelypoorfilmaker on April 26, 2015, 10:58:17 PM
Hey, i just tried to use today the raw2cdng app, by the way, thanks chmee!! :)

As i click on it i receive this error message saying that i need to install one of the following version of .NET frameworks v4.0.30319, i checked on my PC and the .NET frameworks version i have got is 4.0.4etc.

Anyone having this problem? I am running windows 7 64bit on my machine. thanks in advance

Installing Eos utilty 3 solved this problem :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 27, 2015, 01:10:09 PM
:) It wont be about eos-utility, but maybe about .net 4.5.
on win8-systems this problem shouldnt occur, because both (4.0 and 4.5) are installed already.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: extremelypoorfilmaker on April 27, 2015, 06:14:41 PM
correct Chmee! I noticed the installer of EOS Utility 3 installing the .NET 4.5 files. now i got your splendid program working almost fine on my PC :)

I am saying Almost because something quite curious happened. basically, i tried your program yesterday night converting some footage i already converted with Magic Lantern RAW video converter 1.9.1

when i converted some shots using raw2cdng version 1.7.4 and 1.6.5 the resulting footage was all Green. The same footage, converted with Magic Lantern RAW Video converter 1.9.1 was looking fine.   
The first time i converted the files, i only selected the option cdng 16bit.
i set the folder where to export them, but instead of exporting the footage in the folder i created, it exported them in the same folder as where raw2cdng was.

I just exported the same footage using the option 16bit maximized and the footage is looking fine.

Probably you already know about this "issue" (if i can call it a real issue) but i thought i would let you know. :)

Thanks again! Will donate soon! ;)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jankrueck on April 29, 2015, 12:51:05 PM
So Chmee,
I'm back home and ready to talk about some "Lut-in-baking" on cDNG ;)
Still no Skype? (or was it just because of you, not beeing home)

Cheers, Jan

ps: if no skype, what about fb?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: theartofweb on April 29, 2015, 05:14:56 PM
I would like the "start TC" to be always 00:00:00:00 instead of the date of the day (that's what I see on Resolve).

I would love the folder to be named M22-1852_DNG_000001 instead of M22-1852 and the files inside something like M22-1852_000001 instead of M22-1852000001

(first 2 clips generated by raw2cdng, last 2 clips generated by MlRawViewer)

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2F41.media.tumblr.com%2F4730c25d7f91c55697bdcf2851ea0b60%2Ftumblr_nnkpwma0K81s5o348o1_1280.jpg&hash=9dfdec5dd6fee391393eb670c4a09e6e)

After all, I like raw2cndg for its batchlist properties!

Thanks
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 29, 2015, 06:34:05 PM
tc to 0 is something i could implement, but i dont see any benefit on it.. does it really matter for you? second question is solveable with the filenamegenerator, try [F]_DNG_00001(_)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: theartofweb on April 29, 2015, 07:01:10 PM
Come to think of it I can set TC to 00:00:00:00 on Resolve. Everything is ok now.

Thanks
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on April 30, 2015, 11:15:44 AM
Short Info. With ".net Core" raises the usage of this and other .net-tools to work under mac and linux.

Link [german] - http://t3n.de/news/net-core-fuer-linux-mac-os-607819/

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: KelvinK on April 30, 2015, 03:25:39 PM
@KelvinK
please do me a favor, convert the same "problematic" file more than two times - just to inspect, if these dropped frames vary.

regards chmee

sorry for delay, was away. tried convert problematic file 3-4 times, dropping same frame.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jackie 4CHAN on April 30, 2015, 09:20:19 PM
Hey Chmee,

First, thank you for everything you've done. This program works great at converting my .MLV files. However, I got a question about chroma smoothing. What exactly does it do?

The reason I ask is because I had it on when converting some footage shot on a 5dm3. When I import into AE, some of the highlights are pink. Even with using VISIONlog. I could not for the life of me figure out how to get rid of the pink without damaging the image.

After a day of searching for information on both the Adobe and ML forums, I decided to convert my .MLV files again, this time without chroma smoothing on. Whaaalaa! All of the footage is properly displaying. No more pink highlights.

So what's going on? Can you put it in layman's terms on what's happening? I read somewhere that the pink highlights are from the wrong black levels? But those threads were years ago, not sure what's the most up-to-date info.

I'm using RAW2CDNG 1.7.4
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jankrueck on May 14, 2015, 07:23:57 PM
hey chmee,

I just did some raw-shooting. (mlv with sound turned off to be correct)
and when starting to convert, I realized there is some kind of brigtness-shift between 12bit(maximized) and 16bit(maximized)
on raw2cDNG 1.7.4.

Is this intended?

12Bit(max)
https://dl.dropboxusercontent.com/u/26597584/12bit(max)_M14-1620_00000.dng

16Bit(max)
https://dl.dropboxusercontent.com/u/26597584/16bit(max)_M14-1620_00000.dng

16Bit
https://dl.dropboxusercontent.com/u/26597584/16Bit_M14-1620_00000.dng



Cheerio, Jan
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 16, 2015, 09:53:31 AM
@all
sorry for doing nothing atm - i'm really busy.

@jankrueck
will check that. patience please.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jankrueck on May 16, 2015, 11:18:27 AM
actualy I did some new comparison between your conversion
in different modes, mlRawViewer and mlv_mystic.
Because I trie to find the mlv -> cDNG conversion wich "does" quite nothing to my material.
dunno if this is stupid, but I want completly untouched files when I start to edit.

http://i.imgur.com/x6JccSg.jpg
(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fi.imgur.com%2Fx6JccSg.jpg&hash=db0ed04b50323fdb463b2dd9156c9136)
I'm just wondering how different colors can exist.
Maybe I understand that whole mlv thing wrong,
but in my understanding it is just a container in wich some dng's are waiting to be let free :D


dng files will be here for download in some minutes.
https://dl.dropboxusercontent.com/u/26597584/compare.zip


Cheers,
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: theartofweb on May 16, 2015, 01:12:44 PM
Why not just regular 16-bit no smoothing? It works great.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: glubber on May 16, 2015, 03:34:13 PM
@jankrueck:
In the end everything you see on your monitor is an interpretation of the sensor-data depending on your software (debayering).

I recommend reading the cinelog thread of Andy600. He gives a good insight of colorspace and the whole sience behind it.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on May 16, 2015, 04:43:59 PM
It,s a pretty interesting comparison. How is mlrawviewer lossless compression working in these examples? Check the sky next to the flower.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jankrueck on May 16, 2015, 08:34:30 PM
It,s a pretty interesting comparison. How is mlrawviewer lossless compression working in these examples? Check the sky next to the flower.
actualy there is no sky, its blue paper.

What you mean by lostless? dng export is always lostless in my point of view.
The Bottom pictures are all mlrawviewer. Just copy pasted them for comparison.

@jankrueck:
In the end everything you see on your monitor is an interpretation of the sensor-data depending on your software (debayering).

I recommend reading the cinelog thread of Andy600. He gives a good insight of colorspace and the whole sience behind it.

I know how this works, but when you debayer dng's, in my opinion they should be all the same before I start grading.
Also cinelog is not embed into rawfiles. Its just kind of a LUT for makeing h264 files more flat to make grading easier.
This should only affect proxy files, like mov clips.

So if there is no colorspace changes while debayering, all pictures should look the same.
This comparison is a study about the debayering process itself, to ensure there is no lost or adding of information.

Why not just regular 16-bit no smoothing? It works great.
Why no smooting? Because reducing noise on this level improves imagequality before adding Luts.

Aaaand like you see there is no difference between smooth files and not smoothed.
16bit (not maximised) is kinda dark, with smoothing and without.


----------------


mlrawviewer looks pretty much the same as I remember the scene looked like. (color & brightness)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on May 17, 2015, 01:51:26 PM
Mlrawviewer compresses the dng files. Does it show? Said to be lossless. Could you upload the sample files?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: dmilligan on May 17, 2015, 03:30:15 PM
So if there is no colorspace changes while debayering, all pictures should look the same.
This comparison is a study about the debayering process itself, to ensure there is no lost or adding of information.
There is always a colorspace change during debayering, b/c raw has no colorspace and you must convert it to one to even be able to view the file. There is also always "lost and added" information during debayering. Debayering itself is a type of interpolation. You cannot view raw data directly, it must be synthesized to some real colorspace before you can even preview it.

There is a lot of metadata in a DNG file that tells the reader/debayer software how to read/interpret the raw file and how to convert it from the native raw "colorspace" to something standard like XYZ (this metadata can effect color as well as brightness and any manner of other image aesthetics). Also various software vary in how they interpret this data and of course debayering algorithms vary in speed/quality/artifacts/etc. Various tools that produce DNGs from MLV,RAW  may vary in the particular metadata they put in the DNG file, which may affect the initial "look" of the DNG file when first opened, even if the raw data is the same. When you "edit", "grade", and "develop" the DNG file, you are likely going to be changing some of these parameters themselves, and your image is re-debayered, so even if images start out looking slightly different, you are modifying the parameters that made them look different and there will be no difference in the final output.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: jankrueck on May 17, 2015, 05:53:45 PM
Mlrawviewer compresses the dng files. Does it show? Said to be lossless. Could you upload the sample files?
Samplefiles are downloadable in post above



@dmilligan: yeah sure there is some change or lets call it "import" into  a xyz-colorspace.
it seems there is some misinterpretation or at least some problem with the metadata embed into the 16bit dng's.
Or lets not call it problem, but if there is a brightness-shift this strong just on 16bit. It seems to be wrong.

Just wanted to tell chmee what I've found ;)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Andy600 on May 17, 2015, 06:08:18 PM
@dmilligan - I couldn't have put it better myself!


To add to that, I have long suspected that the DCRaw matrices, and for that matter, all the Adobe DNG converter matrices that I recently gleaned are really only useful for Adobe apps and more specifically, stills/print processing - the reason being is that Adobe camera calibration allows out-of-gamut color in the color matrix tags which they have to 'manipulate' with forward matrices and HSV tables to get prettier or more 'accurate' and gamut safe color reproduction. FYI, the Forward matrix tags are also important to any .ICC dependent NLE or color grading app but they are not part of the CinemaDNG standard i.e. so we can't rely on them for any gamut 'correction' outside of Adobe apps.

Some of the work we (Cinelog) are doing for forum member @janoschsimon to calibrate his DIY MiniRaw camera, plus several other custom/scientific camera projects we worked on over the last year has highlighted to me that we can do much better to conform Magic Lantern Raw Video color to fit better in relation to the big raw shooting cinema cameras - afterall, Canon sensors have great color sensitivity so let's make the most of it.

I'm planning a 'Magic Lantern Raw Color' project and will publish details of what I want to do, how I will do it and how ML users can help sometime in the next few days. Hopefully a new set of ML color matrices I plan to develop will not only provide better default color that renders identically across all apps but, if/when accepted into ML source code, will allow me some confidence in developing proper ACES IDTs for each camera - at present, it's a crap shoot and unworkable with the single DCRaw matrix and differences between commercial and non-commercial apps. There is little point basing anything on the Adobe matrices for the reasons stated above. The resulting data will be published for any app dev to use under ML licensing!

To avoid any confusion - the DCRaw matrices (which actually come from Adobe labs) and Adobe DNG matrices are not 'bad' or 'wrong', they are simply intended for another purpose - they work fine for most things but IMO, it's time for Magic Lantern Raw Video Color to becomes it's own 'thing' without reliance on incomplete or borrowed data that 'might' have legal implications.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: DeafEyeJedi on May 17, 2015, 06:51:32 PM
@Andy600 -- great read once again ( along with @dmilligan's from few posts up  ;) )

Looking forward to your much anticipated 'Magic Lantern Raw Color' Project and would be more than happy to help contribute and get these colors up to speed!

Keep up the excellent work and I appreciate your hustle on this -- it has definitely paid off on my part in getting used to the workflows around Adobe apps especially in AE (still can't resist from using ACR) so I must say THANK YOU for getting me on board, keeping me eager enough to stick through the bumps on the road and along with your patience I was able to absorb & learn to the fullest without giving up and still yet I'm learning!

 8)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on May 17, 2015, 07:16:47 PM
Quote
I have long suspected that the DCRaw matrices, and for that matter, all the Adobe DNG converter matrices that I recently gleaned are really only useful for Adobe apps


Thanks for calrification. I,ve been comparing these matrices with dcraw numbers and peeping dng files and came to the same conclusion recently.

Quote
I'm planning a 'Magic Lantern Raw Color' project and will publish details of what I want to do, how I will do it and how ML users can help sometime in the next few days.

How cool is that. Let me know if I can help.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 18, 2015, 01:16:14 AM
@jankrueck
let raw2cdng write the according whitebalance-data into the dng's.
(*) i assume all but AWB will work.
(*) in the beginning there was the way to shot a CR2 and name it like the raw/mlv-file. raw2cdng takes the whitebalance-data from this cr2 and put it into the dng. i didnt tested it the last versions :)

but finally all was said - the raw-data are nearly untouched - in simple 16 bit the least. its up to you and your app to get out most of it. the metadata inside dng are the slightest problem..

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Licaon_Kter on May 23, 2015, 12:44:42 AM
Heads-up,  latest Lightworks 12.5.B beta imports cinemaDNGs and LUTs even in the free version:  http://www.lwks.com/index.php?option=com_lwks&view=changelog&Itemid=207&id=194 (http://www.lwks.com/index.php?option=com_lwks&view=changelog&Itemid=207&id=194)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on May 23, 2015, 06:31:28 AM
Quote
in the beginning there was the way to shot a CR2 and name it like the raw/mlv-file. raw2cdng takes the whitebalance-data from this cr2 and put it into the dng. i didnt tested it the last versions

Hi Chmee! Can I ask you what numbers you were using and applying to the dng files coming from CR2 files? I tried some with no success.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 23, 2015, 10:52:04 AM
exif MeasuredRGGB - calc.getRGGBValues() (line 1287) and calc.convertToFraction() (https://bitbucket.org/chmee/raw2cdng/src/ef55e16df51fc66249f2215d0fffe269f0807766/csharp2013_code/raw2cdng_v2/calc.cs?at=master)
used in io.cs getMLVAttributes line 611 and getRAWAttributes() line 767 (https://bitbucket.org/chmee/raw2cdng/src/ef55e16df51fc66249f2215d0fffe269f0807766/csharp2013_code/raw2cdng_v2/io.cs?at=master)

old discussion with informations (http://www.magiclantern.fm/forum/index.php?topic=6779.0)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on May 23, 2015, 11:22:54 AM
Thank. I keep circling. Will take a dive :).
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on May 23, 2015, 08:20:05 PM
So fiddled with dcraw. When writing
Code: [Select]
dcraw -i -v CR2 it puts out the camera multiplier from the CR2 file. It seems it is almost exact to AsShotNeutral. For instance.

Numbers from CR2 via dcraw
Daylight multipliers: 2.125175 0.943985 1.338680
Camera multipliers: 1722.000000 1024.000000 2032.000000 1024.000000


Dividing by 1024/1722=0.59465737514518  1024/2032=0.50393700787402

I put in following numbers with exiftool
AsShotNeutral=0.59465737514518 1 0.50393700787402

It gives a dng with settings in ACR Temperature 3900 Tint +7
CR2 gives Temperature 4050 Tint +8

Is it possible to get an exact measuring? It is close. I try to follow your links but I am lost ;).




Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on May 24, 2015, 01:15:47 PM
so far this was the simplest way to get "good" (not perfect) body-data. dont forget, you can't count on body-measured data as well, it can differ from cutters opinion of "good" colors.

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on May 24, 2015, 02:21:04 PM
Thanks. Finally some numbers I could use regarding wb in dng files :).
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Ali Oliya on May 28, 2015, 07:43:11 AM
Hi

(My apologizes if this question is already asked?)  I'm using raw2cdng ver 1.7.4 and it always drops 1 frame in every .mlv file conversion. so there is always 1 dng file missed! why is that?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Ali Oliya on May 28, 2015, 09:27:17 AM
I just read this http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/ and found out its a common issue for many and the solution is in progress... thanks.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on May 28, 2015, 12:33:55 PM
For me I see that the dropped frames are 90% of the time within the first 24 frames. Most often frame 17, but 18, 21 and 24 occur very often too. There is some kind of pattern that drops these frames, I don't know what though.

Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Ali Oliya on May 28, 2015, 03:53:06 PM
@kharak

For me its completely random. sometime first frames sometimes last frames. i switched the 1.6.5 and its working fine so far.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Panz on May 31, 2015, 05:15:58 PM
Anybody else has problems with timecode?
When i import audio and dng sequences in davinci resolve the timecode of audio and video is different by a minute, maybe more maybe less, which makes syncing tedious because i have to fix every audio file timecode manually.

Nobody has this problem? I think 1.5something was working ok on that regard.

Still, latest versions are very fast and stable!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: kgv5 on May 31, 2015, 08:08:01 PM
@kharak

For me its completely random. sometime first frames sometimes last frames. i switched the 1.6.5 and its working fine so far.

I am also using 1.6.5, i noticed that 1.7.x gives me red frames from time to time (randomly), i dont know why because the same MLVs mounted with mlvfs have no dropped frames at all.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: X-STATE on June 02, 2015, 04:26:10 PM
@chmee
Hi! Is it possible to add to the raw2cdng a possible correction hot pixels?
Like here in MLVFS Bad pixel correction?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on June 02, 2015, 05:56:26 PM
hot pixel in r and b are corrected with chroma smooting 2x2. yes, this is inside.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: X-STATE on June 02, 2015, 08:15:03 PM
wow! It is very cool! :D
tnx
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Jay429 on June 04, 2015, 09:53:38 PM
Hi for some reason i cant get any version of raw2cdng working it keeps crashing every-time i press convert.. am missing something?
im basically dragging, the MLV/Raw file into the window software so it shows up in the list then press convert and it crashes.
im on a windows 8 pc, using latest build on a Cannon 5D3
if i use MLrawviewer it all works fine
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: allemyr on July 29, 2015, 01:57:35 PM
Thanks for this app. It's awesome I'am using 1.6.5 and have no problems at all. Used it  for over a year now.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: odya_kun on July 30, 2015, 03:58:44 PM
Why DNG files that produced by raw2cdng (12bit & 16bit) are not playable through Resolve media pool?
After dropping it to the timeline playback working ok.
MLVRawViewer DNGs plays good too but they are very small in size, maybe they have a 10bit quality or something?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: dmilligan on July 31, 2015, 01:18:46 AM
No, they are just lossless compressed (there is no loss in quality).
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 04, 2015, 03:01:40 PM
Sorry for my absence - as you see i couldnt find any time for the converter - and there's just this one big bug with lost frames while convertion. next time i will embed raw-convertion from xiaomi yi cams as a sidestep..

@Jay429
often its a problem with the filenaming and/or the path-names. did you tried without special or country specific characters (and no whitespaces)?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: X-STATE on August 16, 2015, 08:34:56 AM
@chmee
Canon 70D:
-Rec RAW video : afte convert in raw2cdng.1.7.4 ACR identifies *.dng file like Canon 6D
-Rec MLV video : afte convert in raw2cdng.1.7.4 ACR identifies *.dng file like Canon 70D
what do you think, is the problem of the raw2cdng or ML?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on August 16, 2015, 08:57:46 AM
70D and 6D shares the same color matrices. Only difference in metadata should be the name of the camera model?
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: X-STATE on August 16, 2015, 09:41:38 AM
I'm worried about VisonLOG
It works correctly in this case?
When a DNG file from 70D be applicable from the camera profile 6D
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Danne on August 16, 2015, 10:01:46 AM
If it works for 6D it should behave exactly the same as for 70D I believe.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Slasheal on August 26, 2015, 05:52:17 PM
I just tried it.
Two problems to report:

1- Lens correction can't be applied with ACR.
2- When oppened with Speedgrade, I get a green cast.
The only way I have found since then, is to convert all the DNGs to a 16bit tiff (11mb per frame  :-[)
Only then, I don't get a green color shift.

Ps:
I'm also using vision log... Because I'd like to beable to apply luts.


I installed bootcamp just because I thought that this software would help me to get usable DNGs on Speedgrade.. It hasn't.  :-\
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on August 27, 2015, 08:33:03 AM
(*) does other Converter give you the ability for automatic lens-corrections? you can set it manually.
(*) Speedgrade needs manual adjustments. often discussed here, with examples.
Quote
..[edit half a year later] speedgrade needs a color-lut/color-matrix to work accordingly with raw-files..
in raw2cdng 1.4.8 (zip) is an example - .look - file....

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: mothaibaphoto on August 27, 2015, 12:02:44 PM
@chmee: Compared to mlv_dump your converter brings correct WB, but looses lens(model, F-number) shutter speed and ISO :( Yes, I can set lens manually, if I can remember :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: dmilligan on August 27, 2015, 12:55:53 PM
(*) does other Converter give you the ability for automatic lens-corrections? you can set it manually.
It's not that you can't apply lens-corrections, it's just that they are applied wrongly (sometimes) and might not be noticeable b/c ACR makes some assumptions about the FOV. Actually, for crop mode, they will already apply correctly because the pixel size is the same as what ACR assumes.

http://www.magiclantern.fm/forum/index.php?topic=13152.msg153252#msg153252

One problem is that there's no way to handle non-centered footage. If you try to apply lens corrections to non-centered footage, they will be wrong, and there's apparently no tag to specify this to ACR.

(*) Speedgrade needs manual adjustments. often discussed here, with examples.
regards chmee
I found a workaround to get the "right" color to start with in SpeedGrade is to create a Premiere project and add the footage to it first, then do a "direct link" and jump over to SpeedGrade.


You might also be interested in this: http://www.magiclantern.fm/forum/index.php?topic=13152.msg152425#msg152425
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Slasheal on August 27, 2015, 08:58:02 PM
Quote
I found a workaround to get the "right" color to start with in SpeedGrade is to create a Premiere project and add the footage to it first, then do a "direct link" and jump over to SpeedGrade.


This is great!
I tried it and I confirm it works 100%. :)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: The Director on October 18, 2015, 12:35:21 PM
big problem with the latest version. it cuts a half of files. on two parts. without sound.
it isn't known why. and so almost all the time. here through RawFlow. unpacks RawFlow correctly.

(https://www.magiclantern.fm/forum/proxy.php?request=http%3A%2F%2Fs020.radikal.ru%2Fi713%2F1510%2Fec%2Ffd809207dd71.jpg&hash=d5c084b7e9c5ab433cb47b6043a3c755)
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: mothaibaphoto on October 18, 2015, 01:58:27 PM
That latest version looses random frames. It's known bug. Resolve treat it this way. Use 1.6.5 instead.
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: andy kh on October 28, 2015, 07:52:27 PM
i never knew that raw2cdng can play mlv raw files. i just drag and drop a mlv file in raw2cdng and it starts playing like a video player
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: chmee on November 17, 2015, 01:00:50 AM
sorry once again i stopped patching raw2cdng. i just had a lot to do and i doubt, i'll find more time for the project. i dont know how many guys are still using raw2cdng :)

@andy_kh :) you're welcome. yes, thats built in since (i think) 1.6.0.

Press Keys:
O - Original Whitebalance
P - Preset Balances
Right/Left Key - change CT manualy
Up/Down - change GM-Shift
Q - change Quality
X or ESC = Exit

You can use raw2cdng as Standard App if doubleclicking .raw or .mlv. (open with..)

regards chmee
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: hjfilmspeed on November 17, 2015, 07:52:01 AM
Hello again! I used to love this! I can't remember when but this app got a little buggy for my set up at one point but I look forward to trying again soon when I get a chance! Thank you for your continued work!
Title: Re: The CinemaDNG Discussion (raw2cdng)
Post by: Kharak on November 17, 2015, 10:21:33 AM
Hey Chmee! Glad to see you around again, I still run everything through raw2cdng 1.6.5.

Your 1.7.4 version has lightning fast conversion, but the random dropped frames made me revert back to 1.6.5.


But I tried to play mlv files in raw2cdng, but I could not succeed, I pull the Mlv's in and they just queue up for conversion, how exactly do I do it? I use MLRawViewer as default video player for my Mlv's, but would like to try out your player too. Could the problem be that everytime I run raw2cdng, windows always pops up the "The Publisher could not be verified" window and I have to click run.

I ca