Hey there.. I'm currently trying to realise a CinemaDNG-Converter. First i thought it would just be an existing xmp-packet inside the dng-frame, but it wasn't. After reading the CinemaDNG-Definition i saw, there are about 35 mandatory tags.
DNG Specification 1.4.0.0 - June 2012 (http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf) - CinemaDNG Specification 1.1.0.0 - September 2011 (http://www.cinema-dng.com/wp-content/uploads/2013/06/CinemaDNG_Format_Specification_v1_1.pdf)
The CinemaDNG-File is just a TIFF/PE File (as DNG is also) with these CDNG-Tags. I wrote an App for Win7, building a CDNG-Header with the raw-pictures from the Camera-raw-file. They are 14Bit, isnt it? I figured out, that Resolve and Speedgrade cant handle 14Bit Raw-Frames! Just 8, 10 or 16Bit, maybe 12Bit. So, now i'm sitting on converting these 14bit to 16Bit-Chunks. These CinemaDNG-Sequences are recognised by Resolve and Speedgrade, but they are chaotic because of wrong handling/converting.
If asking, why the simple DNG-Sequences are not usable, it might be that
(+) missing CDNG-Tags
(+) IFD0-Entry is the jpg-Preview in 128x96px 8Bit - thats the one you see in Resolve.
(+) Raw-Data are packed into a subIFD - as in the DNG-Definition described.
(+) they are 14bit.
Questions:
(*1) Raw-Data - Subpixel-Chunks, MSB or LSB first? LittleEndian, so LSB first, right?
(*2) is it worth that? Maybe another Fileformat would be more usable? DPX? EXR? TIF?
regards chmee
regarding raw2cdng
i decided to stay on the same name and same tool with integrated mlv-convert (since 1.4.5). Resolve, Speedgrade and all ACR-using-Apps are coping well with the cdng's. With 1.5.0 (coming next) there s a new GUI, and on NAB(07.04.2014) Adobe releases a bigger update promising a better workflow with cdng. 1.5.0 is online. since 1.6.5 audiosync seems stable, colormatrices are refreshed. you find more info on my webblog.
please look in my blog-entry (http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/) to find the newest version!
bitbucket-sourcecode c# (with 1.5.0beta2 i've left it into the free world)
https://bitbucket.org/chmee/raw2cdng
actual release : raw2cdng 1.6.5 for windows
http://www.magiclantern.fm/forum/index.php?topic=5618.msg136306#msg136306
regards chmee
1) https://bitbucket.org/hudson/magic-lantern/src/tip/src/raw.h . Don't ask me about endianness stuff, I've just scrambled these bits until it started to look good :P
ARM is little-endian.
lmao.
thx for coming chmee, save us from Adobe hell.
Quote from: chmee on May 23, 2013, 10:46:55 AM
I figured out, that Resolve and Speedgrade cant handle 14Bit Raw-Frames! Just 8, 10 or 16Bit, maybe 12Bit. So, now i'm sitting on converting these 14bit to 16Bit-Chunks.
BMCC DNG's are 12bit log that unpack as 16bit linear in resolve.
Good topic, since a lot of people seems to miss that compatibility.
My suggestion is (if possible), instead of making a converter for win, will not be better to make ML generate a raw with all what needed and make raw2dng able to convert directly to cinemaDNG? That way we will have a multiplatform solution, also 12 and 10 bit raw are under development so probably the converter should be able to handle different bitrates.
of course. Right now the quest on 10/12Bit is the right way to get cinemaDNG(@10Bit) straight ooCam. finally, there will be a converter computerside - and the integration/rewriting of the cdng-header is the far smallest problem. I'm just sitting to solve the 16bit converting thang, maybe after that we will have a cdng-converter till we implement the idea into raw2dng. its the experience "we" are doing now.
ah, if someone misses the thread 10/12bit -> http://www.magiclantern.fm/forum/index.php?topic=5601.100
regards chmee
I work in post, and have assimilate's scratch and resolve, I am ready and willing to test any post workflows that come out of this. Thanks for the great work, I think this one of the most exciting ml threads I have read!
Here's how the ML DNG metadata differs to Cinema DNG (Blackmagic Cinema Camera)
http://www.eoshd.com/uploads/dng-metadata.png
(Blackmagic Cinema Camera EXIF on the left, Canon raw DNG on the right)
As you can see stills standard DNG EXIF is very different to Cinema DNG.
DaVinci Resolve seems to read the reduced-resolution metadata for 5D Mark III footage.
There seems to be other errors like JPEG compression for the full resolution image and uncompressed for the reduced resolution image?
Any update on this?
My research shows the OP was correct, the problem is -
(+) IFD0-Entry is the jpg-Preview in 128x96px 8Bit - thats the one you see in Resolve.
(+) Raw-Data are packed into a subIFD - as in the DNG-Definition described.
Stills DNG has support for multiple images in one DNG at different resolutions (subfiles) and Cinema DNG only has one 'primary-image'. The raw data in the stills DNG is in a subfile, and seemingly the primary file is a thumbnail. Resolve reads the thumbnail data in IFD0-Entry but cannot access the raw data packed into a subIFD since it lacks support for this file structure.
Could we change Raw2DNG so subIFD becomes ifd0? I.e. pack the raw data into IFD0 instead of SUBIFD? Is that a trivial metadata change or more complex? There is a DNG SDK and Cinema DNG docs, someone just needs to understand the file format and write the correct structure into Raw2DNG's code.
I asked a Blackmagic guy via John Brawley about Resolve supporting stills DNG, and Pete confirmed it doesn't yet support DNG only Cinema DNG. I am sure they are working on it but probably quicker to change Raw2DNG.
This thread at EXIFTool was useful in pinpointing the problem - http://u88.n24.queensu.ca/exiftool/forum/index.php?topic=4927.0
just to give you some food :) Here's the link to the app. (win)
*refresh* to 0.91
---
*lol* sorry, compiling while Championsleague-Finals is a bad idea.
0.92 - http://www.dslr-kleinanzeigen.de/Download/raw2cdng.0.92.zip
*!* NOT FUNCTIONAL*!*
(+) you need installed Microsoft .net 4.0 Runtime
(+) converting limited to 5 frames
(+) just for dev/analysing
problems so far:
(*) not enough time to solve, because workin' hard. pokin' around.. tuesday i'm back free on this thing.
(*) 14bit is functional with working image
(*) 16bit shows converting problems - have no time to finish successfully. (but is workable in resolve)
(*) both are not recognised by photoshop/acr, but viewable with fe irfanview. help appreciated (maybe colormatrix)
(*) for investigating on the converting problem try rawdigger, switch between the Sensel-channels.
(*) it looks like an array not NULLED in loop. rewriting the convert-function back to "simple" bitshifting and masking.
@EOSHD
if the IFDs would be changed, it wouldnt be no more a DNG. My Application throws out a CDNG with 14Bit - but they're not readable in Resolve Speedgrade etc pp.. The 16Bit-Files are readable, but not usable :)
For the devs here. Thats the converting loop: Do you see any Problems in it?
// c# code
// source - byte[] - rawstream 14bit
// chunks - int - stripByteCounts
// output -> Dest - byte[]
public static byte[] to16(byte[] source, int chunks)
{
byte[] Dest = new Byte[chunks/14*16];
UInt32 tt = 0;
// because of bytestream i take 7 Bytes for (odd)RGRG (even)GBGB
for (var t = 0; t < chunks; t += 7)
{
Dest[tt++] = (byte)(source[t] >> 2);
Dest[tt++] = (byte)((source[t] << 6) | (source[t + 1] >> 2));
Dest[tt++] = (byte)(((source[t + 1] & 0x3) << 4) | (source[t + 2] >> 4));
Dest[tt++] = (byte)((source[t + 2] << 4) | (source[t + 3] >> 4));
Dest[tt++] = (byte)(((source[t + 3] & 0xF) << 2) | (source[t + 4] >> 6));
Dest[tt++] = (byte)((source[t + 4] << 2) | (source[t + 5] >> 6));
Dest[tt++] = (byte)(source[t + 5] & 0x3F);
Dest[tt++] = source[t + 6];
}
return Dest;
}
regards chmee
i just started playing around with resolve 9 lite - mac version - and have to say that it is hands down the best color grading program i know.
and the best thing is: it is free. the limitation is 1920x1080, but thats ok for me. you can get really great results
the tracking function is so amazing and easy to use.
so i´m really waiting hard for some conversion tools that can convert the raw file from the camera into a raw format that can be used in resolve.
i wanna thank all the developers and enthusiast for all their commitment.
if you had posted the whole code, i would have tried to fix it :)
https://github.com/chmee/misc/tree/master/%5Bcsharp%5Draw2cdng
Sorry, i'm there..
Does anyone know of a Mac version for a cinema DNG converter?
Thanks
Quote from: notdabod on May 27, 2013, 08:16:21 AM
Does anyone know of a Mac version for a cinema DNG converter?
Thanks
There is no cinemaDNG converter yet.
I tried saving a DNG to a DNG in ACR and opened it in resolve, it was just thumbnail size but it was in colour.
Since the basic problem for Resolve is that it cannot read the subIFD info in exif and only reads IFD0, I tried to change the exif IFD0 info by hand and it does work but some crucial tags cannot be changed - deleted - moved .. blocked by exiftool's author for security reasons.
If it worked to the end the next step would be a scipt to automate the whole work and could be a interesting feature of the dng2raw + .. GUIs.
But as it is now I think a request to Phill Harvey to deblock changing those tags and even provide an option for exiftool to make easy these changes is the best I am thinking of. Anyone interested (I am not) can post a request ..
http://u88.n24.queensu.ca/exiftool/forum/index.php/board,4.0.html?PHPSESSID=61s2u55elgkbd4g06blqf67nr5
The other option is, the ML team to change the dng2raw code so that it writes the exif in the IFD0 subgroup looks more difficult to me because an expert in exif data is needed.
Quote from: squig on May 27, 2013, 08:41:58 AM
I tried saving a DNG to a DNG in ACR and opened it in resolve, it was just thumbnail size but it was in colour.
What you saw is the "preview" jpeg which is embedded in every DNG. It could be full size (DNG converter gives 3 size options .. full - medium - small) but even then you would have in Resolve jpegs not raws. It's because Resolve reads only the IFD0 exif info which in PhotoDNG describes the jpeg preview and the raw data are described - taged in the subIFD group.
@IliasG
To be honest.. Did you built a DNG-File that opens in Resolve with 14Bit Data?
What TIFF-Tags do you want to change, you're not able to?
Edit: I tried to use smaller sizes (720x400px) for generating small sample files. Camera Raw behaves really strange with these data.. https://github.com/chmee/misc/tree/master/%5Braw%5Dsamples (look into the dng-metadata. the colormatrix shows an inf"inite" entry)
regards chmee
Hello chmee,
No I didn't build any DNG file, I stopped after I could not change the needed tags.
Blocked tiff tags. 0x0117 (StripByteCounts) - 0x111 (StripOffsets). As you understand these are the most significant tags ..
I cannot open any of your samples (ACR 5.7, Lightroom 4.4-demo, Dcraw,) nor read - change any exif tag with exiftool !!!... "Error: Not a valid TIFF (looks more like a XMP) - 1280_540.dng"
I think, You have to ask for help from Phil Harvey, he is THE EXPERT in exifs and a very good guy.
StripByteCounts and StripOffsets.. thought that. :) It makes no sense to change them in EXIF-Tool. If you want to change them, take a hex-editor. Just not to be misunderstood - by now i dont generate the header, i just use another one and rewrite the important tags. it makes no sense rewriting the whole tiff-header-creation thing, thats already in the magiclantern package - but i m no c(++)-guy.
(*) The samples in the github are dng's converted with raw2dng.exe from 5D3-raws - no changes by me! strange, isnt it? these are raw samples on where we can start talking about - one database and changes in it.
(*) by now i am just interested in the converting from 14 to 16 bit. i m a bit puzzled, that just appending zwo bits to the core data didnt worked, but hey, (michael jackson) "you are not alone" - we're on the way to find a solution..
(*) by the way, irfanview tries his best to show strange data - and in my case it does..
regards chmee
But if they (the samples) straight from raw2dng why there is no exif ??. Ohh now I see it .. corrupted download ..
Redowloading ..
Believe it or not I had not read this thread from the start :(
You improved alot !!.
I am looking at 704X400
When changing the bitdepth to 16 logic say you have to change StripByteCounts
704*400*14/8 = 492800 while it is 504000 in exif !!.
704*400*16/8 = 563200
Don't know if StripOffsets need a change to 38400... as is the usual for raw2dng built DNGs
Quote
704*400*14/8 = 492800 while it is 504000 in exif !!.
because its 720x400@14Bit -> StripByteCounts 504000. StripOffsets is a calculated value, where the raw-data starts. Look with a hexeditor into it.. we're talkin 'bout files coming straight from raw2dng.exe(!)
regarcs chmee
Quote from: chmee on May 29, 2013, 05:07:55 AM
because its 720x400@14Bit -> StripByteCounts 504000. StripOffsets is a calculated value, where the raw-data starts. Look with a hexeditor into it.. we're talkin 'bout files coming straight from raw2dng.exe(!)
regarcs chmee
And where did 704 came from ???!!. This proves that if you are searching Exif tags instead of going to sleep you get blind :)
I have no experience nor the free time to start digging with hex editors ..
You mean that you removed the thumbnail and the subIFD header so the raw data start at 1536 ??.
no. cinema dng and "simple" dng differs much - in terms of mandatory tags. i use a cdng-template, i merged myself from looking into another cdng-frames. just to clarify some things:
(*) dng cant be converted into cinema dng with only two or three another tags. but maybe these videoapps understand dng and dont need any cdng-tags?!
(*) main problem is, 14bit bayer is not useable for cinemadng, because no videosoftware can read 14bit frames as sequence. (my app is able to output 14bit cdng - but there is no software able handling it)
(*) dng batch converting is a "half baked" solution, because it ends in 8bit-sequences with loosing all benefits
(*) converting into tiff-rgb 16bit needs demosaicing and color matrix handling -> not my intention (ffmpeg?)
(*) my app (in actual state) does some weird things while converting to 16bit-bayer - and thats the point of "not-usability"
(+) the bayer structure is ok, because cinema dng can handle it, also the apps, that are handling cdng
(+) it must be converted into 10 or 16bit bayer. 10bit would be worse than 16bit, but definitively better than 8bit baked.
regards chmee
I feel like RAW to Cinema DNG GUI would be amazing!!!! Opening right up in the free version of resolve would be alot of fun!
Just to give you some hope :) The app is working, when i'm writing synthetic data (10, 12 and 16Bit Output) - but i'm stuck understanding the "simple" 14Bit-RAW-Data. Taking the short RAW-Example put on my server (https://github.com/chmee/misc/tree/master/%5Braw%5Dsamples (zipped raw samples 1280x540))
NEED YOUR HELP FOR THE LAST THING!
Example from 1280_540.raw - first 7 Bytes describe the first 4 Sensels ( RGRG)
hex - 40 29 42 c7 4c 9c 2a
dec - 64 41 66 199 76 156 42
binary:
| 01000000 | 00101001 | 01000010 | 11000111 | 01001100 | 10011100 | 00101010 |
as far as i understand restructuring to 14 bit goes:
| 01000000001010 | 01010000101100 | 01110100110010 | 01110000101010
dec - 4106 | 5164 | 7474 | 7210
BUT this data dont correspondend with the image i see in 14bit. its a uniform background - greyish on Blacklevel 2046, reddish with blacklevel 0. both pixels Pixel1(RG) and Pixel2(RG) have to be (more or less) same value, what is going on?
regards chmee
ok, here's a v0.99 for beta testing. thanks to reyalp for the hint.
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
all responses, hints are very welcome. regards chmee
Hello chmee,
this is very exciting. However, after converting a raw file that I created with the 5dmk2 without changing the bitrate, I could not open the dngs with resolve 9. It did not "see" the dngs. So I opened one dng with photoshop where I got presented a very contrasty image, barely recognizable without blue channel (will post dng tomorrow).
same problem as last post, i thought it would make a single cdng file but made a lot of dngs that i cannot open in resolve
thanks for response. actually i just tested with 5DIII-Files. So it would be fine you're leaving somewhere a short & small raw-file from your bodies. Ah btw. No, it is a filesequence with cinema-dng pictures (their ending is dng as well) - for using in resolve its mandatory changing to 16bit! olik, post a screenshot. (seems i have to delete the 14bit-converting :)
regards chmee
I did a quick test with my 5D3 and converting into 16bit, works like a charm. I'll be even more happy when one of the Mac folk ports it to OSX. I am sincerely impressed with your skill.
What's the increase in file size converting to 16bit?
Fairly negligible really. I think it increased by maybe 10% on the quick test that I ran.
@ Chmee,
Congratulations, and many thanks.
Regarding file size.
A good solution would be to store the data in the exported cine-DNG at 12bit log (after applying a LUT on the 14bit linear raw data) and then inject a inverse LUT (12bit log to 16bit linear) in the corresponding exif tag. It's the same as BlackMagick do and Nikon's "visually lossless" raws also. This would give a 12/16 file size.
What happens if we pass this 16bit cine-DNG through Adobe DNG converter. If it does not destroy the cine-DNG format is would give a respectable size decrease as by default it applies lossless compression which for such files can compress at about half the size.
Tested again for the 5dmk2, this time I changed the bitrate from 12 to 16bit. This solved my issue, I can now import to resolve. I have a strong green cast though, but it is perfectly correctable. Wonderful job! I love it :-)
Here the C-dng in resolve
(https://lh5.googleusercontent.com/-4_0OEZ2yHis/UayBKvOC7NI/AAAAAAAANcY/kxVO0LMC8Ik/s640/Screen%2520Shot%25202013-06-03%2520at%25201.32.46%2520PM.png)
vs h264 (same camera settings)
(https://lh4.googleusercontent.com/-DxK0xs3_MJU/UayBxqKFOTI/AAAAAAAANdA/X-Sp4nqpc-Q/s640/Screen%2520Shot%25202013-06-03%2520at%25201.33.26%2520PM.png)
and the first DNG of the sequence
https://picasaweb.google.com/lh/photo/tZ7PtaVRzHpdgX1PKcC_0aHmTw6ITH9KGra-l8uiJak?feat=directlink (https://picasaweb.google.com/lh/photo/tZ7PtaVRzHpdgX1PKcC_0aHmTw6ITH9KGra-l8uiJak?feat=directlink)
Quote from: olik on June 03, 2013, 01:33:35 PM
I have a strong green cast though
The black level is too high; it's probably hardcoded at 2048, which is usually right for 5D3, but too big for 5D2. The one from exif should be a good starting point.
I can load the app in Osx with mono and load in a file, but how to start it?
(http://i.imgur.com/C7Hscex.png)
EDIT, maybe it did all, I got a lot of dng but with some stranges names due to the win \ for folders I suppose
Also mono gives this error when loading it:
mono[75121] <Error>: kCGErrorInvalidConnection: CGSGetWindowTags: Invalid connection
So far on osx when converting a raw file I got this file structure at the end:
rawfilefolder/M0000001.RAW
rawfilefolder\ML_16/ <--this is not a subfolder of rawfilefolder it's another empty folder
rawfilefolder\ML_16\ML_16_000000.dng
rawfilefolder\ML_16\ML_16_000001.dng
rawfilefolder\ML_16\ML_16_000002.dng
... and so on with files having the win folder structure in the name.
While we already talk about using this software on different operating systems, I tried using it on Ubuntu Linux with mono, without luck;
mono '/raw2cdng.0.99.exe'
Missing method EnableVisualStyles in assembly raw2cdng.0.99.exe, type System.Windows.Forms.Application
Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.
File name: 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
[ERROR] FATAL UNHANDLED EXCEPTION: System.IO.FileNotFoundException: Could not load file or assembly 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' or one of its dependencies.
File name: 'System.Windows.Forms, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'
has anyone an idea?
Quote from: chmee on June 03, 2013, 08:19:45 AM
seems i have to delete the 14bit-converting :)
+1
thanks for the testers.
summarizing for the next versions
* >2GB filehandling
* path-strings uniform and no windows-dumb backslash
* pink fringing
* vertical lines (could be solved with 0xc619 or 0xc61b )
* 10 and 12 bit files (no logarithmic with LUT for now - think, there is a tag for it - linearizationTable 0xc618)
* batchmode?!
* handling other bodies with their metadata (colormatrix, whitelevel and so on)
regards chmee
Quote from: chmee on June 03, 2013, 10:02:33 PM
* vertical lines (could be solved with 0xc619 or 0xc61b )
This is black level for each column, but for the banding issue we need EV correction for each column (mod 8 ). Do you know any tag for that?
ahh. ok. there is no metadata-array for ev correction - as i see that now.
Some Question there have been:
14 to 16Bit -> ~14.2% increase (16/14 = 8/7)
14 to 12Bit -> ~14,2% decrease(12/14 = 6/7)
@IliasG
No, logarithmic transforming is not my intention. for 12Bit i will cut LSB 2 Bit - we'll gonna see, how many data in the shadows is going to be lost - i quite think, its negligible :) Dont forget - the Magiclantern Team, all the coders around it gave you a weapon you never had the money for that - and this "techspec-more is better"- blahblah doesnt make a good film - your movie is not depending on logarithmic 16bit, but on "substance"!
@olik
what version-number of .net-framework is running in mono? it needs 4.0
This is truly amazing and I´ve tested your tool shortly tonight with my 5D2 and I had to lower the blacklevel alot I think I had it on 1024 (just trying different settings to figure out how it works..). If it was on 2000 I had a very dark picture with alot of green. What would be the right setting for 5D2? Whys does it doffer from 5D3?
I´m new to resolve so I have to learn alot to be able to grade thorougly in the program but the basics are the same as the comp programs I use daily. (The Foundry Nuke and Eyeon Fusion).
Its so nice to be able to skip the Adobe programs - I don´t trust them at all for grading RAW sequences...
Cheers
Just wanted to chime in, that I managed to open a file originating from 550D in Resolve. I did run raw2cdng under Parallels in OS X Lion, in 16bit mode of course. The file looked basically OK, but it was just something I shot in a minute. I couldn't do any actual testing, as the clip had horrific noise and no real black or white to begin with. Just wanted to see if it works at all. Sorry, I have not time to do anything more helpful right now.
BTW, I tried also in Speedgrade, just for fun. That didn't work at all (I didn't expect it to). The image was mostly green. I don't think Speedgrade even supports CDNG, maybe only Alexa and R3D.
Chmee you are doing as a big favor. No much point in grading raw video footage anywhere else than in Resolve -- at the moment at least. I tested earlier some clips in AE and graded them wit Color Finesse. That workflow didn't make much sense. I mean, to first bake the raw data into static rgb and then start grading. Very frustrating. So thanks, great stuff.
Quote@olik
what version-number of .net-framework is running in mono? it needs 4.0
I have mono-runtime 2.10.8.1 but it seams that 4.0 is not even out, 3.0.10 was just released as a beta (http://www.go-mono.com/mono-downloads/download.html) am I missing sth? :)
@jtkleine - please leave a screenshot, what body you are using? maybe its a 12bit body? is some 12bit-Body able to raw-output? if you want to, just upload a short/small rawfile to analyse, whats happening.
http://dslr-kleinanzeigen.de/Tools/Body_Database/index.php?filter=prodeqcanon|rawbiteq12
@olik dont know much about mono - i just use linux-derivates for xbmc :)
generally, please wait till thursday, for now i'm working all day long - i cant do any changes.
OK, I shoot a small clip and upload the raw file a little later. Where should I upload it? Maybe I just put in on a server and link it here. A screen shot too. But I don't have time today to fiddle with the black level settings in raw2cdng, so I just use the default.
How can I determinate the correct Blacklevel? (for the 50D). Default was at 2037 but that completely crushes the file. I just tried random numbers, around 1800 gives about the same exposure as Raw2DNG, but the colors are different when opened in ACR.
Anyway, it's workable, going to find out if my computer can keep up editing cinemaDNG in resolve... Thanks for the development!
By averaging the black areas from the DNG I've asked you to post a couple of times...
http://www.magiclantern.fm/forum/index.php?topic=5586.msg44299#msg44299
What a cool day! Magic Lantern community making breakthroughs all day long like an absolute BOSS 8) Amazing work!
Thanks for your hard work on this man, I can't wait to finally play with my footage in Resolve... Awesome :)
Thanks for the tool chmee! My footage looked good in resolve.
One question do you set the black level and white level from the raw file or is this a default setting? Should I change this value to suit the 550D? I assume this value is in the EXIF somewhere does it depend on the footage or is it always the same?
In other tools such as RAWanizer, using the tiff files, there was a purple cast and it clipped the top of the whites. While using your tool it cam out fine and I was able to access the camera raw tools in resolve which I couldn't do before.
Quote from: a1ex on June 03, 2013, 11:48:18 PM
By averaging the black areas from the DNG I've asked you to post a couple of times...
http://www.magiclantern.fm/forum/index.php?topic=5586.msg44299#msg44299
Sorry a1ex, there is no Silent DNG option on all of the 50D builds I've been using.
I'd gladly help out, but in this case, I don't know how.
Ask the guys to enable it in features.h.
@chmee, I'm shooting with 550D. I suppose it's a 14bit body.
Here's a link to a raw file:
http://kleine.me/clients/mlraw/M04-0102.RAW (http://kleine.me/clients/mlraw/M04-0102.RAW)
Screenshots from Resove & Speedgrade:
(http://kleine.me/clients/mlraw/resolve_screenshot.png)
(http://kleine.me/clients/mlraw/speedgrade_screenshot.png)
Quote from: a1ex on June 03, 2013, 11:48:18 PM
By averaging the black areas from the DNG I've asked you to post a couple of times...
http://www.magiclantern.fm/forum/index.php?topic=5586.msg44299#msg44299
So I should do the same for my 5D2? Since the black and white levels differs from 5D3..
Or are there any precise nr for the 5D2 I should use?
For 5D2, the black value from raw footer should be OK. I don't know if this app uses it or not.
If not, convert the raw with raw2dng and look in the exif of one DNG file.
Quote from: chmee on June 03, 2013, 10:34:05 PM
....
@IliasG
No, logarithmic transforming is not my intention. for 12Bit i will cut LSB 2 Bit - we'll gonna see, how many data in the shadows is going to be lost - i quite think, its negligible :) Dont forget - the Magiclantern Team, all the coders around it gave you a weapon you never had the money for that - and this "techspec-more is better"- blahblah doesnt make a good film - your movie is not depending on logarithmic 16bit, but on "substance"!
...
I wouldn't have any objection for 14to12bit linear by truncating ..but on the single sample that I have seen with this method there is posterization (middle bottom on the windows frame).
http://www.magiclantern.fm/forum/index.php?topic=5601.msg39482#msg39482
Will you also truncate the "Black Level" value in exif ?.
Don't get me wrong, I am pushing this Log conversion in raw2dng as a first test to try it's effectiveness and if for example 14to12 or 10bit log will be proved of good quality ML team could use this strategy in-camera to improve the recording module for longer and/or faster fps shots ..
I can calculate and upload (within 24h) what I think are optimum LUTs (compress and linearize) if needed.
(*) IliasG, if you're interested in implementing logarithmic scale in every kind of transcoding-process, you're welcome. (the reason why ACR didnt open the file was, the whitelevel was higher than the possible maximum output - if there was for example whitelevel 15.000 - its 12 bit overflow error). and of course the black- and whitelevel have to be corrected. so it means for me, recalculating with lin16bit->log12bit-table and embedding log->lin-LUT into cdng.
(*) i am not so sure, if recalculating in body is possible - as i understand, the possibility to record the sensoroutput in realtime is reasoned by shipping data stright to the dma, without digic and other computing-helper..
(*) jtkleine - i've seen this in my testversion of speedgrade as well - for now i have no clue, why its handling different.
regards chmee
Hi everyone, fantastic work so far, big up.
I've had some time to experiment with the firmware, trying to find a workflow that works for FCPX and Resolve. raw2cdng is great but will be amazing when there's a batch version and hopefully mac also :D
Anyway, for me workflow is the thing that will decide if I'll be able to use the firmware or not, and now raw2cdng is getting close! Was trying all kinds of ways to get a cdng before but it was painful.
So now when I used raw2cdng all was good getting into Resolve, but then the issues started. First take a look at the video:
So a couple of things.
- There's a lot of what I would say is chromatic aberation. Pinkish especially in the trees. This could be a white balance thing combined with the sunset highligt, bit my best guess is that something either happened in the RAW recording or the raw2cdng conversion. Anyone with the same issues?
- Then the workflow. One word: painful. I think the timecode is breaking the connection with resolve, because there wasn't a way to get it to automatically connect to the old files. It has done in my previous tests but they were much smaller and without raw2cdng. So maybe something has gone missing in the raw2cdng conversion? All is ok until the XML is going back to Resolve from FCPX.
Any thoughts on the issues?
And keep up the fantastic work!
(*) pink fringes - it seems, its a canon problem in a strange way. other apps are handling that. it may be, there are some highcontrast-points higher than whitelevel point, and these are badly interpreted - so i have to look into another solutions, whats the best. hard limiting sensel values over whitelevel-value? and so on. (its on the todolist)
(*) timecode. hmm. could you post a xml-file in pastebin? just to see, what strange data there are? (i am win-user, so i have no fcpx)
regards chmee
I see a lot of people saying they want to be able to grade in Resolve with CinemaDNG, and I understand it'd be more convenient that way, but are you losing any data when you grade with converted TIFF files from ACR?
Quote from: yobarry on June 04, 2013, 11:42:43 PM
I see a lot of people saying they want to be able to grade in Resolve with CinemaDNG, and I understand it'd be more convenient that way, but are you losing any data when you grade with converted TIFF files from ACR?
I wouldn't say you loose that much data, it's negligible, you would loose all the benefits of not having processed the image though. Everything from white balance, noise reduction, sharpening would bring out more artefacts etc. But it would probably very subtle.
The biggest reason though is workflow, TIFF is at least 3 times as much space on your hard drive from a RAW file. If in 16 bit probably even more.
There's no way I would think that's a sustainable solution, even though at the moment TIFF is the safest workflow since it's reliable in conversion from the RAW file. Which the CinemaDNG isn't yet, but it will be!
Quote from: chmee on June 04, 2013, 10:10:50 PM
(*) timecode. hmm. could you post a xml-file in pastebin? just to see, what strange data there are? (i am win-user, so i have no fcpx)
I'll try to post both a xml with video that works with timecode and one of the broken ones tomorrow. Thanks for the great development!
Thanks for the info
Quote from: roughstudios on June 04, 2013, 11:56:01 PM
I wouldn't say you loose that much data, it's negligible, you would loose all the benefits of not having processed the image though. Everything from white balance, noise reduction, sharpening would bring out more artefacts etc. But it would probably very subtle.
I'm a little unsure of the difference between raw and TIFF, I thought TIFF was just a container, so it just 'held' all the raw data, changing nothing?
For instance if you set a white balance in ACR and convert to a TIFF, is that white balance still 'raw' or is it baked in? And if you adjusted nothing and put sharpening/NR to zero, does the TIFF file still contain the extra data (exposure leeway) that you could have adjusted in ACR?
Quote from: roughstudios on June 04, 2013, 09:13:13 PM
- There's a lot of what I would say is chromatic aberation. Pinkish especially in the trees. This could be a white balance thing combined with the sunset highligt, bit my best guess is that something either happened in the RAW recording or the raw2cdng conversion. Anyone with the same issues?
Resolve wont do any chromatic aberration fixing to cDNG's like ACR etc will. I see CA with BMCC DNG's as well. You need to fix it yourself in resolve.
Quote from: CaptainHook on June 05, 2013, 02:34:15 AM
Resolve wont do any chromatic aberration fixing to cDNG's like ACR etc will. I see CA with BMCC DNG's as well. You need to fix it yourself in resolve.
I know, that's why it needs to be fixed in the conversion :)
Personally I would choose a BMCC any day over the 5D if the workflow means going into manual RAW processing, then converting to CDNG to get into resolve and then edit then back for grading. It just isn't worth the time spent fixing. It might work for commercial stuff, but for documentary there's just too much material to process.
I'm also getting the pink fringes. It really shows up when adding sharpening in resolve, and goes green when using blur. It seems to be in high contrast areas. I thought it could be related to not down sampling from the raw Bayer data, but a down sample didn't seem to help.
I was watching a tutorial yesterday it was covering how u can change how resolve see the footage. So it looks how u shot it, I'm at a editing job now when I get home I'll post link
Guys is the pink fringing the same as what I reported to chmee? Here's how I see it...
(http://www.eoshd.com/uploads/pink-dots-and-vertical-lines.jpg)
The standard Raw2DNG frames don't have this. What is that code doing differently?
Quote from: EOSHD on June 05, 2013, 11:50:41 PM
Guys is the pink fringing the same as what I reported to chmee? Here's how I see it...
I think it is. It can seem a "fringe" around contrasty areas of the image. I've tried changing to BMC Film interpretation in the Camera Raw tab and it "solves" it. But I don't really like how it re-interpret it as BMC.
Thanks a lot for this tool chmee!!! I'd love to have batch enconding and renaming but it's already great to been able to do direct RAW->CDNG processing.
Here's what I get...
No sharpening (This actually looked fine until ImageShack added compression:
(http://imageshack.us/scaled/thumb/842/screenshot20130605at231.png)
+5 sharpening:
(http://imageshack.us/scaled/thumb/46/screenshot20130605at231.png)
-5 sharpening:
(http://imageshack.us/scaled/thumb/801/screenshot20130605at231.png)
Hey guys,
I've converted a few clips to cinema dng but they're not showing up in DaVinci file browser. Any ideas?
If I use the normal raw2dng app I can see the sequence in DaVinci, but it looks like balls. If I use raw2cdng the sequence doesn't appear at all and the folder presents as though it's empty.
Thanks in advance!
Dave, have you tried setting it at 16 bit instead of 14 bit (in raw2cdng)? That worked for me.
Amazing! Thanks Irri!
Now all I need to do is to solve my black point issue and I can delete 800GB of .TIFs and give Lightroom a breather.
Don't thank me; there are people doing fantastic work here, and I'm not one of them :)
Glad you got it working.
I can't get raw2cdng running under Mono on my Mac, either. I'm running the latest version of Mono (just installed from Macports). No luck in CrossOver, either.
@IliasG
after some thoughts and discussions i think, it makes sense to lower the bitamount without losing any informations - so i would give the logarithmic idea a try. The final challenge would be to get the rawdata in a 10bit@log space and compressing it - because we can :)
(small patchversion is on the way - just got a job today, so i think the "bigger" patches will be on weekend)
Quote from: Oedipax on June 06, 2013, 04:09:28 AM
I can't get raw2cdng running under Mono on my Mac, either. I'm running the latest version of Mono (just installed from Macports). No luck in CrossOver, either.
We won't have to wait much longer http://www.magiclantern.fm/forum/index.php?topic=6218.msg46920;topicseen#msg46920
Quote from: Oedipax on June 06, 2013, 04:09:28 AM
I can't get raw2cdng running under Mono on my Mac, either. I'm running the latest version of Mono (just installed from Macports). No luck in CrossOver, either.
Don't install with macport, use the precompiled binary from mono site, that worked for me. I've posted the link in raw2dng.app topic
Quote from: chmee on June 06, 2013, 10:46:15 AM
@IliasG
after some thoughts and discussions i think, it makes sense to lower the bitamount without losing any informations - so i would give the logarithmic idea a try. The final challenge would be to get the rawdata in a 10bit@log space and compressing it - because we can :)
(small patchversion is on the way - just got a job today, so i think the "bigger" patches will be on weekend)
I was away from my PC last week (and I'll be away again) so I could not give any try on this but today I'll be at home ...
Just a summary of my thoughts for the log curve.
We can clip most of the unused "below black point" levels. Now it is around 2048/14bit, we can keep 64 or 32 of them (or less) if we want a footroom just in case something goes wrong with Black Point. and say 64 or 32 over the Black Point.
It's desirable to keep linearity at the darks.
So for 14to12bit the lowest (say 32 below black + 32 ) could be mapped 1:1 giving a 4X slope at the linear part and then start the log part.
For 14to10bit it is a bit more complicated as a 16X slope would needed for 1:1 mapping but this way we will have heavy compression at highlights. I am thinking of an intermediate solution, slope less than 8X ( 8X gives levels density equivalent to 13bits) and maybe use smaller linear part and less (or zero)"below black" levels.
Anyway we can try for a start with rec.709 curve (linear with slope 4.5 and then gamma=2.22)
http://www.poynton.com/notes/colour_and_gamma/GammaFAQ.html#gamma_correction
@chmee:
As soon as you implement Commandline, I will be happy to include it in BATCHelor. :)
(it's sticky'd if you don't know it.)
Is it planned?
Still need a 5DII .RAW File? I have one.
don't know if this will help but it states here that black level is 1023 and white level is 15600
http://lclevy.free.fr/cr2/
as far as the 5d Mark 2 goes
The values change depending on video mode; 1023 is just a rough starting point. On the same link, it says that black point is found by averaging the black borders from the image (which ML does).
Quote from: IliasG on June 06, 2013, 03:09:11 PM
...
Anyway we can try for a start with rec.709 curve (linear with slope 4.5 and then gamma=2.22)
http://www.poynton.com/notes/colour_and_gamma/GammaFAQ.html#gamma_correction
Quick and dirty calculations in Excel spreadsheet for rec.709 curves.
Input is a typical Canon raw, 14bit linear. Clipped at -31 below the Black Level, mapped to 10bit gamma rec.709.
The inverse mapping (linearization table) which should be injected in exif tag is from 10bit gamma to 16bit linear with Black Level at 141/16bit.
http://rapidshare.com/files/3859944133/curves-14bit_to_10bit_gamma-rec709.xlsx
I changed slightly the offset to fight a discontinuity at the crosspoint.
Gives a good balance between the linear part and highlight compression. The levels's density at the darks is a bit better than 12bit linear while keeping highlight compression robust enough to withstand heavy grading (300 levels for the last stop).
I know it would be an extra step but are there any dng to cinema dng converters?
I was able to successfully convert a couple DNG files to 16 bit DNG and now it only does 14 bit files and then when trying to upconvert to 16 bit I get this error message:
"index was outside the bounds of the array"
Any idea how to fix this?
Thanks for any help.
Zach
RAWMAGIC VS RAW2DNG
Same shot: Build June 6 1920x720 25p no skipped frame. 5D MARK III. Scandisk 60 mb/s
- RAWMAGIC (beta4) open in DaVinci Resolve setting withe balance tungsten - exported as pro res HQ
- RAWTODNG (0.11) open in ACR setting tungten exported from compressor as pro res HQ
Nothing more applied, any corrections, all filters desactivated.
Imported both to FINAL CUT X and exported as h264 1920x1080
Facts:
- the more saturation and black levels in RESOLVE (appears to be sharpness).
- Magenta dots in RESOLVE (See the hair)
- blurry in ACR - maybe less black level
I really don´t know if is RESOLVE whocreates de magenta o RAWMAGIC
The video to download:
https://www.dropbox.com/s/vjrfigp0xi4t2pi/prueba%20RAW%20MAGIC%20vs%20RAW2DNG.mov
just a short sign of life of me. just got the flu (in summer, hate it).
brain apparatus is on 30%-can and -10%-want-strength :)
(*) batchmode is coded
(*) most code is rewritten
(*) no more 14bit-output, because not needed
(*) blacklevel and whitelevel read from raw-data
(*) colormatrix read from raw-data
(+) sitting on 12bit linear mode - after finishing that, exe will be published
by now
(-) no adjustments to new stripesize-changes (extremly urgent on list)
(-) no 10bit logarithmic (urgent on todolist) (thx iliasg, appreciate your work) (think, we have to hide the 14bit in 16bit-words - we know, resolve cant handle 14bit)
(-) banding issue for 5diii-sources (urgent) (thx to thomas worth)
(-) pink fringes (investigating) (its a canon-kind problem - adobe is handling that manually so to say)
(-) commandline-mode (in one of the next versions)
(-) compression (will come after 10bit logarithmic)
@zach - two days ago, a1ex changed the bytestripelength - did you used some "abnormal" picture-sizes? have to update my 5diii with the new code to investigate..
regards chmee
Hey Chmee,
I believe it must have something to do with what Alex did to the bytestripelength as I tried this further today on two other systems and they both came up with the same error. My only change in workflow was using the most recent version of Magic Lantern on my 5D Mark III instead of the late May version I was using before. All footage was shot at 1920 by 1080.
Sorry to hear about you getting the flu that is bad news! Feel better!
Quote from: chmee on June 10, 2013, 01:01:01 PM
(*) batchmode is coded
(*) most code is rewritten
(*) no more 14bit-output, because not needed
(*) blacklevel and whitelevel read from raw-data
(*) colormatrix read from raw-data
(+) sitting on 12bit linear mode - after finishing that, exe will be published
Great to hear you're making progress. Thank you for investing your time in this much needed tool. Now you have succeeded with the 14 to 12 bit conversion, I can't help my curiosity on the question of this process is too heavy to implement in camera or not(as discussed in the 12/10 bit conversion thread), but I guess it is premature to ask those questions. ;)
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
(http://dl.phreekz.de/raw2cdng/_raw2cdng_10.jpg)
version 1.0 is out - you know. its beta - and dont be too much optimistic :) the 12bit mode is built because of the easy way and a hope, that cutting the lsb 2 bits are not that visible :) the result is still far better than the "old" canon output.
@tin2tin:
the conversion from 14 to 12 bit is not that big work - but it needs some memory and some bitshifting and masking. my assumings are, if between sensor and dma-controller is no "processing-unit" able to handle that. hmm. the first approach was, to drive all the sensordata thru the digic - and this was the bottleneck.
regards chmee
Quotelower whitelevel by 15%
On what cameras is this needed? If raw2dng has pink highlights too, it's a bug in ML and you should report it.
so far i understand the pink highlights problem - its no problem of data handling but of managing the "beyond whitelevel sensels". the checkbutton is a fast built in tester if its changing something. if you search after "canon raw pink highlights" you (we) see, its a problem since years - manually managed by the raw-converters - and the dcraw-community is the only one trying to solve that by explaining the problem.
example links:
may 2012 - http://sourceforge.net/p/ufraw/discussion/434060/thread/b43582e3/
march 2009 - http://www.luminous-landscape.com/forum/index.php?topic=32942.0
feb 2010 - http://www.dpreview.com/forums/thread/2748822
oct 2011 - http://code.google.com/p/rawtherapee/issues/detail?id=1037
dec 2009 - http://www.flickr.com/groups/photomatix/discuss/72157623070526860/
regards chmee
In photo mode (CR2): white level depends on digital ISO, e.g. at 160, 320 and so on it's 1/3-stop lower than at full-stop ISO. Oddly enough, at ISO 250 it's not higher than at ISO 200.
In movie LiveView: digital ISO does not affect raw data, so the white level is always the same. The level from exif should be correct.
In photo LiveView, ML alters the white level to adjust the histogram so it matches the CR2 one. That's one of the reasons why you should not record raw video in photo mode.
Quote from: a1ex on June 11, 2013, 12:57:15 PM
.In photo LiveView, ML alters the white level to adjust the histogram so it matches the CR2 one. That's one of the reasons why you should not record raw video in photo mode.
So. meaning, in m and videomode it shouldnt happen?
M is photo mode, movie mode is movie mode. Don't mix them.
A quick test with a file from a 60D in Movie mode, a couple of ML builds back: http://we.tl/4Mv47vaiRS
There seems to be a bug in 12 bit + 15% unchecked which makes the screen black with green monocrome highlights.
the moviemode reacts on the mode-dial.. so, the only 100% functional way is to set on moviemode AND mode-dial on m.
now properly said?
@tin2tin investigating - made a dumb mistake (last minutes) lowering the blacklevel by 4 not by 2 should be the right way. on the way. thx for the raw.
Yes, on 5D3 you need to set both.
fixed. v1.1.1 is out. http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
just now playing with the newest build (http://www.magiclantern.fm/forum/index.php?topic=6362.0) to examine the stripByteCounts-Thing. fixed in 1.1.2
regards chmee
Quote from: chmee on June 11, 2013, 02:07:21 PM
fixed. v1.1.1 is out. http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
just now playing with the newest build (http://www.magiclantern.fm/forum/index.php?topic=6362.0) to examine the stripByteCounts-Thing. fixed in 1.1.2
regards chmee
THANKS! Chmee
to bad these files CinemaDNG don't open in Avid media composer
@jakobmen :o >:( is there any log or error-message? its my concern to see that playing in media composer as well - did you try the mac-version from thomas worth? is his cdng working?
handling bmcc cinema dng
QuoteCurrently, you cannot import the camera raw files directly into Avid Media Composer, Apple Color or Adobe Premiere Pro.
it cant handle bmcc-cdng-files? is that still so?
regards chmee
Quote from: chmee on June 11, 2013, 10:10:46 PM
@jakobmen :o >:( is there any log or error-message? its my concern to see that playing in media composer as well - did you try the mac-version from thomas worth? is his cdng working?
regards chmee
My bad bro im dumb! avid does not open CinemaDNG.. eheh
Quote from: chmee on June 11, 2013, 12:12:40 PM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
(http://dl.phreekz.de/raw2cdng/_raw2cdng_10.jpg)
version 1.0 is out - you know. its beta - and dont be too much optimistic :) the 12bit mode is built because of the easy way and a hope, that cutting the lsb 2 bits are not that visible :) the result is still far better than the "old" canon output.
@tin2tin:
the conversion from 14 to 12 bit is not that big work - but it needs some memory and some bitshifting and masking. my assumings are, if between sensor and dma-controller is no "processing-unit" able to handle that. hmm. the first approach was, to drive all the sensordata thru the digic - and this was the bottleneck.
regards chmee
chmee, thanks for the new version, glad you recovered fast. :)
12bit linear should be OK since there is more than 2bits noise as metered on Black Frames (>6 14bit raw levels). Theoretically, only ISO160 could suffer from this in a case of heavy processing.
But this truncation makes me nervous .. how did you managed to make it better than "old"Canon output ??. Waiting for tests ..
Any way, after you implement 10bit log we can expand using LUTs for 12 bit log ..
Quote from: chmee on June 11, 2013, 12:42:34 PM
so far i understand the pink highlights problem - its no problem of data handling but of managing the "beyond whitelevel sensels". the checkbutton is a fast built in tester if its changing something. if you search after "canon raw pink highlights" you (we) see, its a problem since years - manually managed by the raw-converters - and the dcraw-community is the only one trying to solve that by explaining the problem.
example links:
may 2012 - http://sourceforge.net/p/ufraw/discussion/434060/thread/b43582e3/
march 2009 - http://www.luminous-landscape.com/forum/index.php?topic=32942.0
feb 2010 - http://www.dpreview.com/forums/thread/2748822
oct 2011 - http://code.google.com/p/rawtherapee/issues/detail?id=1037
dec 2009 - http://www.flickr.com/groups/photomatix/discuss/72157623070526860/
regards chmee
In fact the problem comes from Dcraw which uses a single (many times wrong) value for White Level while WL floats depending on ISO settings (different for "integer" ISOs and "intermediate" ISOs) and on len's f-stop (there is digital amplification for larger than f/2.8 settings pushing the WP higher).
With DNGs the solution could be easy if ISO & f-stop was reported because there is an exif tag for WL, raw2dng/raw2cdng can take care of it.
For native raws (*.CR2) there is a need from the decoders side for a database as will be implemented by RT team .. http://code.google.com/p/rawtherapee/issues/detail?id=1752
I am sure Adobe uses a database also with rather conservative settings and as far as I know in DNG conversions they do not increase WL for open f-stops, not a big issue .. just a slight loss of highlights.
As it is now, ML uses the conservative settings that we see in Adobe's DNG conversions and I think they are safe, but we need a decoder to cooperate and maybe it's Resolve's error interpreting the exif tags wrongly .. like RT did a while ago https://code.google.com/p/rawtherapee/issues/detail?id=1695
Quote from: chmee on June 11, 2013, 10:10:46 PM
@jakobmen :o >:( is there any log or error-message? its my concern to see that playing in media composer as well - did you try the mac-version from thomas worth? is his cdng working?
handling bmcc cinema dngit cant handle bmcc-cdng-files? is that still so?
regards chmee
Chmee im trying to do this
raw2cdng to Resolve works great..
resolve to Avid media composer works great..
BUT when coming from Avid back to resolve its not linking right, its only linking the first clip in the edit and replace the other clips with that first clip..
could it be a file name problem or something in your code when making the CDNG files ?
here's what im trying to do
https://www.youtube.com/watch?feature=player_embedded&v=Kpg0k3aD088
english is not my first so forgive me..
chmee, incredible work. Jakobmen, I don't think its a file naming issue. I get a "does not support image sequence" error exporting an xml out of resolve. It doesn't do this with cinema dng's from the black magic camera.
I guess resolve 10 will be out soon, and looks like we'll have enough tools to cut in resolve, rather then round-tripping to premier. So raw2cdng might be the easiest workflow in the long run.
Same thing happened to me with the first clip linking to all in the resolve/avid workflow. Just found this which seems like a much more proper workflow then using proxies:
http://www.youtube.com/watch?v=8GWn3q03tqk
Worth a watch and I believe this will be better. Trying it tomorrow. Unless this method is what you already did?
Well I tried the above method I just posted numerous times as well and am having the same problem. All of the separate clips in the AAF only link to the first DNG file instead of the separate ones that were assigned. It works if I use the proxies but I can't get back to the DNG's. Anyone with a fix or is this in the code?
@iliasG "how did you managed to make it better than "old"Canon output ??"
umph.. maybe wrong said - old means the original-video-output via h.264-mov.
xml-problems - there are timecode and framerate tags for cdng - i do not use them - but maybe its worth to activate them :)
regards chmee
If there is a reel name tag for cdng too, I would suggest using the embedded file creation time as time code and the date as reel name as defaults(most softwares only read first 8 characters of reel name tags).
Quote from: chmee on June 12, 2013, 09:25:28 AM
@iliasG "how did you managed to make it better than "old"Canon output ??"
umph.. maybe wrong said - old means the original-video-output via h.264-mov.
xml-problems - there are timecode and framerate tags for cdng - i do not use them - but maybe its worth to activate them :)
regards chmee
thats what i was thinking.. Thank you again for your work you do..
@IliasG
Just to be on the right track.. Your ConversionLUT 10<->14bit you linked.
on left side is the lin->log, on the right side the inversion log->lin?!
Quote from: chmee on June 13, 2013, 10:51:02 AM
@IliasG
Just to be on the right track.. Your ConversionLUT 10<->14bit you linked.
on left side is the lin->log, on the right side the inversion log->lin?!
Exactly.
14bit linear (A) to 10bit log (H) is on the left.
Just fill in columnA the clipped values (<2017) all mapped to zero in columnH.
As you can see I forgot to adapt columnA to a variable Black Level .. just used the typical for 5DIII.
Normally I should start from the full 14bit range, clip 31 values lower than Black Level (which is variable) for columnB ...
10bit log (J) to 16bit linear (Q) is on the right. With red color (141) is the 16bit black Level.
It was very dirty calculation sorry.
when the children are sleeping i will try to realise the 10bit|log :)
(Reelname-Tag is built in as well as changed sequencenames- we'll test, if this lasts for solving the xml/edl-problems)
by the way - do you all know the hunterLUT for resolve? as seen in the vimeo-example (https://vimeo.com/67970827), its doin' a good job. (downloadlink is found in the vimeo-description)
regards chmee
Quote from: chmee on June 13, 2013, 07:59:51 PM
when the children are sleeping i will try to realise the 10bit|log :)
(Reelname-Tag is built in as well as changed sequencenames- we'll test, if this lasts for solving the xml/edl-problems)
by the way - do you all know the hunterLUT for resolve? as seen in the vimeo-example (https://vimeo.com/67970827), its doin' a good job. (downloadlink is found in the vimeo-description)
regards chmee
I don't think is matters but I use AAF
Thanks!
v1.1.6 is out - its the first attempt to fix the edl/xml-Problem - changing filename-structure and inserting Reelname-Tag into header.
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
regards chmee
Quote from: chmee on June 14, 2013, 01:56:00 PM
v1.1.6 is out - its the first attempt to fix the edl/xml-Problem - changing filename-structure and inserting Reelname-Tag into header.
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
regards chmee
Hm.... I try with edl, xml and aaf, but have same result... no changes. What names should have proxy files? In my case:
Proxy - M0000001.mov
CDNG - M0000001.000000.dng
same problem for me
only relinks first clip in the SEQ from Avid to resolve.
thanks
Quote from: Jakobmen on June 14, 2013, 03:10:32 PM
not working with AAF
My guess is that it might be helpful if you added how you test it.
please just link an edl or xml-example. right now i have no time to test myself -
but could it be, your workflow with proxies is not right? are you sure, you're doing it right?
why should resolve know, that m00001.mov picturesequence has to be replaced with m0001.[].dng?
is it described in the edl/xml?
regards chmee
Quote from: chmee on June 14, 2013, 04:24:43 PM
please just link an edl or xml-example. right now i have no time to test myself -
but could it be, your workflow with proxies is not right? are you sure, you're doing it right?
why should resolve know, that m00001.mov picturesequence has to be replaced with m0001.[].dng?
is it described in the edl/xml?
regards chmee
EDL https://docs.google.com/file/d/0BxDyTWKlBKcWd1k5ekNCOWJtTzA/edit?usp=sharing
XML https://docs.google.com/file/d/0BxDyTWKlBKcWcmlRbjYtOTA3M0E/edit?usp=sharing
So I wanted to ask - how proxy should be named, that they are properly conform in Resolve? Because proxy is a single video file (.mov or mp4) and original is dng sequence... I understand that Resolve don't now about it, but how do this?
Hi all,i have try last version of raw2cdng.1.1.6 and when import sequence in resolve i see hot pixel why? i have canon60d with tecnicolor cinestyle..in camera raw hot pixel not are present in davinci resolve yes...what is wrong? HERE A PIC
----> (http://i40.tinypic.com/2qisigz.jpg) HELP PLEASE,THANK U ALL!
Quote from: chmee on June 14, 2013, 04:24:43 PM
why should resolve know, that m00001.mov picturesequence has to be replaced with m0001.[].dng?
is it described in the edl/xml?
regards chmee
yes edl/aaf/xml..
i will send my AAF file once i get home
That works Chmee!! This is awesome! Here is the workflow:
http://www.hurlbutvisuals.com/blog/2013/04/blackmagic-camera-post-workflow/
@jakobmen
i looked into the linked xml/edl's. i do not find any hints of a picturesequence. masterclip and proxyclip have the same name. So, sorry i play naive - in this problem-case i have to believe in your thoughts - likewise i try to minimise my work on that. :)
@iunknown
is this checkbutton the solution?
Quotewhile re importing edl/xml deselect "Automatically import source clips into media pool."
@keyth
its a bad idea to compare with adobe-products. they have just a little bit too much knowledge managing and fixing "problems". hotpixels are tagged and marked in CR2 and DNG - adobes RAW-Engine is able to detect hotpixels and masks them out. are these hotpixels static? i think so.. (but yes of course, it could be another problem - look into the cr2/dng/cdng with rawjuggler or dcraw)
regards chmee
Quote from: iunknown on June 14, 2013, 08:42:14 PM
That works Chmee!! This is awesome! Here is the workflow:
http://www.hurlbutvisuals.com/blog/2013/04/blackmagic-camera-post-workflow/
HOW??? You work with RAW from Canon Mark? I try this, but at last step Resolve conform xml with wrong content. Cuts right. Please, tell, how?? Maybe some screenshots and xml? I'm trying to solve this problem for several days...
I also cannot seem to get the relinking to work. Many hours trying different methods but no success. I wonder if this is a case where something which worked in Final Cut does not work in AVID which I am using? Confused!
I've got a bunch of DNG files I want to proces in Resolve, but I already deleted the .raw files... Any chance I can put those DNG's back into a .raw or convert them straight to cinema DNG? tips?
artyg/zach915m,
Where exactly does it get tripped up?
workflow:
import all the files into resolve
export proxies
create timeline in premier, edit and export xml
import xml into resolve
deselecting "Automatically import source clips into media pool."
Make sure you save the original project in resolve so that all the original clips are in the media pool.
Here's the workflows I've tried both of them getting tripped up at the same point:
Both of these are CDNG to Resolve to AVID back to Resolve
1. Use CDNG to make image sequences
2. Import into resolve
3. Export using either Avid AAF round trip or just using a regular export to DNX 36 8bit files.
4. Edit in Avid
5. Export AAF
6. Open AAF in Resolve, un-clicking "auto import source clips" I have also tried clicking "ignore file extensions" and using EDL to go back to resolve and the same thing happens every time which is that it links to the first DNG in my "master" bin and uses that clip with all the cut points that are given.
The issue is not that it doesn't recognize the cut points, it's that it doesn't link back to the correct DNG files and nothing seems to remedy this including "reconform from bins." I do hope this is my error and not an issue with the CDNG! Sorry for making a ruckus! It also seems that if others are having success with Premiere and FCP that it could be an avid issue.
-Zach
Quote from: zach915m on June 16, 2013, 05:58:55 AM
Here's the workflows I've tried both of them getting tripped up at the same point:
Both of these are CDNG to Resolve to AVID back to Resolve
1. Use CDNG to make image sequences
2. Import into resolve
3. Export using either Avid AAF round trip or just using a regular export to DNX 36 8bit files.
4. Edit in Avid
5. Export AAF
6. Open AAF in Resolve, un-clicking "auto import source clips" I have also tried clicking "ignore file extensions" and using EDL to go back to resolve and the same thing happens every time which is that it links to the first DNG in my "master" bin and uses that clip with all the cut points that are given.
The issue is not that it doesn't recognize the cut points, it's that it doesn't link back to the correct DNG files and nothing seems to remedy this including "reconform from bins." I do hope this is my error and not an issue with the CDNG! Sorry for making a ruckus! It also seems that if others are having success with Premiere and FCP that it could be an avid issue.
-Zach
This is the same for me...
DNG header need a badpixel_opcode field like files generated by raw2dng. ACR fixes bad/hot pixels automatically with badpixel_opcode. I don't know how it works in Davinci Resolve ..
https://bitbucket.org/hudson/magic-lantern/src/b93d6ae2b30dc6581efeb27d2e4571c6fd92ab6e/src/chdk-dng.c?at=unified
Quote from: keyth on June 14, 2013, 07:34:43 PM
Hi all,i have try last version of raw2cdng.1.1.6 and when import sequence in resolve i see hot pixel why? i have canon60d with tecnicolor cinestyle..in camera raw hot pixel not are present in davinci resolve yes...what is wrong? HERE A PIC
Quote from: zach915m on June 16, 2013, 05:58:55 AM
Here's the workflows I've tried both of them getting tripped up at the same point:
Both of these are CDNG to Resolve to AVID back to Resolve
1. Use CDNG to make image sequences
2. Import into resolve
3. Export using either Avid AAF round trip or just using a regular export to DNX 36 8bit files.
4. Edit in Avid
5. Export AAF
6. Open AAF in Resolve, un-clicking "auto import source clips" I have also tried clicking "ignore file extensions" and using EDL to go back to resolve and the same thing happens every time which is that it links to the first DNG in my "master" bin and uses that clip with all the cut points that are given.
The issue is not that it doesn't recognize the cut points, it's that it doesn't link back to the correct DNG files and nothing seems to remedy this including "reconform from bins." I do hope this is my error and not an issue with the CDNG! Sorry for making a ruckus! It also seems that if others are having success with Premiere and FCP that it could be an avid issue.
-Zach
No, it's not avid problem - i work with Premiere Pro CS6 and export xml and have the same problem.
iunknown,
Quote from: iunknown on June 16, 2013, 01:16:27 AM
artyg/zach915m,
Where exactly does it get tripped up?
workflow:
import all the files into resolve
export proxies
create timeline in premier, edit and export xml
import xml into resolve
deselecting "Automatically import source clips into media pool."
Make sure you save the original project in resolve so that all the original clips are in the media pool.
Can you show your screenshots of folders with original dng files and proxy files. And send your xml file? Becouse we do the same as you, but... strange things happen =(
Here my workflow screenshots
(http://i.pixs.ru/thumbs/5/6/7/Snimok1PNG_2014616_8223567.jpg) (http://pixs.ru/showimage/Snimok1PNG_2014616_8223567.png) (http://i5.pixs.ru/thumbs/5/7/6/Snimok2PNG_9422966_8223576.jpg) (http://pixs.ru/showimage/Snimok2PNG_9422966_8223576.png) (http://i.pixs.ru/thumbs/5/8/1/Snimok3PNG_7362325_8223581.jpg) (http://pixs.ru/showimage/Snimok3PNG_7362325_8223581.png) (http://i5.pixs.ru/thumbs/5/8/5/Snimok4PNG_3585798_8223585.jpg) (http://pixs.ru/showimage/Snimok4PNG_3585798_8223585.png) (http://i5.pixs.ru/thumbs/5/9/5/Snimok5PNG_9067860_8223595.jpg) (http://pixs.ru/showimage/Snimok5PNG_9067860_8223595.png) (http://i5.pixs.ru/thumbs/5/9/8/Snimok6PNG_8969078_8223598.jpg) (http://pixs.ru/showimage/Snimok6PNG_8969078_8223598.png)
Hello chmee.
Thank you for your great work!
I am developing BATCHelor and awaiting eagerly your implementation of commandline. :)
I would like to know if it is possible to not have to join .RAW and .R00 , .R01,... split files beforehand, as this is tedious.
Can you implement the extraction of CDNG from split files natively? :)
@marekk since when is the badpixel thing implemented? i have to look to newest raw-files.. :) thx.
@keyth maybe it helps if you use the inbuilt hotpixel-remapping/cleaning?!
http://www.youtube.com/watch?v=zZLTdVMCwMM
@fatpig the command line is on the todo-list - please be patient. and yes, of course the split-files will be handled as one in my converter - so you have only to give the .raw-file and it will find the rest, if there..
Roundtrip Ideas - im not at home (for next 6 days) so i cant try that - are you able to rename the footage in resolve? as with the last version i tried to differentiate the name from the framecount with the dot - maybe it was wrong.
original - M3115.[00001-00250].dng (it looks like resolve thinks, "M3115." is the name, but we need "M3115")
proxy - M3115.mov
the next attempt will be of course creating timecode into the dng-header.
bytheway - do i see that right? raw2dng is now converting to 16bit? are you able to use these files in resolve? if yes, i would turn my application into a bitrate-converter - because all benefits (badpixels, verticalbanding) are built in..
v1.1.7 is downloadable here (http://212.227.97.228/DSLR_KLEINANZEIGEN/Download/raw2cdng.1.1.7.zip). its a testversion for figuring out about the resolve-roundtripping.
(*) deleted this dot in filename between name and framecount
(*) fixed filename to 8 chars, more will be truncated, less will be filled up with random chars.
regards chmee
Quote from: artyg on June 16, 2013, 05:48:14 PM
No, it's not avid problem - i work with Premiere Pro CS6 and export xml and have the same problem.
(http://i5.pixs.ru/thumbs/5/9/8/Snimok6PNG_8969078_8223598.jpg) (http://pixs.ru/showimage/Snimok6PNG_8969078_8223598.png)
Me too, I'm having the same issue with Premiere Pro CS6. When importing the XML back into Resolve, it has the same source clip repeated. I suspect a Resolve bug. Without being an expert, I looked at the Premiere XML output and it looked fine.
What I don't get about the process is how Resolve knows how to link the proxies back to the original source media. Could it be stored in it's project somehow?
peoplemerge, artyg,
Yeah, your right, its not relinking the files correctly when importing the xml back into resolve.
Chmee,
Just tried it with version 1.1.7, and same thing. I have no idea how resolve knows to point back to the original raw files with the black magic footage (when selecting "Automatically import source clips into media pool." )
Here is the naming for BMCC footage raw folders:
Greg Fiske_1_2013-02-15_1422_C0000
and the default name resolve gives to proxies:
V1-0001_Greg Fiske_1_2013-02-15_1422_C0000_1241904
cinemadng raw folder:
CDNG16_M091158U
and default name resolve gives to proxies:
V1-0002_M091158U000000
Chmee,
What is the variable "_1241904" added to the end of the proxie names on the bmcc files? It is the one thing that is different with the two xml files. The cinemadng comes up as zero, but the cinemadng xml has this:
<frame>1241904</frame>
Here are the xml files:
bmcc xml
https://docs.google.com/file/d/0B-cS4Mdd09ShYkd0YXg5c2dTT0k/edit?usp=sharing
cinemadng xml
https://docs.google.com/file/d/0B-cS4Mdd09ShR3JTTzVJMFNvVXc/edit?usp=sharing
<string>14:22:26:00</string>
<frame>1241904</frame>
seems to be sort of timestamp - with frames as smallest dividor.
1241904 / 24 / 60 /60 = 14 ..... [hour]
modulo (1241904-1209600) = 32304
32304 /24/60 = 22.... [minute]
modulo (32304-31680) = 624
624/24 = 26 [second]
So, it seems, i have to set the timecodes and fps-tag. i hope, this is the final and functional key.
regards chmee
I'm getting weird results in Resolve when loading CinemaDNG's converted with raw2cdng.1.1.6. I'm shooting with the Canon 50D.
If I open the frames in ACR, they look fine. In Resolve there is a faint blue/orange coloured line inside of the frame.
See screenshot here: http://www.magiclantern.fm/forum/index.php?topic=5586.msg52177#msg52177
Could this be caused by raw2cdng? I don't think so since the DNG's look fine in ACR. Could it be a 50D problem? Anyone seen this and knows a fix for Resolve?
maybe you can make blacklevel editable somehow, because some of my 5DII Videos, which should have 1024 - have a wrong blacklevel of 2037 set inside the .RAW and look really bad.
I would like to set it to 1024 again.
gave you an answer in another thread - im not sure if an override-values field is a good idea
..finally it is (as bonus), but by now the important thing is the roundtrip-functionality with other editors.
regards chmee
+1 on the roundtrip! I have a fun story to tell. Go chmee!
Dudes - what raw space should we be working in inside of Resolve with ML RAW? As in what RAW settings are best to use within Resolve? Is it best to decode with CinemaDNG default or with Clip and Rec.709 (or is does this default to the same anyway?).
Is it best to Decode with "Clip" and About to have a play! Sooooooo Excited! (again - can't thank you enough for this!)
Quote from: noisyboy on June 19, 2013, 02:31:57 AM
Dudes - what raw space should we be working in inside of Resolve with ML RAW? As in what RAW settings are best to use within Resolve? Is it best to decode with CinemaDNG default or with Clip and Rec.709 (or is does this default to the same anyway?).
Is it best to Decode with "Clip" and About to have a play! Sooooooo Excited! (again - can't thank you enough for this!)
I find it's best to decode with Clip and film log. Then you can interpret things on the fly with an awesome LUT like one of these:
http://juanmelara.com.au/print-film-emulation-luts-for-download/
nightvision04 is absolutely right. Set it to "Clip" "As Shot" "BMD Film". I'm grading an hour of 5D2 raw footage without LUTs learning resolve lite as I go and it looks very filmic. rec. 709 will show pink fringing and even more moire. The BMD film setting is a vast improvement.
Quote from: peoplemerge on June 19, 2013, 03:01:50 AM
The BMD film setting is a vast improvement.
Thanks dudes! 8)
Check out those LUTS! Big films are known to use Print LUTs to emulate film colors, and Juan is a best kept secret on the internet for high quality LUTS. They are free, and he has a great tutorial to help you install them.
Quote from: chmee on June 17, 2013, 01:24:43 PM
@marekk since when is the badpixel thing implemented? i have to look to newest raw-files.. :) thx.
It's not in raw file. There is a special badpixel_opcode in DNG tags. Look at chdk-dng.c file. With this tag dead/hot pixels can be removed automatically for example by Adobe Camera Raw.
Quote from: peoplemerge on June 19, 2013, 03:01:50 AM
nightvision04 is absolutely right. Set it to "Clip" "As Shot" "BMD Film". I'm grading an hour of 5D2 raw footage without LUTs learning resolve lite as I go and it looks very filmic. rec. 709 will show pink fringing and even more moire. The BMD film setting is a vast improvement.
Where this settings must be set?
@marekk it seems its just a flag for allowing repairing "bad" pixels - described with a number 0-4 - hmm. ok. will put it on toDo-list. but we have to check, if resolve/speedgrade is able to repair badpixels - so inbody masking is imho the better solution)
here is a version 1.1.8 with reelname, timecode and framerate in header - maybe it helps. the framerate is static on 24fps - because we're in testing mode - if it works i will change to variable framerates and timecode (if needed, starttime of filegenerate-datetime)
Download v1.1.8 here (http://dslr-kleinanzeigen.de/Download/raw2cdng.1.1.8.zip)
regards chmee
ToDo-List 20.06.2013
(*) Roundtrip must work
(*) IliasG's idea of 10bit log (state 90%)
(*) strippedFiles
(*) Repair-Flag für badpixels? (simpler workaround: demasking in body)
(*) commandline
Quote from: artyg on June 19, 2013, 12:21:09 PM
Where this settings must be set?
Color pane, bottom left panel there's 4 icons, a circle, bar chart, 3 circles in a triangle, and a movie cam. The last one, the movie cam, is for "Camera Raw". below that is Master Settings pane. Set the Decode Using to "Clip" etc as discussed above
@chmee Great to see you're still going strong! I haven't got resolve on this computer, so I can't test atm. Let's hope the mediainfo guys will add info subraction of cdng files.
Quote from: chmee on June 19, 2013, 09:00:27 PM
@marekk it seems its just a flag for allowing repairing "bad" pixels - described with a number 0-4 - hmm. ok. will put it on toDo-list. but we have to check, if resolve/speedgrade is able to repair badpixels - so inbody masking is imho the better solution)
here is a version 1.1.8 with reelname, timecode and framerate in header - maybe it helps. the framerate is static on 24fps - because we're in testing mode - if it works i will change to variable framerates and timecode (if needed, starttime of filegenerate-datetime)
Download v1.1.8 here (http://dslr-kleinanzeigen.de/Download/raw2cdng.1.1.8.zip)
regards chmee
ToDo-List 20.06.2013
(*) Roundtrip must work
(*) IliasG's idea of 10bit log (state 90%)
(*) strippedFiles
(*) Repair-Flag für badpixels? (simpler workaround: demasking in body)
(*) commandline
still not work about timecode problem
here is the xml file please have a look
http://www.istorm2010.com/temp/A-Resolve-out.xml (http://www.istorm2010.com/temp/A-Resolve-out.xml)
http://www.istorm2010.com/temp/B-premiere out(Importing A).xml (http://www.istorm2010.com/temp/B-premiere%20out(Importing%20A).xml)
http://www.istorm2010.com/temp/C-Resolve out again (impotrting B).xml (http://www.istorm2010.com/temp/C-Resolve%20out%20again%20(impotrting%20B).xml)
(http://www.istorm2010.com/temp/C-Resolve%20out%20again.jpg)
Yeah, tested, same issue, duplicate clips on the xml import into resolve. Thanks for the effort chmee.
I was wondering if anyone has had luck roundtriping from Resolve with the "AAF Roundtrip" function and using AVID to edit. I used RAWMagic to decode the RAW to CDNG, then followed the steps in this video: (http://youtu.be/8GWn3q03tqk)
This worked perfectly for me! I edited a one minute short with multiple clips and it went back and forth between the programs without a hitch. Anyone else had luck trying this method?
I'm on a PC so I can't test RawMagic. But I did use the AAF roundtrip to test this and it didnt work...=(
OpcodeLists are allowed in CinemaDNG file format so maybe Davinci Resolve support this feature..
Quote from: chmee on June 19, 2013, 09:00:27 PM
@marekk it seems its just a flag for allowing repairing "bad" pixels - described with a number 0-4 - hmm. ok. will put it on toDo-list. but we have to check, if resolve/speedgrade is able to repair badpixels - so inbody masking is imho the better solution)
"maybe" is the right description. As of version 1.1.0.0 of the CDNG-specification opcodes are optional by decoders, not mandatory. i put it on toDo-list, but there are some fixes before that..
by now i'm out for the next three/four days - because i cant solve the roundtrip-problem here on my netbook - when i'm back home i will analyse differences between (functional) bmcc-roundtrips and our stuff.
regards chmee
@nowhereman86 this is a substantially different workflow from the final cut XML. Interesting! My resolve with fcp dailies crashes trying to export AAF with some of my existing projects but I'll try with smaller ones and the Avid MXF dailies.
right before implementing the 10Bit-log-Curve i'm in doubt, if some bodies have a blacklevel lower than 2017..?!
Someone any info for me?
Yes, 1024, 1500...
It's computed on the fly; I have no idea what the absolute range is.
FYI: http://lclevy.free.fr/cr2/#wb
so it seems i have to compute the rec709-log on the fly also. ok. thanks.
regards chmee
@nowhereman86 Same problem with importing/exporting Premiere Pro, which doesn't work with MXF files. Just noticed Adobe has updates for all products, Premiere CC, which does support DNxHD MXF files! However with both versions, the relinking still fails using both aaf and xml exports. You might not notice if you only have 1-2 clips. Can you confirm this works in Avid on a project with 7-10 clips? Are you able to export xml from avid, and if so does the workflow still work?
Quote from: chmee on June 20, 2013, 09:16:54 PM
right before implementing the 10Bit-log-Curve i'm in doubt, if some bodies have a blacklevel lower than 2017..?!
Someone any info for me?
Yes, models older than 7D (5DII, 50D, 40D) have BL at around 1024. This in photo-raws, video-raws can be a bit different.
In any case for a correct calculation we need the full range limits
Except BL which is computed in camera by ML and accessible in metadata, there is the White Level which is set at a predefined value depending on model and based on photo-raw WL as used in CR2 to DNG conversions. This is not accurate enough. For example metadata for 5DIII say WL=15000 while histograms show 16382 ..
I think that samples for every model with burned areas are necessary to declare the WL based on real video-raw data ..
Quote from: a1ex on June 20, 2013, 09:37:47 PM
Yes, 1024, 1500...
It's computed on the fly; I have no idea what the absolute range is.
FYI: http://lclevy.free.fr/cr2/#wb
I didn't knew a case of BL at 1500 !!.
Alex, as I saw on your 5DIII black frames the BL calculated by ML is truncated to the next lower integer so for values 2047.8-2047.9 it becomes 2047. Not a serious error for 14bit DNG, but after truncating to 12bit (as raw2cdng does for the moment) it can be a bit harmful ..
Quote from: chmee on June 20, 2013, 10:41:01 PM
so it seems i have to compute the rec709-log on the fly also. ok. thanks.
regards chmee
Or use predefined "caseA", caseB" ..
A small detail which might help if searching for "perfection" ..
While I started with BL-32, I used BL-31 to keep the BL=2048 at the center when mapping to 10bit (2047, 2048, 2049 all mapped to 10) so as to have the smallest error. With radically different range (WL-BL) this could change.
@IliasG as the log-curve goes to full 14bit (16384) and the highlevel end is quite flat, i dont think a whitelevel-difference f.e. 15.000 or 13.500 is any significant problem. now thinking about two log-lists (one with ~2017 start and another one with ~1010) or really computing on the fly..
(like the caseA/caseB-thing much more :) )
regards chmee
I don't know how to test it now because Davinci Resolve doesn't open 14bit DNGs.. So I need a 12 or 16bit DNG with hot pixels and badpixel_opcode..
Quote from: chmee on June 20, 2013, 12:37:46 PM
"maybe" is the right description. As of version 1.1.0.0 of the CDNG-specification opcodes are optional by decoders, not mandatory. i put it on toDo-list, but there are some fixes before that..
hello, have tried your program in wine, under 1.5.31 with mono, it starts up perfectly, but the drag&drop seems to be disfuct. Really looking forward to the commandline version then.
@revast minimum requirement is .net4 compatibility.
http://appdb.winehq.org/objectManager.php?sClass=version&iId=17886
regards chmee
@peoplemere Yeah, this workflow seemed to move along just fine with 10 clips in a thing I put together. I didn't use XMl though I used exclusively the AAF Roundtrip function and followed the steps in this video. I think the order in which you execute these steps is very critical since. If you have Avid and resolve, try it out and see if it works okay.
http://youtu.be/8GWn3q03tqk
Yeah that's the video I've been using too but the roundtrip doesn't work for me... :'(
are you aware that the progress bars lock after processing about 1/4th of the frames?
it still processes all of them, but the GUI greys out until its done if I click inside.
Quote from: fatpig on June 23, 2013, 11:30:36 AM
are you aware that the progress bars lock after processing about 1/4th of the frames?
it still processes all of them, but the GUI greys out until its done if I click inside.
Same here (windows 7).
OK I figured out the round trip for CinemaDNG raw, Premiere CC, Da Vinci Resolve Light! The trick is to get the reel names consistent, I'm using the source clip path, by jumping through some hoops to rename the directories. Details here: http://www.youtube.com/watch?v=AJzgpIdoUqw
Quote from: peoplemerge on June 24, 2013, 05:56:44 AM
OK I figured out the round trip for CinemaDNG raw, Premiere CC, Da Vinci Resolve Light! The trick is to get the real names consistent, I'm using the source clip path, by jumping through some hoops to rename the directories. Details here: http://www.youtube.com/watch?v=AJzgpIdoUqw
Excellent! Very useful tutorial. Thanks :)
@peoplemerge great. nice finding. will try that. YOURE THE STAR OF THE DAY! workin! the important thing is the creation of the ReelNo in Resolve.
@fatpig thx. seems, i have to refresh the gui from time to time.
Great find peoplemerge. Thanks so much for all the hard work you guys!
For everyone else, discription of the problem (reelno variable missing) at timestamp 7:11.
Quote from: peoplemerge on June 24, 2013, 05:56:44 AM
OK I figured out the round trip for CinemaDNG raw, Premiere CC, Da Vinci Resolve Light! The trick is to get the reel names consistent, I'm using the source clip path, by jumping through some hoops to rename the directories. Details here: http://www.youtube.com/watch?v=AJzgpIdoUqw
anyone try this on windows yet ?
trying to get home to try it lol ;D
I'm on windows 8 and I can't use drag/drop at all it seems like
does anyone have a solution?
Works on windows 7.
@swaneon is drag and drop a general problem on your computer/win8? maybe thats a solution?
http://www.c-sharpcorner.com/UploadFile/nipuntomar/drag-drop-not-working-in-windows-8/
@chmee:
is the roundtrip something that can be solved in future versions of raw2cdng or do we have to follow these steps every time from now on?
(at work, cant watch right now)
hmm. i wrote in another thread, i tried a simple xml-roundtrip with blackmagic-CC-Files - it failed the same way as with canon-cdng's. the bmcc-files also had no ReelNo - so, my assumption is, if filename/path/reelName isn't 100% in right order (for resolve) it fails.. i built it in reelname, framerate and timecode and it didnt helped. by now the oneclick in the settings for the reelNo-generation is a no-brainer. for myself i will leave this roundtrip-salvation and make it a sidejob, because its too easy to solve..
todolist 25.06.2013
(*) 10bit log (nearly ready, maybe tonight for first testing)
(*) splitted files support (try to solve it today as well)
(*) vertical banding
(*) compression
(*) command line
(*) threading for multi-cpu-support
(-) dragndrop problems in win8 (or leaving till command line is working)
regards chmee
Quote from: chmee on June 25, 2013, 06:26:43 PM
the bmcc-files also had no ReelNo - so, my assumption is, if filename/path/reelName isn't 100% in right order (for resolve) it fails.. i built it in reelname, framerate and timecode and it didnt helped. by now the oneclick in the settings for the reelNo-generation is a no-brainer. for myself i will leave this roundtrip-salvation and make it a sidejob, because its too easy to solve..
Is it true that you can add the reel number into cdng metadata? Otherwise creating distinct folders would take out half of the pain of this process.
Just throwing out ideas, but you might leave it up to the user to multi select [and append the name]:
[ ] use the raw file sequence (eg M0000000) - though this is not sufficient since we can expect the user to want to use clips from multiple cards and they reset to M0000000 when you power down ML.
[ ] use some string
[ ] use the file creation time of the rawfile
[ ] use the current time
Quote from: chmee on June 25, 2013, 06:26:43 PM
(*) vertical banding
(*) compression
(*) command line
(*) threading for multi-cpu-support
Vertical banding ++ !!!
What do you have in mind for compression?
On 60D still not working 100% :( I hope you can include bad pixel opticode...in my case cdng files after conversion have black and hot pixel :(
@peoplemerge since v1.1.8 there is reelname, framerate and timecode written in the cdng-header - but reelname is not shown in resolve, like the bmcc-files as well. In consistency with edl CMX3600 the reelname is only 8 uppercase-chars or numbers long - with v1.2.0 the path will be named with a timedate (dont know if timestamp or humanreadable ymdhm) before the reelname. the file will be reelname.sequencenumber.dng - simply. i decided this way, because the roundtrip-solution is so simple, i care more of readable file-structure than of (possible) reelno-understanding by resolve - as i said the bmcc werent recognised as well (link to the thread (http://www.magiclantern.fm/forum/index.php?topic=6271.msg53796#msg53796))
btw - m00000.raw is no filenaming-convention anymore - with the newer versions of rec_raw its written
mXX-YYYY.raw - f.e. M35-6148.raw
@sarotaz there is no guarantee, its working, but i will give it a try. badpixel-operations and recognition are no must of cdng-decoders.
regards chmee
Quote from: chmee on June 25, 2013, 07:47:45 AM
@swaneon is drag and drop a general problem on your computer/win8? maybe thats a solution?
http://www.c-sharpcorner.com/UploadFile/nipuntomar/drag-drop-not-working-in-windows-8/
I tried that and the interesting thing is that every other program is fine with Drag + Drop—only raw2cdng does not work
@swaneon thx for info - will figure that out - but have no win8 here ;)
tried to run the app as administrator? some coder write, its about UAC..
regards chmee
great todo-list! Exactly what I hoped for. :)
As for the drag n drop,
when commandline is implemented, people can use different GUIs like RAWanizer and BATCHelor.
I am looking forward to input and output path parameters and compression.
I can imagine that we would use our CF card as input on set and just save the (compressed?) DNGs to the laptops HDD as output, saving time and space. :)
got drag and drop to work
downloaded and tried 1.1.2 and it worked
then tried 1.1.6 and that worked
then retried 1.1.8 and it works!
Quote from: chmee on June 25, 2013, 08:02:33 PM
btw - m00000.raw is no filenaming-convention anymore - with the newer versions of rec_raw its written
mXX-YYYY.raw - f.e. M35-6148.raw
the naming convention for raw files currently is:
M for magic lantern
dd for 2 digit day of the month
hhmm for 2 digit hour and 2 digit minute when recording began.
(the last two digits get incremented if you've already recorded videos in the same minute up to 99 (if anyone could click that fast))
so every file name is unique.
so your example would never be generated by magic lantern.
M01-M31 first part range
0000 - 2399 second part range (though seldom will there be values greater than 2359)
it may be helpful to preserve these dates and times in yours and other post-processing tools.
(i was looking at this in case there was a need to insert a timecode into the cinemaDNG exif data)
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
raw2cdng v1.1.9 is online
- 10bit log is in (but experimental) - no linearizationtable (thx IliasG)
- naming conventions slightly changed
- fixed freezing gui
(!) 10bit log only works for bodies with blacklevel higher 2000 (!) next patch will have a lut for blacklevels around 1024 (!)
@IliasG
is your excel-calc that stable to calculate this lut by changing start values?
(lol. short test in resolve - 10bit/log is not looking that bad :) and hotpixels are gone in this one sample i have :D - if the red tint is too much, lower red gain to ~0.86 - but now its time to test the ability to grade and work with them - we squeezed all information into the 10bit - as a side advantage/drawback theres nearly no more data in blownout or dark parts of the picture - i dont really know if 10bit/log is a good grading base - possibly good for directly working in premiere/fcp/avid, after transcoding into a useable codec)
@gregoryOfManhattan
ah ok. so, my example was wrong - basically it was just as a information about the changed naming of raw-files. finally i leave this numbers in the [reelname] - additionally i put timedate of creation into the pathname.
@swaneon
good news :)
urgent on todoList for next days
- support of splitted files
- 10bit|log for blacklevel ~1024-bodies.
regards chmee
Quote from: chmee on June 27, 2013, 08:28:19 AM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
raw2cdng v1.1.9 is online
- 10bit log is in (but experimental) - no linearizationtable (thx IliasG)
- naming conventions slightly changed
- fixed freezing gui
(!) 10bit log only works for bodies with blacklevel higher 2000 (!) next patch will have a lut for blacklevels around 1024 (!)
@IliasG
is your excel-calc that stable to calculate this lut by changing start values?
(lol. short test in resolve - 10bit/log is not looking that bad :) and hotpixels are gone in this one sample i have :D - if the red tint is too much, lower red gain to ~0.86 - but now its time to test the ability to grade and work with them - we squeezed all information into the 10bit - as a side advantage/drawback theres nearly no more data in blownout or dark parts of the picture - i dont really know if 10bit/log is a good grading base - possibly good for directly working in premiere/fcp/avid, after transcoding into a useable codec)
@gregoryOfManhattan
ah ok. so, my example was wrong - basically it was just as a information about the changed naming of raw-files. finally i leave this numbers in the [reelname] - additionally i put timedate of creation into the pathname.
@swaneon
good news :)
urgent on todoList for next days
- support of splitted files
- 10bit|log for blacklevel ~1024-bodies.
regards chmee
This version killed drag and drop for me. It still works with previous versions though. It's only this program.
@iaremrsir
under what OS? are some other users with the same problem?
did you rename your files? is it showing the dragndrop-icon (arrow with plus) on drag-action?
regards chmee
First time I ran the 1.1.9 version on Win 8 d'n d didn't work. Second time it did work.
I just tried your converter (1.1.9) for the first time, and it's working very slow for me (like one frame every two or more seconds) but I have 0% CPU usage, am I doing something wrong? Maybe I have an old verstion of raw2dng, I don't know.
Ok I was doing it wrong, just drag and drop the RAW file, not the dng sequence, thanks for the converter!
Quote from: chmee on June 28, 2013, 08:27:42 AM
@iaremrsir
under what OS? are some other users with the same problem?
did you rename your files? is it showing the dragndrop-icon (arrow with plus) on drag-action?
regards chmee
Windows 7 x64. I dragged the files straight from the card without altering them. It shows the + sign cursor, but after dropping nothing happens. The clips don't show in the queue.
@iaremrsir the decoded dngs will be saved on the location, where the raw-files are - so its no good idea to drag them straight from the card into the app. did you tested with a copy from a hdd?
regards chmee
Quote from: chmee on June 30, 2013, 07:00:24 AM
@iaremrsir the decoded dngs will be saved on the location, where the raw-files are - so its no good idea to drag them straight from the card into the app. did you tested with a copy from a hdd?
regards chmee
I tried both ways, neither worked. I also tried turning off my security software to see if it might have been blocking functionality of the program because its happened before. But even then it still didn't work.
what camera, what version (date) of your raw-rec-modul? could you upload a short (and small) example file (~100mb), so that i can try it on my computer? it seems, something's different in your files..
!FOUND! something with the filenames/analyzing is wrong - if you rename them, they will be imported, but they dont work - bug, solved in the coming version.
v1.2.0 is out - splitted Files support
(*) you just have to drag the first (.RAW) file - app searches for split-files..
(*) i had to rewrite the bytestream-reader - so please tell, if there are bugs in "normal" non-splitted dng-rewrite-mode..
(*) filename bug fixed
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
(please test and give me some info if workin & bugs)
regards chmee
Windows 7 Pro x64 / NET 4 standalone directly from Microsoft installed.
Drag & Drop doesn't work for me. It just refuses to eat my 5D2 raw. Is there other way to supply the files? Simple [Open] maybe?
Thnx for your work.
P.S. WinXP SP3 works just fine.
you've got 1.2.0?
is the drag-icon showing up? (throwing into listview!)
starting as administrator?
thread about dragndrop-problems in win7 -> http://answers.microsoft.com/en-us/windows/forum/windows_7-performance/drag-and-drop-not-working-in-windows-7/f6dcfa11-8c51-e011-8dfc-68b599b31bf5
Sure, 1.2.0
Got it. its lovely windows UAC. Once disabled, Drag & Drop works. But ... I dont want it to be off permanently )
thanks for this great utility!
sorry i'm lost with this: how do i get the 5D2 specific raw2dng (5D2_WIN_raw2dng) info to work with raw2cdng?
raw2cdng is showing 5D2 in the batchlist but the DNG reads as Canon EOS PHR in ACR with embedded profile.
is only meant to work with Resolve?
thanks again!
@Yaros is it working, if you start it as administrator (without disabling UAC)? - you can change your rights on this application as its starting every time with admin-rights - rightclick -> properties -> compatibility - check "run this program as an administrator"
@blackroom i read it the first time, there is a specific 5d2-version of raw2dng? if i know, what changes there are, i can give you an answer ;) but finally, if you're generally working with adobe-workflow (acr), there's no real benefit using raw2cdng. (ok, 12bit and 10bit|log :D but thats no benefit, if youre saving in 16bit tif..)
:)
QuoteIt adds Camera Model: Canon EOS 5D Mark II to the DNG
Is that all? Is there any benefit in ACR, if the Camera Model is set accordingly? "Canon EOS PHR" is just a footprint of me - PHR is short for PHReekz.de, my website :)
regards chmee
Please someone upload somewhere a short and small RAW-File with hot/deadpixels and link it here. I just 've got an idea :)
regards chmee
aha cool, its your name! thanks for the reply! well one thing the camera profile can be loaded in ACR with the proper exif to set how it was shot , Neutral or Standard etc. so quicker to get to a place for FilmConvert profiles in AE, for instance.
i'm kinda between ACR / AE and resolve, just getting into da Davinci:)
1.2.0 fixed drag and drop for me
Is there a fix for Speedgrade?
I will upload sample RAW file with hot pixels from 60D in the evening.
Quote from: chmee on June 30, 2013, 10:17:45 PM
Please someone upload somewhere a short and small RAW-File with hot/deadpixels and link it here. I just 've got an idea :)
regards chmee
@DTSET123 You mean the wrong whitebalance? so, first, try to set it manually. and then :) we're discussing saving whitebalance-point to the raws. http://www.magiclantern.fm/forum/index.php?topic=6779.0
regards chmee
hey chmee, how about making an output folder option until commandline is finished?
I would love to just set my CF card as source and output to my project folder.
regards
@fatpig :) just started yesterday to implement this.. on the way.
Quote from: chmee on June 27, 2013, 08:28:19 AM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
raw2cdng v1.1.9 is online
- 10bit log is in (but experimental) - no linearizationtable (thx IliasG)
- naming conventions slightly changed
- fixed freezing gui
(!) 10bit log only works for bodies with blacklevel higher 2000 (!) next patch will have a lut for blacklevels around 1024 (!)
@IliasG
is your excel-calc that stable to calculate this lut by changing start values?
(lol. short test in resolve - 10bit/log is not looking that bad :) and hotpixels are gone in this one sample i have :D - if the red tint is too much, lower red gain to ~0.86 - but now its time to test the ability to grade and work with them - we squeezed all information into the 10bit - as a side advantage/drawback theres nearly no more data in blownout or dark parts of the picture - i dont really know if 10bit/log is a good grading base - possibly good for directly working in premiere/fcp/avid, after transcoding into a useable codec)
urgent on todoList for next days
- support of splitted files
- 10bit|log for blacklevel ~1024-bodies.
regards chmee
chmee, thanks for your work !!.
Sorry for the late responce, I was offline for a week.
Excel-calc should be stable (not 100% sure about this) for calculating on different black/white levels, only needs to pay attention on the details to have the smallest possible error at Black Level. It's easy (because the rec.709 gamma curve is linear there) by just increasing/decreasing the clipping point by 1-2 steps. I can give a hand on this as I have plenty of free time these days ..
About wrong colors .. don't bother with dirty corrections. Both WB and color matrices need to be applied on linear data or else we take strong color shifts. So we have to wait for the linearization table to be injected in exif data before we can make valuable tests. If this "injection" is a hard work there is a possibility we can surpass this need by a dedicated icc profile and use a converter which can make use of icc profiles (RawTherapee).
Talking about tests .. can you upload some RAW files which include dark and blown areas, made with the latest ML code ?.
The 6D white level shows at 15K... it should be 13K. Maybe there is a problem in raw.c code and the undef didn't work.
The 12 bit has been nice... lop off the least bits and it saves some space. Didn't see much impact in quality.. now if it zip compressed the dngs it would be all winner.
@iliasG i'd be happy if you would do the calculation for the 1024 blacklevel - the linearizationtable isnt a problem, it's prepared :)
@1% thx for the tests..
regards chmee
v1.2.1 is out - selectable Folder - setting will be stored, so it stays in app, until you abort while selecting folder. check-box for short disabling static folder and it will save the dng-folder as previous, in the same folder as the raw file is..
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
regards chmee
thanks for implementing this so fast! :) great! :)
1% mentioned zip compression. Is there a possibility for lossless compression like in Adobes DNG Converter? Remains directly editable.
if i finish 10bit|log, compression will be next. i have to read the dng-papers to look, whats possible.
( im a bit longer AFK, holiday-roundtrip to spain for 14 days)
Do your work, test, check and make some good videos :)
regards chmee
1.2.1 and 1.2.0 are crashing on me. i converted a clip to test then i deleted the CinemaDNG the converted the same clip again and it started crashing. Seems to be running slower to now. im not sure what happened. 1.1.9 still works for me. 1.2.1 and 1.2.0 crash after one clip finishes
I love your software though its amazing how you are able to this thank you!!!
Latest version is crashing on my Win8 64bit machine because of insufficient disk space, I haven't checked older versions but I suspect them to behave in the same way.
Maybe you could add an error message and/or a check if the needed space is available on the volume?
Keep up the good work chmee!
my workflow is as follow.
RAW>raw2cdng>CDNG sequence>Resolve>premiere>Resolve for final render
anyone know a way to properly white balance in ACR?
the method i saw on Resolve is rubbish.
Quote from: vikado on July 09, 2013, 11:09:07 PM
my workflow is as follow.
RAW>raw2cdng>CDNG sequence>Resolve>premiere>Resolve for final render
you can use adobe cinema dng importer (for adobe premiere cs 5.5 but it works with cs6 too) for import your cdng sequences in premiere. After you edit your movie, you can import xml and conform your sequence in Resolve then grading and finally render.
Quote from: vikado on July 09, 2013, 11:09:07 PM
my workflow is as follow.
RAW>raw2cdng>CDNG sequence>Resolve>premiere>Resolve for final render
anyone know a way to properly white balance in ACR?
the method i saw on Resolve is rubbish.
I don't know how you are white balancing in ACR?, but if there is a reliable reference white in the frame then you can use the picker tool at the top.
My white balance is a more involved process. I try to take an image of a dedicated white card/grey card. I am only using one of the cheaper ones, but they seem to be as accurate as I need them to be.
I will usually take a normal shot of the white card (as a photo)in the light conditions that I am in and then use that for the following videos in that light by saving the white balance collected from that image (as a preset).
Also I try to get a white balance for every single shot but that can just be too time consuming and frankly it's not something you can do for every single shot, every time.
The 10bit log mode is a lot of fun!!! has a real nice filmic look to it.
Quote from: sarotaz on July 09, 2013, 11:46:45 PM
you can use adobe cinema dng importer (for adobe premiere cs 5.5 but it works with cs6 too) for import your cdng sequences in premiere. After you edit your movie, you can import xml and conform your sequence in Resolve then grading and finally render.
where do i download this importer?
so this workflow is RAW>>ACDI>>premiere>>resolve?
there's no use of proxy files?
this seems like an easier way as i dont have to do no goofy roundtripping.
Quote from: mageye on July 10, 2013, 01:21:45 AM
I don't know how you are white balancing in ACR?, but if there is a reliable reference white in the frame then you can use the picker tool at the top.
My white balance is a more involved process. I try to take an image of a dedicated white card/grey card. I am only using one of the cheaper ones, but they seem to be as accurate as I need them to be.
I will usually take a normal shot of the white card (as a photo)in the light conditions that I am in and then use that for the following videos in that light by saving the white balance collected from that image (as a preset).
in ACR, i change "As shot" to "Auto".
it does a fantastic job of white balancing.
unless there's a more "pro" manual route.
Quote from: mageye on July 10, 2013, 01:21:45 AM
Also I try to get a white balance for every single shot but that can just be too time consuming and frankly it's not something you can do for every single shot, every time.
in my last shoot, we used a calibration target in conjunction with the slate for every take in a scene.
the calibration target was used to denoise footage in post. (i used neatvideo to denoise the footage).
obviously using the target for every take would be a pain in the ass, but we soon learned that using the target only on the first set of takes in a new scene was sufficient.
we only used the target in a new scene. not necessarily all takes of the same scene.
hope i explained it clearly.
edit:in new scene, i meant not new camera angle, i mean new physical scene (i.e., outdoors to indoor, or other room)
wait, does premiere convert these files to 8-bit or does it retain the 12-bit color space when i process the raw files in raw2CDNG?
Quote from: vikado on July 10, 2013, 03:02:32 AM
where do i download this importer?
so this workflow is RAW>>ACDI>>premiere>>resolve?
there's no use of proxy files?
this seems like an easier way as i dont have to do no goofy roundtripping.
https://dl.dropboxusercontent.com/u/26702029/cinemadng_p4_cs5-5importer_060211.zip (https://dl.dropboxusercontent.com/u/26702029/cinemadng_p4_cs5-5importer_060211.zip)
I've tested it with premiere cs6 and a short example sequence from blackmagic cinema camera and works perfect.
Follow this tutorial. You can adjust white-balance, black point, white point ecc. like ACR. :)
Hope this helps.
Quote from: sarotaz on July 10, 2013, 09:18:29 AM
https://dl.dropboxusercontent.com/u/26702029/cinemadng_p4_cs5-5importer_060211.zip (https://dl.dropboxusercontent.com/u/26702029/cinemadng_p4_cs5-5importer_060211.zip)
I've tested it with premiere cs6 and a short example sequence from blackmagic cinema camera and works perfect.
Follow this tutorial. You can adjust white-balance, black point, white point ecc. like ACR. :)
Hope this helps.
thanks!
just tried this myself.
everything is fine except for realtime playback. it chugs along. resolution at 1880x800 @ 12bit.
the same sequence plays fine on resolve but its having a difficult time in premiere.
i've tried cs5.5 and CC. and its the same.
i have a modest i7 2600k OC at 4.9ghz, 32GB 1600 memory.
i've edited R3D 5k footage before as a 1080p sequence on premiere and my computer has had no issues crunching those files.
i just wanted access to ACR in my workflow for proper balance and noise reduction. :/
plus premiere converts these to 8-bit color space, this wont be acceptable in the silver screen.
bummer.
I thought that was a 8 bit importer :o
I tried to import 12 bit footage and it could't do so- I wonder what I did wrong?
Hi, thank you very much for this fine software.
Which bit depth do I need to choose for ML raw on 5D3? Thank you.
Just found a way so that Davinci Resolve Recognizes CinemaDNG sequences while they still remain in the folder where all the files are.
This makes it a lot easier sorting and browsing trough your video's without having to open an extra folder before you can see it.
I don't know if i'm terribly late with this discovery or one of the first but just wanted to share it with you guys.
You can see what i mean in this picture:
(http://img849.imageshack.us/img849/2374/65ql.png)
This saves a lot of time and frustration, now i don't have to put all the CinemaDNG files from different video's in one folder so i could browse through them.
What i did was take the name of the folder of a shoot i did with the Blackmagic Cinema Camera and just renamed one of my 5D RAW map names to this new one.
Edit: Just scraped a lot of my findings since i just found out that all you have to do is add a C befor the last four numbers.
Example, if your file is called "M16-2227" all you have to do is change it to "M16-C2227" and Davinci Recognizes it as a movie instead of a folder
@chmee
any news for bad pixel opcode tag in your converter?
@scarluuk:
I dont have to change anything in the names, and can use the CDNG folders in resolve rightaway.
Quote from: fatpig on July 11, 2013, 11:05:03 AM
@scarluuk:
I dont have to change anything in the names, and can use the CDNG folders in resolve rightaway.
It's a bit hard to explain for me, English isn't my native language.
Adding the C makes Resolve interpret the folder as a clip instead of a folder.
Normally you have to open the folder the see the clip, now you can see the folder itself as a clip so it's easyer to browse through your clips.
Quote from: vikado on July 10, 2013, 08:26:08 PM
everything is fine except for realtime playback. it chugs along. resolution at 1880x800 @ 12bit.
the same sequence plays fine on resolve but its having a difficult time in premiere.
i've tried cs5.5 and CC. and its the same.
i have a modest i7 2600k OC at 4.9ghz, 32GB 1600 memory.
i've edited R3D 5k footage before as a 1080p sequence on premiere and my computer has had no issues crunching those files.
It appears as though the CinemaDNG import "experimental" plugin is no longer available from Adobe. They cite poor real-time performance as the primary reason for pulling the plugin from the Labs section.
I too have had a great deal of positive experience working with .r3d files within Premiere, even on an aging 1st gen Mac Pro with no Mercury acceleration. It seems that if Resolve is able to provide realtime or near-realtime playback of these files, Premiere Pro should too.
Please consider sending Adobe a feature request, the more they hear from our community, the more likely they are to revisit the functionality.
Quote from: scarluuk on July 12, 2013, 06:57:49 PM
Adding the C makes Resolve interpret the folder as a clip instead of a folder.
You sure about this? If I add a C to the name, I can't get Resolve to see anything. Is this a Mac/PC thing?
I can't get it to work, all I know is that the way it works without the C seems pretty efficient and fast.
Try to new Resolve 9.1.5, in changelog they wrote "Improved DNG format support". maybe this was fixed.
Quote from: Anisotropic on July 13, 2013, 07:36:08 AM
Try to new Resolve 9.1.5, in changelog they wrote "Improved DNG format support". maybe this was fixed.
THANKS!! New version of resolve read dng sequence from raw2dng!
I experienced crashes with raw2cdng when coverting some raw movies from 5D3.
Hey there, back from holiday (picture? (http://www.flickr.com/photos/chmee/9269206557/)):) So.. where to start. With the important thing: With resolve 9.15 support of 14bit dng's i dont see any need paying more attention to my app. (i'm a bit puzzled, blackmagic did that - thought, they have some more important things to code and to distant their own camera - but finally they get more users with this decision.) if there's no real pro argument, v1.2.1 will stay the last one :)
@pascal - no infos, no help.
@all - thanks to all, especially them used my tool.
Quote from: chmee on July 14, 2013, 05:08:40 PM
Hey there, back from holiday :) So.. where to start. With the important thing: With resolve 9.15 support of 14bit dng's i dont see any need paying more attention to my app. (i'm a bit puzzled, blackmagic solved that - thought, they have some more important things to code and to distant their own camera - but finally they get more users with this decision.) if there's no real pro argument, v1.2.1 will stay the last one :)
@pascal - no infos, no help.
@all - thanks to all, especially them used my tool.
Hi Chmee - Welcome back! :)
Even with the Resolve update there are still benefits to using your app. It's simple to use for a start. The 12bit compression is useful for saving space and I like being able to convert from the card to a HD folder without copying the raw files. I hope you continue developing it.
Quote from: chmee on July 14, 2013, 05:08:40 PM
Hey there, back from holiday (picture? (http://www.flickr.com/photos/chmee/9269206557/)):) So.. where to start. With the important thing: With resolve 9.15 support of 14bit dng's i dont see any need paying more attention to my app. (i'm a bit puzzled, blackmagic did that - thought, they have some more important things to code and to distant their own camera - but finally they get more users with this decision.) if there's no real pro argument, v1.2.1 will stay the last one :)
@pascal - no infos, no help.
@all - thanks to all, especially them used my tool.
Well come back.
I was expecting you back on July 17 so I didn't upload a pack of LUTs (I added 12bit log also) which are ready. I just want to do some refining and maybe include linearization to 14bit now that Resolve supports it. If nothing goes wrong I 'll upload in the next 12 hours.
Do the new Resolve support losslessly compressed DNGs as those we take after running through Adobe DNG converter ?.
I think 10bit log (and 12bit) losslesly compressed is a great space saver. Everyone needs it.
Quote from: chmee on July 14, 2013, 05:08:40 PM
@pascal - no infos, no help.
Hi, I wasn't sure if someone was interested in details. Thanks for your interest.
I uploaded a RAW file that crashes the program upon processing.
http://www.speedyshare.com/XQHVT/M13-1629.RAW
(click on Download: M13-1629.RAW on top)
PS: there is no option in raw2cdng for 14 bit DNG? 10, 12, 16, what does ML Raw need?
@andy600 thx appreciate that :)
@iliasG :D ok, so "our" benefit is the space saving :) beside the bugpatching the next bigger step will be finishing 10bit/log and 12bit/log with compression. fine fine :)
@pascal i removed 14bit, because it was not needed (it was in the first versions) - resolve couldnt handle that and apps like in the adobeworld dont need cdng (ok, speedgrade, but this one needs 10 12 or 16bit) - tomorrow i'll download your file and try to figure out, whats happening
@fatpig you're not forgotten.. but the cards now are reshuffled (is it a known english phrase? :)) - the users can use the official raw2dng to get into resolve - we try to optimize the filesizes without losing quality. to be honest, i have to learn/figureout about commandline how to implement it into an existing winform-app. ;)
regards chmee
Quote from: chmee on July 14, 2013, 11:14:30 PM
i removed 14bit, because it was not needed (it was in the first versions) - resolve couldnt handle that and apps like in the adobeworld dont need cdng (ok, speedgrade, but this one needs 10 12 or 16bit) - tomorrow i'll download your file and try to figure out, whats happening
Hmm I use DNG with Adobe. Why would Adobe users not need DNG?
for them raw2dng is the right converter.
Is there a way to batch convert conveniently with raw2dng.exe?
Quote from: scarluuk on July 10, 2013, 11:21:18 PM
Edit: Just scraped a lot of my findings since i just found out that all you have to do is add a C befor the last four numbers.
Example, if your file is called "M16-2227" all you have to do is change it to "M16-C2227" and Davinci Recognizes it as a movie instead of a folder
Works for me on Windows, although it introduces the same problem AE does in that all my clips come in as
"DNG[000000-000670000million].dng". I'll find a way round that though. Great find, Scarluuk.
Welcome back, Chmee. Your app was vital till BM upgraded Resolve, as was Fatpig's till I needed to use Resolve, so thank you both for that. What I like about R2C is its ease of use, whilst BATCHelor is great for organising projects. RAWanizer has its uses though - the file information it provides and it opening the last folder you selected upon restart. I'd love a raw converter that would combine all of these things and maybe use a Resolve style GUI that exchanged Media, Conform, Colour... with Basic, On set, Power user, Info... Either way, the work done by you guys on the raw converters doesn't go unappreciated at all.
@pascal:
http://www.magiclantern.fm/forum/index.php?topic=5645.125
@chmee:
ok, i get it.
still: raw2dng.exe does not support split files.
Hey there, i'm much workin.. so, finally i dont really will continue the raw2cdng-tool - all changes and small patches (if..!) are only tests for the forthcoming mlv2cdng-tool - MLV is in every form the better approach to stabilize the raw-revolution - and mlv2cdng will be coded from scratch as a commandline-tool with an external GUI. please be patient.
But back to raw2cdng: i thought about integrating exif-data from a cr2-file into the metadata of the dng. by now i only read the RGGB_Levels_measured from the cr2-makernotes and convert them into the dng-AsShotNeutral-Values. If someone's up to test that, feel free to write, if it helps/works in ACR or Resolve.
The Approach is easy - right before recording movie, do a photo. put both into the same path and give the CR2-file the same name.
Example:
M02-2221.RAW
M02-2221.CR2
raw2cdng will recognise the file automatically (it will be shown up with +CR2 in the importlist, same as splitted files..). It's tested by now only with a 5DIII. By the Way, if you're recording a whole set with the same whitebalance, you can name this one photofile ALL.CR2 and all files in the importlist will use this data - except the files having their own CR2.
Example:
M02-2201.RAW (will use ALL.CR2)
M02-2221.RAW (will use M02-2221.CR2)
M02-2221.CR2
M02-2312.RAW (will use ALL.CR2)
ALL.CR2
raw2cdng v1.3.0 - with CR2-whitebalance (http://dl.phreekz.de/raw2cdng/raw2cdng.1.3.0.zip)
regards chmee
oh wow, that's a fantastic addition chmee!
i did a quick test myself. color balance looks spot on.
that's one less step on resolve for color balancing!
awesome job, bro! ;D
Quote from: chmee on August 12, 2013, 12:14:02 AM
Hey there, i'm much workin.. so, finally i dont really will continue the raw2cdng-tool - all changes and small patches (if..!) are only tests for the forthcoming mlv2cdng-tool - MLV is in every form the better approach to stabilize the raw-revolution - and mlv2cdng will be coded from scratch as a commandline-tool with an external GUI. please be patient.
But back to raw2cdng: i thought about integrating exif-data from a cr2-file into the metadata of the dng. by now i only read the RGGB_Levels_measured from the cr2-makernotes and convert them into the dng-AsShotNeutral-Values. If someone's up to test that, feel free to write, if it helps/works in ACR or Resolve.
The Approach is easy - right before recording movie, do a photo. put both into the same path and give the CR2-file the same name.
Example:
M02-2221.RAW
M02-2221.CR2
raw2cdng will recognise the file automatically (it will be shown up with +CR2 in the importlist, same as splitted files..). It's tested by now only with a 5DIII. By the Way, if you're recording a whole set with the same whitebalance, you can name this one photofile ALL.CR2 and all files in the importlist will use this data - except the files having their own CR2.
Example:
M02-2201.RAW (will use ALL.CR2)
M02-2221.RAW (will use M02-2221.CR2)
M02-2221.CR2
M02-2312.RAW (will use ALL.CR2)
ALL.CR2
raw2cdng v1.3.0 - with CR2-whitebalance (http://dl.phreekz.de/raw2cdng/raw2cdng.1.3.0.zip)
regards chmee
can't wait for the mlv2cdng . . fingers crossed
@chmee
Maybe this has already been answered but, if I got it right the cdng format is just a tiff with some metadata (or whatever) this means it's already debayered, right? But when I put these clips into Resolve (coming from a 50d) I get false colores and artifacts from debayering, although they show up correctly in ACR. I don't get this ???
Quote from: araucaria on August 13, 2013, 05:55:35 PM..if I got it right the cdng format is just a tiff with some metadata (or whatever) this means it's already debayered, right..
* tiff is a containerformat.
* tiff can contain pattern-oriented picturedata as well, here bayer-pattern from a sensor.
* dng is in general a tiff with some mandatory tags in the metadata
* btw CR2 is as well a tiff with another file-ending ;)
saying this is very important for me :)
* cinemadng is quite the same as dng, with some videorelevant tags (f.i. framerate, reelname, timecode)
shortly saying, no, the dngs (cdngs too) are not debayered - and by now i think, the debayering-algorithm in resolve is not the best - far away from the adobe-world, which is able to de-vignette or remove badpixels (automatically or/and with help of metadata) and so on..
regards chmee
Thank you :D
Quote from: Canon eos m on August 13, 2013, 05:29:33 PM
can't wait for the mlv2cdng . . fingers crossed
Hi,
I am missing something here. I copied a *.CR2 file and name it the same as the RAW video file and dragged both on the raw2cdng convertor. The interface opens. What next?
? You just drag/drop the RAW-File into the list inside the exe - NOT on the exe-icon! The Converter recognises itself, if the file is splitted and if a cr2-file exists. if it finds splitted files, it will be shown in the importlist with (+[Number]). if it finds a corresponding cr2-file, it will be shown as (+CR2). Thats it. Then convert via button.
regards chmee
Quote from: Canon eos m on August 14, 2013, 04:22:01 PM
Hi,
I am missing something here. I copied a *.CR2 file and name it the same as the RAW video file and dragged both on the raw2cdng convertor. The interface opens. What next?
open the utility first and then drag and drop the .cr2 file and the raw. you'll see a (+cr2) after the raw file.
plus, you dont have to rename the .cr2 file, and if you do, just rename it "ALL.cr2" like he mentioned and use that for all your conversions.
btw, chmee, i noticed a noticeable difference of white balance between your utility and adobe camera raw(7.0).
adobe camera raw white balance was set to "auto". temperature at "2850". and Tint set to "+24"
the light source was a typical tungsten lightbulb on a lamp, where the shade of the lamp gave off a warmer diffusion of the light.
i felt the proper white balance was more true on adobe camera raw vs your utility.
even though yours looked right, it was a bit off based on the light source.
hope this was insightful :)
link below of comparison.
http://i.imgur.com/cFDv5B2.jpg
@vikado because i m out of home im thankful to see some picture-examples and im excited, my idea works :) So.. facing this example i'm not sure, if my whitebalance-correction is needed.. Did you try (asShot) in ACR? My idea was indeed a neutral whitebalance, not a pleasing (subjective!) one - the question is, what is easier to handle? changing a neutral picture into a warm one or correcting a tint into neutral :)
regards chmee
Quote from: chmee on August 14, 2013, 09:02:23 PM
@vikado because i m out of home im thankful to see some picture-examples and im excited, my idea works :) So.. facing this example i'm not sure, if my whitebalance-correction is needed.. Did you try (asShot) in ACR? My idea was indeed a neutral whitebalance, not a pleasing (subjective!) one - the question is, what is easier to handle? changing a neutral picture into a warm one or correcting a tint into neutral :)
regards chmee
here is the breakdown between the two files. http://i.imgur.com/JRuSweJ.jpg
left side is the dng with 1.3.0 and the right is the 1.2.1.
both having "as shot" and "auto" for comparison.
based on what the light source is (single tungsten), you'll see that ACR set to auto is the most accurate, in my humble opinion.
Quotethe question is, what is easier to handle? changing a neutral picture into a warm one or correcting a tint into neutral :)
changing the kelvin values and not touching the tint is best, (i think)
Hi, I am new in the magic lantern world, just got a brand new 5d m3, I was wondering, what kind of a computer do I need to process all this data? Thanks!
Quote from: nico on August 14, 2013, 10:30:19 PM
Hi, I am new in the magic lantern world, just got a brand new 5d m3, I was wondering, what kind of a computer do I need to process all this data? Thanks!
just to process raw to dng sequence or dnxhd444/prores444?
you dont need a lot of cpu power to crunch through raw files.
the difficulty is playing back those DNGs in realtime on resolve or your NLE.
i have two computers running an overclocked 2600k set to 4.9ghz, 32gb 1600mhz RAM, one has a gtx 570 and the other a gtx 580.
playback in realtime is fine. i use proxies on premiere so the real resource hog is in resolve, where i'll get dropped frames during playback when i start to use noise reduction on a node.
so my recommendation is to get a gtx 680 or above to be on the safe side or maybe a 670 if you're short on funds.
as for myself, im going to buy a gtx780 before we start on our feature.
@vikado yeah, if i see the comparison, the auto-definition made by adobe is a pleasing one - by "mathematical" view i'd say, the AsShot-measured RGB-Values are ok, because neutral-gray :) have to sleep and sit back home on my desktop on that :)
The Resolve readme for the latest versions says:
"What's new in DaVinci Resolve 9.1.5
...• Improved DNG format support
"
Is it really DNG they mean or cinemaDNG?
Can't get ordinary photo DNG recognised.
Thanks
Since Davinci Resolve won't work fine (even after blurring the chroma,etc...) I wanted to try out speedgrade. It plays the cdngs at realtime (with a shitty 560ti, resolve wouldn't do more than 12fps with 1080p footage) but everything has a green cast to it. Is there already a cure? I'm a total n00b to speedgrade, so any help is welcome!
The Adobeforum's got some really good guys - there are already some threads about it. bmcc-user have this problem as well.
the weakest (with good links in it ) :) http://forums.adobe.com/message/5579981
regards chmee
edit: please do me a favour: open one (ML)dng with ACR - save it as 16bit-DNG-uncompressed and import it in speedgrade. Did it changed its behaviour?
It won't show, it only accepts footage in cdng. I took a small cdng secuences made with your converter and opened it in PS as 16bit then clicked save as, DNG, uncompressed, and then tried to open in Speedgrade without luck (it won't render it "Failed to decode frame).
I'll try to read through the Adobe forums, thanks a lot!
marvelous News - Premiere CC Update in October will give us native CDNG-Support.
http://blogs.adobe.com/premierepro/2013/09/adobe-premiere-pro-cc-october-2013-release.html
regards chmee
... and Cinema DNG can be debayered on a supported GPU (in Premiere CC October release) for even better performance. :) ;) :'(
i got the chance to play with some cdng sequences in the new adobe CC (converted via this tool) - works okay - but the picture turns out very pinkish (maybe some metadata are missing ?? or maybe premiere dosent read the black/white levels correct).
But can be played in realtime in fullres. :-D
- a shame that ARC is not available as in AE.
(http://s9.postimg.org/x6y3g2y7z/woods.jpg)
Hi. When I try to open a folder, where imported by raw2cdng files are located, in Resolve it hangs up and closed with error. What can be wrong? Im using 16bit linear and 15% white unchecked when importing. AE opens without problem.
Some example of DNGs:
https://drive.google.com/folderview?id=0B2NWwDUimG9cOGEyOGEwMjUtZDYyNC00ZmUyLWIyZDgtMzhmNjE4ZWU5NTMy&usp=sharing
I have seen green shifting with cdng's as well. I believe it is from incorrect color matrix data in the meta data. I tried a sequence in another app and got a green shift, setting the app to not use the color matrix fixed the issue. I guess it is possible that the magenta shifting is from the same issue.
im getting green fringing with the oct.4 build for my mk2.
but only on the 10-bit, but 12bit and 16bit are fine.
this is for the 1.3.0 version
Quote from: mityazabuben on October 10, 2013, 01:21:31 PM
Hi. When I try to open a folder, where imported by raw2cdng files are located, in Resolve it hangs up and closed with error. What can be wrong? Im using 16bit linear and 15% white unchecked when importing. AE opens without problem.
Some example of DNGs:
https://drive.google.com/folderview?id=0B2NWwDUimG9cOGEyOGEwMjUtZDYyNC00ZmUyLWIyZDgtMzhmNjE4ZWU5NTMy&usp=sharing
Resolve 10.b3 works fine.
after investing time and brain for work and other projects i'll spend next time back on raw2cdng and mlv2cdng. next days i'll analyze the color shifts. btw. 10bit log output is ONLY for bodies with blacklevel ~2048 - i think, in raw2cdng i'll code a selector depending on body if 10bit-output is enabled or not. (multithreaded transcoding almost done, will be in v1.3.5)
@bnvm
interesting opinion, i'll look if its true.
The crashes in beta 2 are confirmed by other users (in rawmagic-thread) - solved with beta 3 as said by mityazabuben.
regards chmee
Quote from: mityazabuben on October 10, 2013, 01:21:31 PM
Hi. When I try to open a folder, where imported by raw2cdng files are located, in Resolve it hangs up and closed with error. What can be wrong? Im using 16bit linear and 15% white unchecked when importing. AE opens without problem.
Some example of DNGs:
https://drive.google.com/folderview?id=0B2NWwDUimG9cOGEyOGEwMjUtZDYyNC00ZmUyLWIyZDgtMzhmNjE4ZWU5NTMy&usp=sharing
i had the same issue.
do you have a graphics card installed?
if you do, turn off your integrated graphics chip in your bios.
i turned mines off and it solved the problem.
something about how resolve crashes if there's a gpu and integrated gpu both on.
Quoteafter investing time and brain for work and other projects i'll spend next time back on raw2cdng and mlv2cdng. next days i'll analyze the color shifts. btw. 10bit log output is ONLY for bodies with blacklevel ~2048 - i think, in raw2cdng i'll code a selector depending on body if 10bit-output is enabled or not. (multithreaded transcoding almost done, will be in v1.3.5)
Thank you Chmee! Your RAW2CDNG is a great solution, if you could fix this problem it would be greatly appreciated!
Hello. with the October update of SpeedGrade the program doesn't want to handle them. Odd right? Premier does well with them, but SG just crashes. Any ideas?
because i'm no subscriber of Adobe, i've got no "new" adobe-apps. simply i can't check.
(but im thinkin' about it after reading some workflow-experiences)
Quote from: chmee on November 01, 2013, 12:07:07 PM
because i'm no subscriber of Adobe, i've got no "new" adobe-apps. simply i can't check.
(but im thinkin' about it after reading some workflow-experiences)
chmee, As of right now, we have no way to use 5D3 DNG or CinemaDNG in the new Premiere Pro CC 7.1
DNG (with newest raw2dng) will not import because it says there is no metadata, stops dead on error without importing.
CDNG (with newest raw2cdng) imports but has a terrible pink cast which cannot be removed.
If there was some way around this via a new raw2cdng or raw2dng we would be very grateful for the chance to try this new workflow.
Quote from: eoshq on November 01, 2013, 06:04:38 PM
DNG (with newest raw2dng) will not import because it says there is no metadata, stops dead on error without importing.
Think thats because its 14bit.
Quote from: eoshq on November 01, 2013, 06:04:38 PM
CDNG (with newest raw2cdng) imports but has a terrible pink cast which cannot be removed.
Read more here about it: http://www.magiclantern.fm/forum/index.php?topic=9131.msg86286;topicseen#msg86286
As far as i can say from premiere cs6, it does not import any 14 bit files, except via dynamik link
Does it import dual ISO files? (DNGs saved by cr2hdr - they are 16 bit)
If you re talking about premiere cs6, i can try it if someone has a file for me
Quote from: a1ex on November 01, 2013, 06:48:04 PM
Does it import dual ISO files? (DNGs saved by cr2hdr - they are 16 bit)
nope, same error as when processing with raw2dng.. (would not take them at all - says "The file appears to have no media data")
So maybe its more a header (DNG vs CDNG) problem - than a 14bit vs 16bit thing (or could be a mix of both?)..
In raw2cdng 1.3 there is the chance to import the whitebalance-data from a taken cr2-photo. did you tried that? did it changed the colors? description is in blog entry. http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
regards chmee
Quote from: chmee on November 01, 2013, 07:43:47 PM
In raw2cdng 1.3 there is the chance to import the whitebalance-data from a taken cr2-photo. did you tried that? did it changed the colors? description is in blog entry. http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
regards chmee
Yep, tried that - did not remove the pinky stuff :-)
But if i change the bit depth to 10bit CDNG via your tool it looks better (easier to fix) 8)
(http://s15.postimg.org/734ptwux7/woods10bit.jpg)
:D nevermind, decided to jump on the cc-train. so will check that. please be patient 8)
Quote from: chmee on November 01, 2013, 08:28:34 PM
:D nevermind, decided to jump on the cc-train. so will check that. please be patient 8)
Great news, Thank you 1000 times for trying to help us.
I read today on the Adobe forum that they never even tried other cameras besides BMCC, but they did not tell anyone that until today. Adobe are the worst at deceptive marketing, they knew since the September 8th announcement that CinemaDNG support was only going to work with BMCC files. Despicable.
Quote from: chmee on November 01, 2013, 08:28:34 PM
:D nevermind, decided to jump on the cc-train. so will check that. please be patient 8)
awesome.
this is what you call dedication.
Quote from: chmee on November 01, 2013, 07:43:47 PM
In raw2cdng 1.3 there is the chance to import the whitebalance-data from a taken cr2-photo. did you tried that?
It's not a WB issue, but a black level one. The Adobe editor reads the black level most likely as 0, instead of 1024 or 2048 or whatever the right value is.
My hypothesis: the metadata in the CDNG files is corrupted (http://www.magiclantern.fm/forum/index.php?topic=9131.msg86286;topicseen#msg86286) (e.g. having a wrong type for a tag can cause read errors on all subsequent tags in an IFD), and some parsers happen to get over it and others not, depending on how they choose to handle this error. If this is true, the problem is in the DNG converter, not on Adobe's side.
ahh, thanks. i used the validator long time ago.. generally i saw the problem on my side, of course.
Quote from: hookah on October 08, 2013, 02:39:46 PM
- a shame that ARC is not available as in AE.
Have you tried right clicking on the dng and clicking "Edit Original"? It opens ACR for me.
(http://s14.postimg.org/ppin3moo1/Screen_Shot_2013_11_01_at_11_19_24_PM.png)
Quote from: jrumans on November 02, 2013, 05:28:42 AM
Have you tried right clicking on the dng and clicking "Edit Original"? It opens ACR for me.
And what does this accomplish? Absolutely nothing! Sure it will open the first cdng file of the sequence in Photoshop hosted camera raw, but it doesn't do anything to help this pink cast situation. What is the point of your posting this? Do you not try things before you post about them on a forum?
Quote from: eoshq on November 02, 2013, 07:18:11 AM
but it doesn't do anything to help this pink cast situation.
I found a work around , with AE link below
http://www.magiclantern.fm/forum/index.php?topic=8177.msg86395#msg86395
Quote from: reddeercity on November 02, 2013, 07:29:59 AM
I found a work around , with AE link below
http://www.magiclantern.fm/forum/index.php?topic=8177.msg86395#msg86395
This really gains us nothing because you could have been doing this all along, and all the way back to at least CS5, maybe further. There has been a Dynamic Link between After Effects and Premiere pro for a long time. So, yes this does get your cdng (or dng, BTW) footage into Premiere Pro without the pink cast, but I would not call it a workaround. It is really apples and oranges, two totally different processes. Let me explain further. This way negates any advantages of the new cdng functionality of PP 7.1. By using Dynamic Link back to After Effects it is massively slower and using tons of extra CPU because it is literally using After Effects to render the footage in the background. Check the Process viewer while doing this and you will see that an instance of After Effects is running along side Premiere Pro.
Oh and there is no reason to save a copy of your After Effects project as a CS6 copy when you do this Dynamic Link method as described in your video, not sure where that idea came from but it does nothing to benefit this process.
Sorry, but that s wrong.
At the moment, importing dngs in AE, making proxys and open this AE Sequence (not like in the video safe as premiere) in premiere, you have the best workflow.
no exporting, no xml, no metadata and no comlicated workflows with davinci.
unless Premiere is able to open dng s via acr, this is the best non destructiv workflow.
At the moment, Premiere only applys some Camera metadata to the dngs when opening them, no way to work with that.
Quote from: dude on November 02, 2013, 12:36:06 PM
Sorry, but that s wrong.
At the moment, importing dngs in AE, making proxys and open this AE Sequence (not like in the video safe as premiere) in premiere, you have the best workflow.
no exporting, no xml, no metadata and no comlicated workflows with davinci.
unless Premiere is able to open dng s via acr, this is the best non destructiv workflow.
At the moment, Premiere only applys some Camera metadata to the dngs when opening them, no way to work with that.
This workflow was possible all the time - but its almost impossible to work that way. (even on my workstation 24core xeon, 24Gb ram, quadro 5000)
Its okay for small sequences/projects.
..in terms of efficiency its wrong calling it workflow - its tea- and coffeetime with some moments of interacting ;D
I just did a 200 GB project with this workflow,
it s the only one which holds up the full quality during the post process.
I am editing videos for about 15 years now, and with my new i7, i can play for and back in the timeline as i want it. i remeber some time when even viewing h264 footage was not really able with premiere.
Feel free to show a more efficient workflow...
discussion has been done in other threads - the resolve-roundtrip-workflow "was" an efficient one imho. the simple things to change, core-values like exposure or whitebalance means, re-rendering raw-files - a no-go (!) in terms of efficency(!). NOW we have a better resolve10 and hopefully a doped adobe-world. Did you try roundtrippin/dynamiclinking with speedgrade now? It's to me/to us to optimizing now the converter-tools. (davinci-xml-roundtrip was not complicated and gave you all the time control over these core-values. acr is the better debayer-tool, but fast? never.)
regards chmee
ive been doing this workflow http://www.youtube.com/watch?v=spwR2bsbvls
and its been perfect since i cant run resolve with my dngs.. they force close my resolve
by now i sat at the pink or green cast in the raws - its not scientific, but a first help if you're using speedgrade with dynamic link. actually i downloaded speedgrade CC as a trial. give it a try, because: you're able to twist the wrong color matrix into a useable one. i bought for this reason a colorchecker passport, just to see how good the raws are.
(1) in "clip" you'll find the "color space", the third tab. to give you an idea of the direction i linked a picture with my settings on a greenish picture (dont know why its not magenta). you can make a first justage on greycards or some other black/white things. look on the histogram on the left to match the upper and lower boundaries, being same value.
http://dl.phreekz.de/raw2cdng/magiclantern_correctMLI.jpg
(2) after that i pushed the contrasts, working good with the colormatch-layer..
http://dl.phreekz.de/raw2cdng/magiclantern_correctMLII.jpg
(3) this example just took 10minutes to understand speedgrade and to solve..
here's one picture from the file
http://dl.phreekz.de/raw2cdng/colorchecker_example_16bit.dng
if you rebuild my example, you'll see, only with the green value for blacklevel you can barely eliminate green or magenta fill-light
dont hesitate, try this video to understand speedgrade.
http://tv.adobe.com/watch/no-stupid-questions-with-colin-smith/introduction-to-speedgrade/
but, i'm sitting on that to figure out, how to solve that without this rework. i assume, till now the apps didnt look into the colormatrices, speedgrade is the first one. (!) please do not forget, adobe will work with 10, 12 or 16bitfiles. not with 14bit.
regards chmee
Quote from: chmee on November 03, 2013, 02:06:03 AMyou're able to twist the wrong color matrix into a useable one
A related trick: if you have problems with highlights, try turning off the color matrix. I've fixed this bug in ufraw (http://www.magiclantern.fm/forum/index.php?topic=5197) (basically I'm shifting the matrix towards eye(3) as you approach the clipping point), but judging from a recent sample from Audionut (http://www.magiclantern.fm/forum/index.php?topic=7139.msg85905#msg85905), Lightroom has the same problem (and probably all other Adobe apps).
Hi Alex, what editing software are you using?
Do you think a modification in the DNG export softwares could fix that pink color in Premiere/speedgrade, I saw this question asked somewhere but with no answer, is it possible to export cdng from raw so that it would be like the blackmagic cdng ?
@malex we're workin' on that. (i wrote that 1 entry before!)
Guys we have been sold on a false idea when it comes to editing Cinema DNGs in the new Premiere Pro 7.1
Even when it works correctly, as with Black Magic Cinema Camera (BMCC) files, it is pretty much worthless as you lose the main reason for shooting raw in the first place, raw adjustments.
Over on the Creative Cow forums, one user has put it like this:
" It appears that Premiere Pro uses only part of the latitude of the RAW format, there's apparently no way to change it, and it's apparently suddenly impossible to access the full tonal range of RAW.
As an example, I have some footage with burnt out clouds. If I open a DNG image in Photoshop or After Effects, I can edit the exposure upon import, and bring back all the latitude on the clouds. That's the whole point of using RAW.
In Premiere Pro, CinemaDNG seems cropped to video levels, and I can't bring back any information in the highlights. I simply turn down blotchy over-exposed highlights."
You can read the whole thread here http://forums.creativecow.net/thread/3/947862
Sadly, as it stands now I see no point in editing cdng natively in Premiere Pro.
Quote from: eoshq on November 04, 2013, 07:37:46 AM
Sadly, as it stands now I see no point in editing cdng natively in Premiere Pro.
Damm We was so close :-)
But adobe said:
We understand your frustration and are continuing to work hard on this. Getting correctly debaryed GPU accelerated CinemaDNG was quite a success for us. Adding the options is now possible.So we must wait a little more..
Quote from: eoshq on November 04, 2013, 07:37:46 AM
Guys we have been sold on a false idea when it comes to editing Cinema DNGs in the new Premiere Pro 7.1
I was upset after the latest PPro update. But, I have been happy using Ginger HDR workflow. It's pretty easy to wrap a DNG seq to a gnr file and import that gnr into PPro CC.
Gary
:) I'm a little bit sneering :) Adobe only said, Premiere will have cdng-support - they never said it will support full bitdepth-range including all effects. but it works good, if you're working with speedgrade as "controling-unit" via dynamic link. no coffeetime-pre-, re- or backgroundrendering (as with AE), and fulltime-control over the important values, in a significantly better and faster GUI than ae. Please, think about it - if you have been fooled by adobe or you have fooled yourself because of high-hopes - now you can put your cndg-sequences directly into premiere, then linking from there into the better tools to refine the footage - non-destructive.
But, i hope as well, they put at least a "simple" cdng-properties-window into premiere to change some core-values as wb/temperature and exposure.
my 2cents. chmee
I was avle to import ML DNG seqs before the 7.1 update. So, nothing new there. And I was able to get/use ACR on an import of ML DNG seqa and now that is gone with the latest AE CC update?? The least I wanted/expected was ACR on import of DNGs to PPro and to maintain access to the 14 bit raw for grading in PPro. If, if I needed more tools for grading, trehn a trip to SG. I was hoping to recall ACR in PPro like Photoshop has always done. Maybe ML can hack Photoshop into being a good raw video editor/color app? :)
Last Sept when Adobe announced this update, I was talking in email to an Adobe rep and he did lead me to believe what I just said. I told him I was using Canon with ML and I wanted to import raw DNG seqs and edit them "natively" with all the beauty of the original raw 14 bit. He said that was coming in the Oct " 15th" 7.1 update. And now it's a different story. We need a new editor app from a small hip upstart company. They can be Black Lantern.
..by now you've got Speedgrade as your Realtime ACR, with Premiere as the main Holder of your timeline - neither with proxies nor with baken 8bit-sequences. Thats my thoughts..
Premiere, AE and SG are changing the DNG files. Didn't you read the thread we had in the COW forum? Per kivked it off wit-
Per Holmes on Oct 31, 2013 at 5:11:17 pm
Hi,
I just got the new Premiere Pro 7.1, and excitedly started testing the CinemaDNG workflow, and I've hit a massive wall. It appears that Premiere Pro uses only part of the latitude of the RAW format, there's apparently no way to change it, and it's apparently suddenly impossible to access the full tonal range of RAW, even with Speedgrade or After Effects.
As an example, I have some footage with burnt out clouds. If I open a DNG image in Photoshop or After Effects, I can edit the exposure upon import, and bring back all the latitude on the clouds. That's the whole point of using RAW.
In Premiere Pro, CinemaDNG seems cropped to video levels, and I can't bring back any information in the highlights. I simply turn down blotchy over-exposed highlights.
If I take the sequence over into SpeedGrade, it's the same. I'm not able to pull any information out of the clouds, only reduce the brightness of already blotchy over-exposed clouds.
If this really is designed this way, what the heck is the point? The ONLY, I feel I can safely repeat, ONLY reason to shoot RAW is to be able to access the full tonal range of the image. But Premiere Pro seems to crop the latitude range to a video range.
But then there's no point in shooting CinemaDNG, and not simply shoot ProRes.
What am I getting wrong? Or what in the world are the designers thinking here, killing the one and the only reason anyone would use RAW in the first place?
Further, it appears that it's not even possible to "upgrade" to full CinemaDNG. If you've imported CinemaDNG in Premiere Pro, the manual says that the clip will be locked to that tonal range, even as you bring it over into SpeedGrade, which I've confirmed.
I can't figure it out. If this is true, CinemaDNG playback is an completely pointless non-feature.
What the heck?
Per
sorry, didnt read the cow-thread - ok. thats bad. so, i'd take speedgrade as footageholder and import them via dynamiclink into premiere - just be patient with my opinions, i'll upgrade to CC-subscription end of this week.
I am sorry. I forgot the link. check out all the posts in that thread.
http://forums.creativecow.net/readpost/3/947862
Quote from: chmee on November 04, 2013, 11:39:40 AM
so, i'd take speedgrade as footageholder and import them via dynamiclink into premiere
I will have to try that out. but, on the other hand, that can be a drag having a ton of footage sitting there in SG that does not make the final edit.seems backwards.
actually i'm playing around with the dng-tags from a 5DIII-Frame. while irfanview shows massive changes (changing values for colormatrix, analogbalance,asShotNeutral and so on) and photoshop cant even import them (colormatrices doesnt map equal camera values to XYZ D50) -> Speedgrade isnt interested in these changes, it looks all the same. tried that with bmcc-files as well (colormatrix to 100 010 001) - nothings changing.
It seems, Speedgrade is reading only the raw/bayer-data while not been interested in correctiondata. i dont like the idea of dividing green sensels by 2 to get "working" pictures, because thus i destroy the philosophy of raw-data. so it must be done in spedgrade.
please some do me a favour, give me a dng-file with a magentacast (mine have clearly greencast, because of the two green sensels (double the value) and raw output without colormatrix.. the only working way would be to build a LUT. The DCP-Files (4750°K) from X-Rite doesnt work, but i found a Tool (http://dcptool.sourceforge.net/Installation.html) to convert them into xml, so here's the pastebin-link (http://pastebin.com/8JUa7Xs4). its not a lut, but a colormatrix.
btw. put the RGBtoXYZ-"Correction" on the Footage, (its the +Sign in the layer-window) does it help?
regards chmee
uhh will play with it later :-)
Download a single DNG frame with pinkcast here.
https://docs.google.com/file/d/0BxHiL_qPbYP0VURRcUdISWdoSDA/edit?usp=sharing
i just played a lot more :)
i wrote a colormatrix (as template/example) you just can put on the whole timeline. Take this XML-Data (http://pastebin.com/1uMLpJrV), save it into a .look-File (f.i 5DIII_Greencast.look). Save this File into
C:\Program Files\Adobe\Adobe SpeedGrade CC\settings\looks\CinemaDNG
now you can use it as a saved colormatrix as seen in this screenshot:
http://dl.phreekz.de/raw2cdng/5DIII_Greencast_Example.jpg
(!) Thats a first understanding of RGB2XYZ and this sort of problems. Shortly saying, its not impossible. I dont know if i'd solve it (when using SG) in the cdng itself.
regards chmee
@hookah
if i use this file in Speedgrade, its got a greencast - and its looking quite good with my template.
http://dl.phreekz.de/raw2cdng/Hookah_Example.jpg
Okay
If i import direct into SG i also get a greenisy image
- If i use your look-file it looks OK
If i import into premiere i get pinkish image - if i send it to SG via dynamiclink, its still pink.
Then i made a new look/lut with inspiration from yours (quick adjusted by hand - need some more scientific fine tuning) for use in premiere in its Lumetri effect
- no roundtrip to SG, can play in full res, realtime inside premier.. SO COOL !!!
Here is my look/lut
http://pastebin.com/ituVygkk (http://pastebin.com/ituVygkk)
(http://s8.postimg.org/a00h26ghh/premiere.jpg)
delete.. double post
@hookah (good find)
Now its important to find out, if this premiere-lumetri-LUT is used before baking into 8bit - that means finally, you're able to highlight/shadow-recover and prevent banding issues, shortly benefiting the 14bit-bitdepth via effects/looks prepared in speedgrade. And then its not 100% fine, but it makes all the ae/converter-things obsolete.
mfg chmee
i cant seem to get a good upload.. youtube always makes my videos look like shit. why?
hookah & chmee -- Just ran some tests with this LUT this evening. It appears the LUT offsets the magenta cast somewhat, but is still a workaround as I'm getting some harsh banding and bizarre coloration. I see a greenish frame before my media loads, so it's acting like another intermediate color corrector which is another summing operation on the raw data, further destroying it. When I apply further image adjustment it just gets worse. This LUT is, however, useful as a temporary workaround - and chmee you pretty much nailed the color balance, whatever you used.
In terms of the differences between the BMCC CDNG and ML CDNG that EOSHD pointed out, is there a way to populate those fields with dummy or equivalent data to 'trick' premiere into seeing the correct color matrix?
This reminds me of what happens when I try to view 5D3 CR2 files in a lesser photo program like Picasa - EXACT SAME pink cast. Perhaps there's something there worth taking a look at?
I'm committed to helping you guys solve this! It will be a HUGE breakthrough and I'm convinced it will revolutionize the world of RAW.....dun dun dun
@CinematicSyndicate yesterday i tried an evening shot, and it behaves quite bad in the shadows. of course we/ i will investigate further, but as i wrote: speedgrade does not react on the correction-tags of the dng. the .look we used is a colormatrix outside the dng instead of being embedded inside the dng. did you played in premiere with lumetri-layer or in speedgrade?
i'll try to nail the color-matrix to D50 or D65 (http://en.wikipedia.org/wiki/CIE_Standard_Illuminant_D65). and it seems i have to combine the standard colormatrix (of the body) with the additional colormatrix of the colorchecker-measurements.. AND every Canon-Sensor behaves different. 5DIII-Colormatrices on 50D-Footage will not work well.
ah, and i tried with the hunter-lut (its a cube-file) - it made it worse inside speedgrade (but as a LUT-layer it pushes the colors positively on my testcharts)
regards chmee
Why not just subtract the black level from the raw data when generating the DNG? Since the video editor thinks it's always 0, you can try to make it 0.
So, set exif black level = 0, exif white level -= old_black, and for each raw pixel, p -= old_black.
worth a try, but
(a) is the calculated blacklevel 100% accurate?
(b) i assume, this is not the solution for the double-valued green-sensel (greencast)
(c) these data are no more useable in other software.
(d) whitelevel isnt read by speedgrade as well.
in terms of color-calibration, matrices and measurements i'm clueless, so now its poking in the dark. it's time for a colorchecker-superman :)
-> Here's a 1.3.3.BETA(!) (http://dl.phreekz.de/raw2cdng/raw2cdng.1.3.3.BETA.zip) with Blacklevel to zero-Switch. (ahh. theres a bug.its working wrong!) patience.
-> 1.3.3.1.BETA (http://dl.phreekz.de/raw2cdng/raw2cdng.1.3.3.1.BETA.zip) (it seems, there are sensels darker than blacklevel-value, its all crazy :D)
-> Here are 2 dng's from yesterday evening (http://dl.phreekz.de/raw2cdng/phr_darkExample_a.zip). They behave quite good in Photoshop/ACR, but i cant get the flowers back in speedgrade, kind of lack of "exposure compensation"
regards chmee
Quote from: chmee on November 05, 2013, 10:46:26 AM
it seems, there are sensels darker than blacklevel-value, its all crazy :D
Ufraw simply clips them to black. The only disadvantage I can think of is that a noise reduction algorithm will no longer perform well in deep shadows, but if you expose correctly it shouldn't be a big problem.
wanted to avoid if-routines inside the converter-loops. thats been my thoughts ;D
BTW, I still get the same validation errors in your DNGs. I thought you already fixed these and still have problems.
I've explained why this error may trick a parser into using a wrong black level, so try solving this first.
EDIT : Sorry, a1ex, found the problem. :facepalm: instead of writing the "long" 4 into the whitebalance-type, it wrote it into scale. changes a lot, because on that tag speedgrade reacts *huhh*
patience.. i have to go work for 3 days in another town, i'll be back i promise :)
regards chmee
i must be exporting different.. cause when i play my master video.. and i play my premiere player... it looks so nice..
but once i get it on youtube it STILL looks like crap?
12mps render? an i doing anything wrong?
@mnrv45 - you should post that to a different forum topic since this one is dedicated to the ongoing discussion about properly decoding CDNG files in editing applications. See the PM I sent for more help :)
raw2cdng 1.4.0.Beta online
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
(!) It's experimental (!)
* by 99% stripped files won't work.
* optional reducing of blacklevel by 90%
* framerate fixed
* UniqueModelID now changeable
* multithreaded
* speedgradeCC-look inside
* readme inside
regards chmee
Quote from: mrnv45 on November 05, 2013, 10:50:39 PM
i must be exporting different.. cause when i play my master video.. and i play my premiere player... it looks so nice..
but once i get it on youtube it STILL looks like crap?
12mps render? an i doing anything wrong?
h264 or prores?
Quote from: chmee on November 08, 2013, 11:08:18 PM
raw2cdng 1.4.0.Beta online
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
(!) It's experimental (!)
* by 99% stripped files won't work.
* optional reducing of blacklevel by 90%
* framerate fixed
* UniqueModelID now changeable
* multithreaded
* speedgradeCC-look inside
* readme inside
regards chmee
awesome work chmee, i'll do some tests over the weekend
I tried setting the Unique Model to 50D and it keeps giving me 50Dchmee :D . Other than that it seems to be working ok (Win7 64bit). ACR needs Camera Model Name for DCP to work correctly. It's currently set as Canon EOS PHR.
:) lol. ok.
So i asked in another thread. what are the important nametags for ACR? UniqueCameraModel (0xC614) or Model (0x0110). Both maybe?
regards chmee
Quote from: chmee on November 09, 2013, 11:45:01 PM
:) lol. ok.
So i asked in another thread. what are the important nametags for ACR? UniqueCameraModel (0xC614) or Model (0x0110). Both maybe?
regards chmee
For ACR 'model' is the important one. As raw_rec is hardcoded for daylight Can you maybe add WB as a selectable field? I usually pop off a still or 2 before hitting record so manually choosing ISO/WB etc would be very useful as you can refer to the still for exposure info.
BTW, 12bit DNGs look way better in Resolve than 14bit DNG. Thanks for keeping this going Chmee! :)
Quote from: Andy600 on November 10, 2013, 12:27:13 AM
For ACR 'model' is the important one. As raw_rec is hardcoded for daylight Can you maybe add WB as a selectable field? I usually pop off a still or 2 before hitting record so manually choosing ISO/WB etc would be very useful as you can refer to the still for exposure info.
it's already built in since 1.3.0 - just rename the cr2 into the same as your videofootage, it will be recognised and "measured!" wb will be taken. look into my blog description on 1.3.0 - http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
How did I miss that gem? :o
Quote from: chmee on November 08, 2013, 11:08:18 PM
raw2cdng 1.4.0.Beta online
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
Hi Chmee
Sorry to be a dunce, but I don't know what I should put in the 'Unique Model' field. Also, when I try to drag a Raw file in, the app does nothing (all paths are set etc.). All previous versions of your great app have worked perfectly for me! Do I need to uninstall them first maybe?
Any idea what I'm doing wrong?
Many thanks
Win 7
5DMk2
Speedgrade / Premiere Pro
Hi Chmee
thanks for your work
the version 1.4 o cdng crash when i start converting i have no problem with version 1.3
windows 7 64 bit
i hope you can help me
thanks
ok it 's ok now i change the name of model
but i can put canon eos 6d but canon eos 5d mark iii is to long
thanks
@timbytheriver hmm. if you drag it does not appear in the list? does it appear but the values are strange?
vaisse thanks. i will lengthen the modelname to ~24 chars. and the modelname inputform is not very inutitive i think, will change that as well.
@Chmee
Thanks for answering. It does appear in the list. But then when I press the button, the list entry turns red but doesn't process. The dng folder is created – but no files inside. :o
Also I'm really not sure what to put in the 'Unique model' box! Do you have an example I should use for my 5dMk2 please?
Thanks again for your work! :)
1.40 crashes for me when I press convert...I'm on windows 7 x64.
Also - where are we in the progress of getting Premiere to natively see cDNG without the magenta/pink hue?
important informations for a reasonably answer:
(*) the metadata are read accurate? are resolution, frames, framerate, body accurate?
(*) maybe it's an wrong converted mlv?
(*) firmware/ml-version from when?
(*) stripped or not?
(todo for me: clickable debug-output for crash-situations)
magenta/pink cast are showing, when the whitebalance is set wrong, but not on the line of warm/cold whites, but in right angle to it (looking on a colorgraph). Thats why there are two sliders for adjusting wb.
(*) did you tested the cr2-wb-embedder?
(*) are these problems as well with ACR/Speedgrade/Resolve?
(!) havin' a lot of work - tomorrow in the german tv is the bambi. means, after that i hope i can return to patch and update to a better/usable version. btw i test and code this tool under win7/x64/c#2010 - but this time i wrote a readme because it was fast coding without heavy testing.
regards chmee
Hi Chmee
Here's some of my debugging info if it's useful:
* Metadata for White Level (16383) I think is wrongly reported. Should be 15000 (?) All other is correct (frame rate, body etc.)
* I use ML build from 10.9.2013
* Not using mlv.
* I don't know what 'stripped' means! :-[
Screen grab of my Win 7 (It's running on Virtual Box on Mac OSX 10.7.5. All previous versions of your app have worked perfectly on this same setup!)
https://www.dropbox.com/s/96nu51crl0d3sbo/grab.jpg
Thanks again for your help.
Compare raw2cdng v.1.3.0 and raw2dng with 7D footage. Here is a result: https://drive.google.com/folderview?id=0B2NWwDUimG9cbS1mWFRtaXgzSFU&usp=sharing (https://drive.google.com/folderview?id=0B2NWwDUimG9cbS1mWFRtaXgzSFU&usp=sharing)
Left part is a raw2dng, right is a raw2Cdng. Import to Photoshop via ACR as is - nothing was changed. On a raw2cdng dng file clearly one can see color moire on a roofs (blue + yellow). What am I doing wrong?
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
raw2cdng v1.4.5 is online
* fixed modeltag (should work now with acr-profiles)
* multithreading seems workable with stripped files
* whitelevel back to 15.000
* mlv-workaround (read in blog-description)
@mityazabuben
maybe because of profiles. in the raw2dng-picture you can see the yellow/blue artifacts as well, but less harsh
Thanks chmee!
Unfortunately, v1.4.5 is still not processing any file and/or crashing for me :( Both on Win 7 and XP. If I try running as 'Admin' on Win7, the drag and drop does not work!
I'm running on Virtual Box on Mac OSX10.7.5 – but as I mentioned before all previous/early versions of your app have worked perfectly on it!
Any ideas please chmee / anyone?
Many thanks
is it showing any crash-error? with 1.4.0 multithreading came into source, i suppose, there's the problem.
drag&drop-problem is known - search for that.
f.i. http://answers.microsoft.com/en-us/windows/forum/windows_7-performance/drag-and-drop-not-working-in-windows-7/f6dcfa11-8c51-e011-8dfc-68b599b31bf5
regards chmee
Thanks for the reply! :)
Win XP is showing the 'Windows has encountered a problem and has to close' box
Win7 just fails to process the files (an empty folder is created, but no green progress bars) when I press the 'convert' button
@chmee Thanks for the link. The drag and drop fix mentioned there didn't work for me.
This time I ran the app I got a crash report. Does it say anything to you?
https://www.dropbox.com/s/sf70khxpv9qgb06/grab2.jpg
Thanks again.
next time, try to copy/paste the whole data inside the window. btw i'm writing the debug-Option.
regards chmee
Sorry! Stoopid of me. :o Here it is:
Problem signature:
Problem Event Name: CLR20r3
Problem Signature 01: raw2cdng.1.4.5.exe
Problem Signature 02: 1.0.0.0
Problem Signature 03: 52911779
Problem Signature 04: raw2cdng
Problem Signature 05: 1.0.0.0
Problem Signature 06: 52911779
Problem Signature 07: 5c
Problem Signature 08: 4ba
Problem Signature 09: System.NullReferenceException
OS Version: 6.1.7601.2.1.0.256.1
Locale ID: 2057
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789
Quote from: CinematicSyndicate on November 13, 2013, 10:38:37 PM
1.40 crashes for me when I press convert...I'm on windows 7 x64.
Also - where are we in the progress of getting Premiere to natively see cDNG without the magenta/pink hue?
1+
so thanks to timbytheriver we found a bug.
Dont switch the ModelID-Field on, if there's nothing in it.
if you switch on, fill in something.
OR
let it switched off, if you dont need it.
maybe you have to restart the application. patch on the way.
After pressing "convert" program crashes. Win7 64bit. Tried all boxes checked/unchecked.
v1.4.6 online
* modelID-switch-bug fixed
* switchable debuglog
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
I got strange issue new raw2cdng 1.46, when you load 9 split files, see wrong info of clips
See images below.
(http://farm6.staticflickr.com/5525/11037971356_8158f0dc3e_o.png) (http://www.flickr.com/photos/96806538@N08/11037971356/)
raw2cdng_1 (http://www.flickr.com/photos/96806538@N08/11037971356/) by davidmiazga1 (http://www.flickr.com/people/96806538@N08/), on Flickr
(http://farm8.staticflickr.com/7339/11037880455_698bef5d87_o.png) (http://www.flickr.com/photos/96806538@N08/11037880455/)
raw2Cdng (http://www.flickr.com/photos/96806538@N08/11037880455/) by davidmiazga1 (http://www.flickr.com/people/96806538@N08/), on Flickr
:) ok.lol. 9 Split-Files. Just give me a day to record this kind of file as well, and i will find the bug :) holy..
btw. is it working accordingly in 1.3.0?
But when you load after join manually 42GB total works ok :D
Quote from: chmee on November 25, 2013, 12:29:07 AM
:) ok.lol. 9 Split-Files. Just give me a day to record this kind of file as well, and i will find the bug :) holy..
btw. is it working accordingly in 1.3.0?
I don't know , i will try right now
Edit: Yes, 1.3.0 has the same issue.
But has no problem converting 42GB 2k frame size to cdng :)
@chmee
Love the Lut you made to help combat the problem (Seen on page 13). I think it would be really helpful to put that on the First post for people looking.
@reddeercity You're using the 1.4.6 with debug-output. please copy/paste the debug-log into a pastebin.
Quote from: chmee on November 25, 2013, 09:10:42 AM
@reddeercity You're using the 1.4.6 with debug-output. please copy/paste the debug-log into a pastebin.
hint: place a try-catch block around every thread function and event handlers.
and open a messagebox there displaying the stack trace.
thats the most simple and effective method i usually chose.
actually, much simpler is to add a handler for AppDomain.CurrentDomain.UnhandledException during initialization and log the error from there.
did you looked into the debug-output? to catch only exception is one thing - to see via outputlogs, what variables has been set and where it stops was my concern. i can read that.
Quote from: dmilligan on November 25, 2013, 07:11:21 PM
actually, much simpler is to add a handler for AppDomain.CurrentDomain.UnhandledException during initialization and log the error from there.
had certain problems with forms not honoring that well.
this works for you?
All of my projects are WPF, and I've never had it not catch anything. I actually capture most exceptions one level higher from the thread's dispatcher (a few slip through the cracks and reach the AppDomain handler though, at that point, the app is probably going to have to terminate, catching at the thread dispatcher and you can mark it 'handled' and keep going). http://msdn.microsoft.com/en-us/library/system.windows.application.dispatcherunhandledexception(v=vs.110).aspx
Creating handlers for these two events is the first thing I do when I start writing a new program.
Any particular reason you are still using forms?
thanks for that hint :)
> Any particular reason you are still using forms?
it's because it is "simple" and most likely being handled well by mono.
plus maybe because my visual studio professional license is quite old and i have the fear that WPF in .net 3.x is not good enough.
and the most important of all: my gut feeling says no :)
@dmilligan
i simply didnt started to "convert myself" into wpf. so, i didnt thought, it could be a drawback in this project.
regards chmee
You're welcome ;)
WPF in 3.5 is actually fairly decent (main thing missing is DataGrid). MVVM is incredibly powerful paradigm once you get to know it. I use the WAF (WPF Application Framework), it's extremely modular.
I'd love to help with some C# stuff since that is actually my area of expertise, but I don't have access to a Windows computer and Visual Studio outside of work.
Quote from: dmilligan on November 25, 2013, 11:05:59 PM
I'd love to help with some C# stuff since that is actually my area of expertise, but I don't have access to a Windows computer and Visual Studio outside of work.
you are welcome to add a monodelop project file or whatever you prefer ;)
Quote from: chmee on November 25, 2013, 09:10:42 AM
@reddeercity You're using the 1.4.6 with debug-output. please copy/paste the debug-log into a pastebin.
I'm sure howto use & copy in to a "pastebin"
Here the log info by my Drop Box
There was more 8 spitting file actually 11 plus the .raw make that 12.
but only reported 8. run it then program crashed
https://www.dropbox.com/s/0j0i7gsofz16guk/raw2cdng_multi_file_crash_debug.log
this was after i join manually total of 46GB
https://www.dropbox.com/s/npe51sf3vz8kzcc/raw2cdng_single-46GB-file_debug.log
How long would it take to incorporate an option to create a proxy video? That would be awesome, and maybe you and the developer of rawanizer could exchange code.
@reddeercity
thanks. i think i know where the problem is. patch in the evening.
@painya
jpg-filesequence is prepared. one of the next patches.
regards chmee
Where exhaust look-files for the lumetri-effect?
as i now tested, the .look file i made won't work in premiere. so i read about the iridas ilut-scripting and compatible formats.
http://doc.iridas.com/index.php?title=LUT_Formats
put the code into a textfile, name it "NameItAsYouWant.cube" (ending .cube is important). its a very simple 3d-lut and i just played around for 15-20 minutes, its far from perfect, but a first insight into the lumetri-thing.
TITLE "5DIII N"
LUT_3D_SIZE 2
DOMAIN_MIN 0.315 0.0 0.158
DOMAIN_MAX 0.47 0.49 0.52
-0.2 -0.2 -0.2
1.2 -0.2 -0.2
-0.2 1.2 -0.2
1.2 1.2 -0.2
-0.2 -0.2 1.2
1.2 -0.2 1.2
-0.2 1.2 1.2
1.2 1.2 1.2
use it with the lumetri effect and push with contrasts-effects. justify the black- and whitelevels with the Domain- values for r g and b. in small steps please :) regards chmee
the good news:
Yeah. It works. Without some strange LUTs and Precomposings. Straight inside Premiere. Files do well in Resolve, Photoshop/ACR and Speedgrade too.
the bad news:
Not online today ;) some small bugs inside.
regards chmee
raw2cdng 1.4.8 is online
(http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.8.gif)
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
regards chmee
Hi Chmee :)
I take the new Cdng for a test drive and thing are much better.
I have only done 3 test to date,
First, i still can't Link out of Premiere Pro to Speed Grade, it just Crashes.
But if you import the Cdng in to Speed Grade that works,
So far the setting that works for me (i will need more testing) is
Party mode on Gamma 1.0, at that you still need to do a fair bit of adjustment.
It take a little time to remove the some Pink cast at the high end & some Green cast at the Low end.
then i find if you use the speed grade gamma 2.2 ilut it put it in line .
Found that under the timeline options "setup" in the Calibration box .
So are still Getting the Full 16bit, CDNG or is it a Proxy file that adobe is using ?
Because it still look very flat and dose not have the same high fidelity image that i see in AE or FCPX.
:)
i thought, coding a gamma-corrector/curve would optimize the flat-amount, but it it does not. it turns into red'ish or blue/greenish cast. seems, that the cdng-alogorithm of premiere (*A) has an "optimized" to somewhat-blackmagic-log-curve and (*B) it crops the 8bit to video-range. i will start a cdng-discussion in the adobe forums - its not clear for me, what they have coded..
gamma 2.2 - thanks for finding.
regards chmee
Quote from: chmee on December 04, 2013, 07:01:44 AM
i thought, coding a gamma-corrector/curve would optimize the flat-amount, but it it does not. it turnes into red'is or blue/greenish cast. seems, that the cdng-alogorithm of premiere (*A) has an "optimized" to somewhat-blackmagic-log-curve and (*B) it crops the 8bit to video-range. i will start an cdng-discussion in the adobe forums - its not clear for me, what they have coded..
gamma 2.2 - thanks for finding.
regards chmee
Glad I could Help
Hi, I'm trying to get started with raw2cdng, and can't get past the first step. I downloaded 1.4.6, extracted it to my desktop, double-clicked, and immediately get the message "raw2cdng has stopped working. Windows can check online... etc." I downloaded 1.3 and the same thing happened. I also tried dragging a raw file onto it like with raw2dng and got the same result. I must be making some dumb mistake. Can someone please tell me what I'm doing wrong?
@mva
(1) Do you have .net4.0 installed?
(2) what OS? Windows-core or emulation/virtual system?
(as every time: more information leads to more accurate answer)
regards chmee
Quote from: chmee on December 04, 2013, 12:37:09 PM
@mva
(1) Do you have .net4.0 installed?
(2) what OS? Windows-core or emulation/virtual system?
(as every time: more information leads to more accurate answer)
regards chmee
Hi chmee,
(1) I have "Microsoft .NET Framework 4 Client Profile" installed
(2) Windows 7 64 bit (core, I suppose, since it's not running on another OS)
my tool is writing/reading settings-data inside the users/appdata-folder. Did you tried running the tool as administrator? (right-click)
Yes, I had tried running as an administrator. No difference.
Not sure this will help, but these are "problem details" I can view when I get the error message:
(http://img837.imageshack.us/img837/6389/6a54.jpg)
Well, chmee, thanks anyway for trying. Looks like there may be something quirky going on with my computer.
(A) please look into this folder:
C:\Users\[Username]\AppData\Roaming
There should be a file named
raw2cdng.settings.json
if not, create an empty one. (Thats the only file that "couldnt be found" on app-start.)
(B) if this file exists, look if there's an entry named
debugLogEnabled
{"outputPath":"C:\\","uniqueName":"Canon EOS 5D Mark III","debugLogEnabled":true,"debugLogFile":"C:\\Users\\chmee\\Documents\\Visual Studio 2010\\Projects\\raw2cdng\\raw2cdng\\bin\\Debug\\raw2cdng_debug.log","gamma":1,"maximizeEnabled":true}
(this is my example settings-file here on this laptop. Try to put this data into the file. Change the debugLogFile value into a useful path :)
(C) You extracted the path from the zip? really? :)
regards chmee
(A) raw2cdng.settings.json wasn't in the folder you asked me to check. Unfortunately, however, I don't know how to create an empty file. :( (I did some Googling but found too many different answers mixed with too much uncertainty.) Can you tell me how to do that? Is it done from a command prompt?
(C) Since I sometimes just drag files out of WinRAR without properly extracting them, I tried again making sure I extracted it properly, and had the same problem.
EDIT: chmee, I saw your post following this one (#397), and I'll reply soon. Want to do some reading and testing, and I'm a bit busy now to do it. Thanks!
Open the folder, right click and use -> "new" -> textdocument. i hope, you have switched off hiding extensions.
-> http://www.sevenforums.com/tutorials/10570-file-extensions-hide-show.html
It seems, my app does not have the permission to write this file.
http://support.microsoft.com/kb/308419
regards chmee
chmee, I really appreciate your help, but I think I'm going to give up for now. Creating the empty file as you suggested didn't work. I tried the app on two other computers running Windows 7 64-bit in my house, and weirdly it immediately crashed each time on those too, even when running it as an administrator. I haven't had this problem with any other program/app/tool so there's something quirky going on. I Googled "System.IO.FileNotFoundException crash" and found lots of descriptions of problems similar to what I'm experiencing, with people being advised to try all sorts of things which don't work (including reinstalling .net 4.0), but I couldn't find anything I wanted to try, partly because I don't want to spend more time on this now. So I think I'll just periodically retry the app (e.g., after Windows updates) in hope that the problem has gone away, but for the time being I guess I'll be sticking entirely with DNGs. Thanks again.
@mva
can i ask you, if you're from a country with "abnormal" characters or did you try to change the location of the app? root folder of a harddisk for example? but non the less, for me as well an interesting thing.
Beyond that, the adobe-Forum is quite strange, on "our" part of dng-discussion they only've got the thinking, no, it seems, its not planned to open the native dng-support to other files than bmcc.. where the heck is the idea of the "widely-universal-photographic/filmdata-format"? DNG is not.
http://forums.adobe.com/thread/1326153?tstart=0
edit: i think, this is the main answer:
QuoteCinemaDNG is a huge spec, we targeted getting the colours correct from the BlackMagic camera first.
regards chmee
chmee, yes, although I do pretty much everything in English on my computer(s), all are enabled for Hebrew characters as well, I have Microsoft Office Hebrew language packs installed in them, etc. Could this be a factor?
Yes, I did try running the app from the root directory of a different drive, but without success.
my thoughts went to unicode/ascii (chracterencoding) and something like that. maybe it cant read/write/find the proper file because of differences in right that. wrong character-encoding leads to "FileNotFound". i didnt had such an issue, so finally cant help by now, but maybe "our" worldwideweb can.. have to search and read about it.
regards chmee
Quote from: chmee on December 05, 2013, 11:30:05 PM
my thoughts went to unicode/ascii (chracterencoding) and something like that. maybe it cant read/write/find the proper file because of differences in right that. wrong character-encoding leads to "FileNotFound". i didnt had such an issue, so finally cant help by now, but maybe "our" worldwideweb can.. have to search and read about it.
regards chmee
FYI, I updated to .NET 4.5.1 and now raw2cdng works! :)
lol. really.. i have to check, what code-excerpt uses .net 4.5 (i'm kind of pissed, VisualStudio doesnt show up those things).
Thank you.
Thanks for this awesome tool. I've been doing a couple of test shots just to get the workflow down.
I'm currently going: EOS-M(crop mode) -> focus pixel remover -> raw2cdng -> Premiere(editing) -> speedgrade (grading)
The latest raw2cdng w/ partymode is really great, and seems to have more lattitude in speedgrade vs h264, but I have noticed an issue where clipped values in raw show as magenta in speedgrade and premiere. Any help or tips would be appreciated, I've tried going through this whole topic, but maybe I missed something along the way. Thanks!
@chmee Thank you for v1.4.8! I have much better results now in 'Partymode' and Gamma 1.0 getting rid of the pink going into PPro 7.1
Something else I have noticed is this chromatic aberration (?) in Cdng. I don't see it in straight dng. Two 100% crops from the same .raw file to illustrate here (You can see it clearly on the ripples):
https://dl.dropboxusercontent.com/u/21590714/raw2Cdng.jpg
https://dl.dropboxusercontent.com/u/21590714/raw2dng.jpg
Where is this coming from?
If this is old ground already covered / fixed, I do apologise! :)
rgds
PS Both the files here were debayered in ACR from the original .raws and then cropped.
@jordancolburn
appreciate that :) canon raw have different black and whitelevel for the components (r, g and b) it seems. if you use the partymode, all values lower blacklevel (~1023 or ~2047) will be cutted and all values higher than 15000 will be cutted as well. So, yes, it seems, there is an under- or overflow of values (i already have switches for that, but your picture shows theres another thing i missed)
@timbytheriver
you're welcome. one or two sites ago, there was an example with the same problem. it seems (in your picture as well) there is CA in dng as well, but much weaker. by now i dont have any clue to avoid that. did you try to set a according modelID? Maybe ACR is reacting well with the right Body ID?
so, sorry i cant help by now. but the tool isn't finished yet (g3gg0 just offered his help to some questions) and i hope i can throw out a nutty version till christmas :)
@chmee Good news about the xmas version! Can it have chocolate, brandy and cream included? :P
So model ID/no model ID seemed to make no difference to the chromatic aberration issue. I did notice something odd though: Mac OSX finder window reports all my dngs from raw2cdng as being dimensions: 1864 x 1042, whilst all other tools (RawMagic, raw2dng, etc.) report them as 1876 x 1054 (I shot at 1880 x 1058 ad build 4.10.13)
Odd! :o Could this be a clue?
Thanks again.
rgds
but the raw-infos inside raw2cdng are accurate? in dng there s an attribute for padding - i set it to zero. will look on that. thanks.
@chmee raw2cdng gui reports the dimensions as 1880 x 1058
Hi, I have just tried 1.4.8 and great! but...
I did have few bugs, I guess it has to do with speedgrade not being stable enough with dng files ? I had this before too. When I use the direct link from premiere to speedgrade, it opens the files fine, OR it opens it with some greenish look... it's random, and I guess it has nothing to do with raw2cdng. It looks good in premiere, even though I think the white balance is off a little bit.
Great job chmee!
Quote from: chmee on December 07, 2013, 11:18:59 PM
@jordancolburn
appreciate that :) canon raw have different black and whitelevel for the components (r, g and b) it seems. if you use the partymode, all values lower blacklevel (~1023 or ~2047) will be cutted and all values higher than 15000 will be cutted as well. So, yes, it seems, there is an under- or overflow of values (i already have switches for that, but your picture shows theres another thing i missed)
Thanks for the reply. I'll see if I can recreate it with some more details, but for now here is an example image:
(http://jordancolburn.com/wp-content/uploads/2013/12/Sequence-01.Still003-800x450.jpg)
I was trying to do a tough test vs h264 and tried to shoot inside with the lights off and a bright snowy outside, so to get the inside exposed right, the outside was very overexposed, but the highlights came back ok in SG (except the magenta area).
@jordancolburn
i'm searching/reading for this red/pink highlight thing. It seems, the whitelevel is higher set than needed. in RT there is a bugreport right about that, but they mention lowering the WL (after debiasing?). i'll try some of the things :)
links:
magenta highlights - https://code.google.com/p/rawtherapee/issues/detail?id=1695
WB-subtraction and RGB to XYZ - http://lclevy.free.fr/cr2/#wb
regards chmee
@jordancolburn: may I see your DNG?
(it's offtopic, I'm just experimenting with some highlight recovery algorithms)
Quote from: a1ex on December 09, 2013, 05:11:13 PM
@jordancolburn: may I see your DNG?
(it's offtopic, I'm just experimenting with some highlight recovery algorithms)
Sure, when I get home today, I'll grab the DNG frame send it to you.
Quote from: chmee on December 09, 2013, 05:03:33 PM
i'm searching/reading for this red/pink highlight thing. It seems, the whitelevel is higher set than needed. in RT there is a bugreport right about that, but they mention lowering the WL (after debiasing?). i'll try some of the things :)
Thanks for looking into this! Out side of that bug, raw2cdng really was the final part of the workflow I needed to get into editing and grading. Now I can actually do a real-ish world test comparing h264 and raw with the eos-m.
https://infogr.am/bmcc-delinearization-curve?src=web
(some brainfood) (now both curves lin>log and log>lin inside)
full csv-data for log>lin on http://dl.phreekz.de/raw2cdng/bmccLog.csv
regards chmee
Quote from: chmee on December 18, 2013, 10:44:28 AM
https://infogr.am/bmcc-delinearization-curve?src=web
(some brainfood) (now both curves lin>log and log>lin inside)
full csv-data for log>lin on http://dl.phreekz.de/raw2cdng/bmccLog.csv
regards chmee
Chmee hi,
Do you mean that you (will) use Bmcc curves ?. At which bitdepth ?. If you need something just ask. I think the old gamma curve I provided needs a bit refinement. Of course it was proposed for 14 bit linear to 10 bit log so we will need something different if our intention is to use gamma encoding for 14lin to 12log or 16 to 12.
Regarding magenta highlights, because the clipping White level for each model depends on ISO and apperture, in latest RT there is now an external file (camconst.json) where we can set per ISO White Level and a multiplier depending on apperture. I think a database is the only way to be safe while not loose highlights.
Or you can use exif (maker data) info in CR2 files, latest exiftool decodes more Canon tags and White Levels are among them although they are a bit conservative sometimes ...
Currently, for RT, there is a need of Canon raw samples with at least a "burned" area shot with intermediate ISOs (125, 160, 250, 320 ...) to fill the database ..
5D2 and 5D3 are the only models whith complete data, all others (and 6D) are missing.
Interesting.... i was stumbling too upon the camconst.json and the researchs of the RT community just today.
But in purpose of finding a solution for the pink highlights in dcraw/UFRAW.
=> http://www.magiclantern.fm/forum/index.php?topic=5197 (http://www.magiclantern.fm/forum/index.php?topic=5197)
As i don't have PremierePro, i wasn't aware that the pink cast problem could have the same origins as the dcraw case.
Maybe the new raw engine of Premiere is basing upon dcraw? ;D
Edit: I just remembered seeing your name on the RT issue board. :)
i made a second logarithmic convert-mode (12bit with bmcc gamma) - for the shadows and the whole picture grace it does a good job, but it cant eliminate the red tint, because of the different component blacklevel and amplitude - it's time to go deeper - something i wouldnt do, because i recalculate the raw-data more i supposed to. it's a real shame, that the dng-importer in premiere doesnt have a colormatrix-option or reads the colormatrix/cameracalibration-data accordingly. omg.
@IliasG just send me a binary file with 16384 int (2byte) entries (14bit->10bit) 16bit->10bit.
@glubber my thoughts, its not really about a red-overflow. if its magenta (red-blue'ish), the green-channel isnt accordingly read, if its green, it was not taking into account, there are two green sensels..
regards chmee
Binary ??. I don't know how .. a simple text file with input and output columns is the best I can ..
I'll have to reopen and remember what the refinement is .. hopefully tomorrow
Did you managed to insert the linearization table in DNG exif data ?.
@iliasG
yeah. do that. i'll convert it here via php. i didnt inserted the linearizationtable by now, because i assume, premiere does not take account of it. For Speedgrade and Resolve it has to exist, so yes, i'll do that today.
edit(!) IliasG, forgot to say: dont cope with the uneven black- (1023/2047) and whitelevel (~15000). take 0 as blacklevel and 65535 as whitelevel for the gammacurve. internally i do now all from 16bit to destination-bitdepth.
Hello Chmee,
I've been long a fan of your application, but only recently discovered this thread (stupid me).
I read it wholly before I was going to make some points.
First I want to thank you for your efforts and readiness to help.
I'll try to be brief in my points:
1. DaVinci Resolve. As I got from this forum, there was a trend to move to Adobe from Resolve due to a better de-bayering. It was true with Resolve 9 (I hated it) but Resolve 10 for me show far FAR better results than Adobe ACR. I shoot with 50D, and artifacts from Adobe are huge compared to Resolve 10.
Things move quickly, and maybe somebody should give Resolve 10 Lite a try.
2. Roundtrip. So I use R2C for Resolve and Premiere roundtrip. I read the whole discussion about naming and XML problems and while reading really had a desire to jump back into the past ))
2.1. FPS. Now, I am shooting 25 FPS, but when imported to Resolve, all clips are 24FPS. Have to change it manually. When I forget, I have a de-sycnh with audio, and have to re-render. But this is the beginning for surprises ))
2.2. Folder naming. So I have 2 clips from shooting raw video:
1.M24-1803.RAW +2 spanning files
2.M24-1806.RAW +2 spanning files
They were shot in consequence, one after another at 18:03 and 18:06 yesterday, 24 Dec.
After conversion (made today on 25 Dec at about 15:00) I have 2 folders:
131225_1235_M241803K
131224_1235_M241806N
Why 131225_ and 131224_ ? Why _1235_ and _1235_?
All this makes basically impossible to sort these folders by name.
2.3. Resolve clip naming. When imported into Resolve, these clips show as:
M241803K.[000000-003194].dng
M241806N.[000000-004983].dng
That is a step back -- I can sort them now, which is good. But after grading when I render my dailies with "Use source filenames" checked, I have files named:
M241803K.mov
M241806N.mov
2.4. XML issue. After editing these files in Premiere, I export XML file for Resolve. And of course I have that problem where only the first clip shows ok, and others have error. Yes now I know about the solution with reelname suggested by a French-style contributor.
But I also solved this problem in my way: I searched through XML file and changed:
M241803K.mov to M241803K.[000000-003194].mov
M241806N.mov to M241806N.[000000-004983].mov
And now the problem was solved -- Resolve made all linking correctly, without reel name.
Of course you cannot do that for big projects, but I think solution is in file naming -- to make Resolve import clips and render clips with the SAME NAME.
3. 1.4.8 version.
Presently I use 1.3.0 because 1.4.8 is totally unstable for me -- it crashes in the middle of every second conversion. 1.3.0 is as solid as a rock ))
I also tried MLV conversion after placing mlv_dump.exe into the R2C folder, but the program shows only one frame from the clip. Failed to convert.
And by the way, on the screen shot you have 1.4.9. Are you hiding anything? :D Just kidding!
Let me say big THANK YOU for your sincere efforts! Happy Christmas!
same problem other conver is work but raw2cdng have wrong in this :
import in premiere is same problem
2.1. FPS. Now, I am shooting 25 FPS, but when imported to Resolve, all clips are 24FPS. Have to change it manually. When I forget, I have a de-sycnh with audio, and have to re-render. But this is the beginning for surprises ))
Happy Christmas, mediali & rommex.
Thanks. So, the fps-problem is a thing inside resolve. You have to adjust the basic settings of your project. It seems, resolve is not interested in the fps-metadata.
The problems Wirth the newer versions will come From the multithreaded reading. I'll rewrite that. So stay on 1.3, for resolve it makes no difference.
Quote from: chmee on December 26, 2013, 11:03:49 AM
Happy Christmas, mediali & rommex.
Thanks. So, the fps-problem is a thing inside resolve. You have to adjust the basic settings of your project. It seems, resolve is not interested in the fps-metadata.
The problems Wirth the newer versions will come From the multithreaded reading. I'll rewrite that. So stay on 1.3, for resolve it makes no difference.
Thanks, chmee. FPS is not big deal, really. But what about the naming issues? Any plans to resolve them?
Im not home, writing on a Smartphone. When im back, i'll answer the important things
Hi,
I tried to convert an old file using raw2cdng 1.3.0 and view it on PPro CC - came out all pink.
I tried doing the same thing on AE CC and came out just fine. I think it is the PPro CC that is causing the issue.
@Chmee - Do you know how to change/update exif in CDNGs?
I want to add camera model to some old CDNG files converted with an old version of raw2cdng. I don't have the original .raw files. I've tried using Exiftool but it keeps throwing up errors. It seems to work with normal DNG files.
Chemee, my question too, how do I add camera model to version 1.3.0. 1.4.8 is not very stable yet.
changing the model ID is not a one-liner, sorry. (use a hex-editor) It's the tag id's 110 (hexoffset 5e) and C614 (hexoffset 106) - it writes its string-length and the string offset into the tag-data. i dont know if i wrote the string on the end of the metadata, possibly the length of the string changes the offset of forthcoming data - makes following entries illegal/defective.
so, i tried to write a more stable and better version to christmas (you've seen the 1.4.9 :)) but the time before christmas is full of familiy-action. give me some days after newyear to stabilize raw2cdng.
toDo for the next version.
* stable read and write with mutlithreading (had a chat with g3gg0 about it)
* dng-header generation - modul rewrite
* IliasG's new LUT for 10bit/log (will work for older bodies with blacklevel 1023 as well)
* 12bit/log with bmcc like data (hopefully we're coming to an lovely workflow with premiere)
as is mentioned before, Resolve 10 is no longer a problem, so i dont want to "improve" any things there. speedgrade is working as well - but premiere and the dynamiclink-workflow lacks of being consistent/stable .
(happy new year - and: magiclantern rocked this year, so all my technical love goes to the ml-crew that made this happen)
regards chmee
thnx chmee works like a charm!!! :D
Can i ask if i need to tick partymode?
Should i tick TIF Model Tag ?
thank you
@pind the partymode is for better pictures in premiere, "could" also help in resolve and speedgrade, but it worked without it - its up to you. the TIF Model Tag is for using profiles in ACR, leave it untouched if you dont need it.
Quote from: chmee on January 04, 2014, 10:18:39 AM
@pind the partymode is for better pictures in premiere, "could" also help in resolve and speedgrade, but it worked without it - its up to you. the TIF Model Tag is for using profiles in ACR, leave it untouched if you dont need it.
Thank you
To bulk edit: http://freeweb.siol.net/hrastni3/foto/exif/exiftoolgui.htm
Hey,
I made a small GUI for "RAW2DNG".
You can download and test it here (Adfly) (http://adf.ly/cC9up) or here (Direct) (http://www.mediafire.com/download/o47z423ge69yesq/RAW2DNG%20GUI%201.2.zip).
If there are any bugs/problems PM me, thanks.
Screenshots (http://imgur.com/a/pOZR3)
Changelog:
Version 1.1:
- some bugs fixed
- add "Export DNG's to RAW file folder"
Version 1.2:
- some bugs fixed
- 500% faster
Quote from: Freddy2142 on January 05, 2014, 04:07:36 PM
Hey,
I made a small GUI for "RAW2DNG".
You can download and test it here (http://www.mediafire.com/download/bv4nv66qceog18q/RAW2DNG%20GUI.zip).
If there are any bugs/problems PM me, thanks.
Screenshots (http://imgur.com/a/pOZR3)
This is AWESOME! Thanks for putting this together Freddy!
Would it be possible to have an option to export the DNG files to the Raw file folder?
Many thanks again!
Quote from: ulysnep on January 13, 2014, 01:38:23 AM
Would it be possible to have an option to export the DNG files to the Raw file folder?
Here is the updated version: Adfly (http://adf.ly/c65aN) or Direct (http://www.mediafire.com/download/xgoh5halh0uf9si/RAW2DNG%20GUI%201.1.zip)
nice work here, hope to see native support for mlv format and dual iso mlv format somewhere in the near feature.
tried to use latest version with a bunch of mlv files, and the app crashed. bummer.
i was kind of wrong, it did convert all of the .mlv files to the .raw extension, it crashes when i clicked the convert to dng files.
Quote from: arturochu on January 16, 2014, 07:46:48 AM
i was kind of wrong, it did convert all of the .mlv files to the .raw extension, it crashes when i clicked the convert to dng files.
i had some crashes with this too, so i used this tehnique
Import your raw files from CF card to Data storage (HDD) and after you do this take one raw file out from HDD and convert.. when this will work click on other 50Gigz of raw files you have (on your 64GB CF card) and convert all ... thats what i do and dont have crashes...
Can we hope for a 1.5 version that works with MLV+sound, and still works with .RAW also...? :-)
Quote from: Malex on January 20, 2014, 11:16:38 PM
Can we hope for a 1.5 version that works with MLV+sound, and still works with .RAW also...? :-)
That's a good point and could save us some time on the workflow.
Yes, you can hope for a MLV-Version - sound is not on my primary list, but it would make less sense doing it without the audio part :) please be patient some more days. you're not forgotten..
regards chmee
I read the last 5 sites now. I'm sure I missed why the discussion on the pink cast in premiere with CDNG files was dropped.
Does your app Raw2CDNG fix the problem so we have access to ACR when importing files to Premiere?
Currently im using RAWMagic with some kind of Ae workaround since my director needs to do many animations and effects in Ae anyways.
Question unanswered to me yet is 8-bit, 10-bit, 12-bit thing-
Do we at least have the full RAW quality in Ae?
Thx for getting me on track :)
Could some one please help. I can convert raw footage easy with raw2cdng but cannot do this for mlv_dump files despite mlv_dump being enabled. Read chmee's website and instructions but cannot figure out what I am doing wrong.
@canon eos m
the more you tell about the problem, the faster you can get real help :)
* if you drag/drop mlv-files, the tool starts immediately with converting to raw.
* these temporarily converted raw-files should lay inside the raw2cdng-folder. you see them?
* after that you should see the filename and main specs of the footage in the application-filelist. do they? are they ok?
* they behave as normal raw-files (because they are :)
But, i need some help of you, guys. i'm kind of desperated, because of the adobe-behaviour (http://forums.adobe.com/message/6014956). my goal was to get a functional import into premiere (still it is) - importing into speedgrade and resolve isn't anymore a problem, it works. i thought of writing a standalone-2tif-converter(16bit) with built in ssd-demosaicing (http://www.ipol.im/pub/art/2011/bcms-ssdd/). biggest drawback is the monstrous filesize-increase, but the usability straight inside premiere could be worth it. by now, i'm stucked, and ml-raw seems to me like a "proof of concept" with a big impact on the professional vendors/scene/market, but less benefit for the workhorse, doin its job. simply i think, cdng is/was the wrong choice.. actually i rather go h.264-vanilla because of the instant workability.
so finally, whats your thoughts about efficient workflow? staticaly converting with ACR is not (imho).
regards chmee
I am beginning to feel the same way - H.264 codec (with the .mov container) pretty much rules for casual / leisure recording.
I just got back from vacation with my family and recorded both H.264 and mlv video. The mlv video runs into 100's of MB and I don't know what to do with it now. I have been searching for a good workflow to sort the files out and render the video's but nothing seems easy. But then ML is not for those that are not ready to toil and sweat it out.
I consider (IMHO) raw_rec as a better option than mlv_rec (if you are not relying on raw_rec for sound recording). I also prefer H.264 over ML video (both raw and mlv) in most situations. If I want to record professionally then the extra effort will be worth it ... else H.264 quality if fine for now.
Most frustrating is not being able to use premier pro to collate the dng's as this is the first and essential step for a rewarding dynamic exchange between after effect and speedgrade.
My workflow for now is:
1. View the mlv's on 'mlvrawviewer'
2. Convert mlv using 'mlvconverter' (now want to use 'raw2cdng' but struggling a bit with mlv conversion). I think hereafter I will work more with raw_rec rather than mlv_rec.
3. Collate the files on AE CC and then transfer on PPRO CC for editing. Bring back to AE CC for effects /Speedgrade for grading, etc. and finally render with PPRO CC.
Everything else seems to be a waste of time.
Edit: I read that PPRO CC supports Blackmagic's version of DNG. Is it possible for the developers to somehow tinker with the Canon metadata during conversion to replicate the BM DNG specs. This will then make them readable to PPRO CC. Adobe has really let us down. DNG is an Adobe creation but .......
Quote from: chmee on February 03, 2014, 11:18:23 AM
@canon eos m
the more you tell about the problem, the faster you can get real help :)
* if you drag/drop mlv-files, the tool starts immediately with converting to raw.
* these temporarily converted raw-files should lay inside the raw2cdng-folder. you see them?
* after that you should see the filename and main specs of the footage in the application-filelist. do they? are they ok?
* they behave as normal raw-files (because they are :)
But, i need some help of you, guys. i'm kind of desperated, because of the adobe-behaviour (http://forums.adobe.com/message/6014956). my goal was to get a functional import into premiere (still it is) - importing into speedgrade and resolve isn't anymore a problem, it works. i thought of writing a standalone-2tif-converter(16bit) with built in ssd-demosaicing (http://www.ipol.im/pub/art/2011/bcms-ssdd/). biggest drawback is the monstrous filesize-increase, but the usability straight inside premiere could be worth it. by now, i'm stucked, and ml-raw seems to me like a "proof of concept" with a big impact on the professional vendors/scene/market, but less benefit for the workhorse, doin its job. simply i think, cdng is/was the wrong choice.. actually i rather go h.264-vanilla because of the instant workability.
so finally, whats your thoughts about efficient workflow? staticaly converting with ACR is not (imho).
regards chmee
Still no luck chmee! I am missing a crucial step here. Not sure what else I can do. But will keep trying ;) till I get it. Will let you know once I succeed. :)
@canon eos m
can you confirm some of the steps i described?
* if you drag/drop mlv-files, the tool starts immediately with converting to raw. confirm dragging and dropping
* these temporarily converted raw-files should lay inside the raw2cdng-folder. you see them? cannot see these filed in the raw2cdng-folder or the output folder
* after that you should see the filename and main specs of the footage in the application-filelist. do they? are they ok? cannot see these in the application
Quote from: Canon eos m on February 03, 2014, 04:12:17 PM
* if you drag/drop mlv-files, the tool starts immediately with converting to raw. confirm dragging and dropping
* these temporarily converted raw-files should lay inside the raw2cdng-folder. you see them? cannot see these filed in the raw2cdng-folder or the output folder
* after that you should see the filename and main specs of the footage in the application-filelist. do they? are they ok? cannot see these in the application
Error message:
MLV Dumper v1.0
Mode of Operation:
- Input MLV files: 'E:/5DM3'
- Convert to legacy RAW
- Output into 'M01-1554.RAW'
[ERROR] Failed to open file 'E:/5DM3'
Edit: Problem fixed. It was something to do with my directory. Shifted to another directory and now mlv's are converted with ease :) Thanks!
Edit 2: Another problem. The files are all pink in ACR.
(http://dl.phreekz.de/raw2cdng/raw2cdng_desc.jpg)
For ACR-Applications (AE,Photoshop), Speedgrade and Resolve please use
*(5) 16bit linear
*(6) no partymode, Gamma 1.0
*(4) enable and fill in the according Model ID for right profiles. Examples (http://photo.net/casual-conversations-forum/00aJ4B)
*(+) for Speedgrade you have to use a calibration-lut, i put one example into the zip.
For Premiere CC (works with Resolve as well):
*(5) 12bit linear
*(6) partymode enabled, Gamma 1.0
*(4) Model ID not needed.
Should work.
Wow! Worked wonders! Thanks a million tons! ;D
Edit: Files okay in ACR but pink in PPRO CC.
Hi chmee, have you seen this thread of a new MLV converter : http://www.magiclantern.fm/forum/index.php?topic=10198.0
I think that you and tonybeccar should join force to unify both of your apps.
since MLV got not the sound + this new MLV converter, many people are moving to .MLV
This converter is working fine with just few bugs with proxies. What this converter is missing is the partymode that you created! and it misses dragndrop also.
I guess just like MLdevs are working together to make ML, you guys could work together to make a great app for the ML community ;)
Yes, of course i've seen. quite happy that he's doin' in the time i stuck because of a lot of work. But..
The reason i cant help there is (i think) his modular way. Giving the MLV to mlv_dump, recreating DNGs with the adobe tools or Prores with ffmpeg. To get it working with premiere, you have to recalculate the raw-data and write own TIFF-Tags into the DNG-Header. By now i dont know if the adobe-converter is able to do that.
But, i appreciate and respect his work really.
regards chmee
Hey chmee and Malex, thanks so much for the appreciation!
As I said in my thread, I am NO programmer whatsoever!! Hehe.. I'm just a filmmaker who happened to do a little programming as a kid! So by now using Visual Basic wasnt that hard to understand. And chmee, you are absolutely right:
As an excited filmmaker I found this raw discovery AMAZING, yeah, as we all. So I can see that there is still work to be done, holes to fill, etc. So I decided with my little knowledge to try to make a workflow app, and yes, it points out to other programs, everything points out to either mlv_dump, dng converter or ffmpeg. My app is simply joining ALL the forces together, creating a chain and uniting the workflow. I understand that you chmee had programmed the converter from scratch, and that's very different. But maybe, my modular system can be embraced with a command line tool for cDNGs, if they could be extracted from the MLV and I get access to a command line, then we're grand. But of course, the best solution would be to implement it right inside the program.
If I'd to tell you the aim to my app, is to have just ONE app for magic lantern (excuse the ambition, hehe), but really, not to complicate things anymore.. just one app that can transform those hot MLV files into anything you want and you're ready to edit. And well I noticed that with cDNGs you get pretty much an easy workflow with Premiere.. anyway.. will be willing to help in what I can!!
Think about it ;)
Cheers!!!
I'd like to take the opportunity to thank both of you and the rest of the authors, programmers, and members of this community for all they have done and continue to do. ML is an amazing piece of work!
Yes, MLV2CDNG is the "Holy Grail" for us ML users (.MLV format) and I await the day when I can just drop files onto a GUI and open up the converted files in PPro CC without any glitches or pink cast. I was considering hiring one of those whiz kid programmers from freelancer.com and just have them write the program.
:) i took a test yesterday with a 5DIII-Build from Feb01.2014 (65572fa87394a2598ef4ee75c547b750d46c6963) (http://builds.magiclantern.fm/#/) - MLV-recorded, raw2cdng 1.48 with 12bit linear and partymode gamma 1.0 looked quite good - i was not expecting this.
online: mlv-file with a testchart - 141MB - ~100 frames 1280x720
http://dl.phreekz.de/M09-0151_testchart.zip
drag,drop,do - import into Premiere CC - worked..
* actually i should implement audio-output.. done.
* and i'm sitting on better dng-colormatrices.. fi for speedgrade. (fi D50 is not implemented)
regards chmee
it would be so great if we can sync the audio and cdng automatically in resolve
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
raw2cdng v1.4.9 is out. MLV-audio-implementation. please test.
mlv_dump.exe (feb08?!) with audio-support from g3gg0 is inside the zip.
* audiofile is found in the same path as dng's
* after conversion, in premiere you just have to import the dng-sequence, the audiofile will be merged automatically.
toDo for 1.5
* native mlv-implementation (but not that important, because we'll get all news from mlv_dump :)
* calibration-luts/matrices could be better.
* maybe whitelevel should be adjusted, you try and report, i hope.
* multithreads optimizing.
* things you report.
@andy_kh
resolve's got a strange (better: simple) behaviour with framerates. You have to "re-set" the project-setting to use another framerate - maybe that's the reason.. in premiere it work's well.
regards chmee
wow!! this is amazing. one problem with me.. i can drag and drop mlv to raw2cdng from my memory card but i cant do it from hard drive. i wil do more test. raw2cdng v.1.4.9 is fantastic. this wil make my worflow very fast. thanks a lot chemme.
try another pathes or partitions, just for tests and report, if it changes..
fi raw/mlv-files on root, and export-path just one path deeper..
tried the new convertor, works well . I got all white frames when I selected 12bit log bmcc
in premiere pro.
yeah it works when i tried another partition
@reddeercity
thanks for info, please try 12bit linear with partymode/g1.0
@andy_kh
like that :)
with the 12bit log bmcc I adjusted gamma to 1.0 & use partymode
that was better. can see the image now
Edit: that much better 12bit linear partymode + gamma 1.0
looks very close to ARC .
dual iso support plz!
[dualISO] will look on that. but, be sure i will use the cr2hdr.exe-converter from a1ex..
regards chmee
Quote from: chmee on February 10, 2014, 12:50:25 AM
[dualISO] will look on that. but, be sure i will use the cr2hdr.exe-converter from a1ex..
regards chmee
nice to hear that, one constant problem i have doing most of dual iso video post processing is flickering (sometimes really bad, sometimes subtle)
Hey chmee, so I finally tried your program, your workflow, AND IT IS AWESOME!!!! :D Amazing, simply amazing... the quality you get inside premiere pro with its rudimentary color correction tools it's the same as ACR, suuuuuper cool!
So was doing color correction and encounters pink edges on the picture, when you push the colors and add sharpen especially, here are 2 frame grabs, original cdng and after the grade:
Before: https://www.dropbox.com/s/5oy8fyl6bhsycnf/Before.tif
After: https://www.dropbox.com/s/ecrznjbfu1p7puk/After.tif
Do you know what's happening? I did cDNG 12 bit linear, tiff header thingy and partymode gamma 1.0.
Thanks and awesome work mate!
PS: Changing topic.. as I installed the new premiere pro, I noticed that there is an option to encode prores in windows, WOW? It doesnt work for me as I had to downgrade quicktime because it didnt let me import image sequences in ae. But oh my, is it really?!
Quote from: tonybeccar on February 10, 2014, 06:07:15 AM
I installed the new premiere pro, I noticed that there is an option to encode prores in windows, WOW? It doesnt work for me as I had to downgrade quicktime because it didnt let me import image sequences in ae. But oh my, is it really?!
?? ProRes on Windows in Premiere Pro CC ???
Where are you seeing this. I have been working in there and don't see this.
The only way to encode prores on windows it though FFmeg in a standalone program
like "Cinemartin" or 5DtoRGB (I use that for all my h264)
@tonybeccar i cant really differentiate between recalculating-problems on my site and dng-importer/handling-failures on adobe-site anymore.. i've seen such problems with bmcc and bolex-files as well..
!!!! important to know: when working with cdng-files, you should only use 32bit-filters - the rest is 8bit and will destroy all advantages of more bitdepth.
@arturochu i didnt used dualISO yet, so we will see, if i can handle that :)
regards chmee
raw2cdng v1.49 is very slick. It is fast! raw2cdng>>ACR >>batch adjust/save>>Import from PPro media browser>> done. So far this is the best option I have found, works really good chmee. I use 12/16 bit linear/party mode/Gamma 1.0, works good, no pink cast files look good in PPro. One question, is this supposed to work with .mlv files also? If so, how to convert .mlv?
Very nice! Good work.
Here are my tests with v.1.49...
Convert RAW files from SD card:
16 bit linear, party mode(ON) Gamma 1.0 = Result - no issues.
12 bit logarithmic, party mode(ON) Gamma 1.0 = Result - Strong green cast.
12 bit linear, party mode(ON) Gamma 1.0 = Result - no issues.
10 bit logarithmic, party mode(ON) Gamma 1.0 = Result - no issues.
16 bit linear, party mode(OFF) Gamma 1.0 = Result - no issues. Faster conversion speeds!
12 bit logarithmic, party mode(OFF) Gamma 1.0 = Result - Strong green cast. Faster conversion speeds!
12 bit linear, party mode(OFF) Gamma 1.0 = Result - no issues. Very fast conversion!
10 bit logarithmic, party mode(OFF) Gamma 1.0 = Result - Slight pink cast and lots of added noise.
Question: Does v1.49 work with .mlv files? I tried dropping a couple on the GUI but nothing happened. I have mlv_dump.exe in the raw2cdng folder as well. The conversion speeds are very fast on my 1280x720 15 second RAW/MLV clips. This is definitely a time saver.
One other thing, I noticed a few cold/dead pixels in my raw/mlv files, wondering if that is from the conversion or my raw settings?
@duey101
[MLV] it should. mlv_dump.exe (with audiosupport) is inside the zip. If you drag&drop mlv, you should generally see a question by windows, if it can open mlv_dump.exe. Please enable logging and look into the logfile. maybe it turns out, you have to copy the mlv-files first onto the hdd instead of converting straight from the card.
[coldpixels] i cant decide, from where the pixels are coming. Do you have tested 16bit/without partymode? (these files are openable in speedgrade, rawdigger or photoshop/acr) i do not recalculate the raw-data in this setting..
@chmee, holly molly I didnt see your new version yesterday.. will try it now! Abd oh, too bad then, but good advice, will try the 32bit filters then..
I wonder what news about mlv_dump we'll be seeing... :) And also I hope you make the command line a priority!!!
Thanks SO much chmee!!! NOW we have a workflow! ;)
Quote from: reddeercity on February 10, 2014, 06:29:31 AM
?? ProRes on Windows in Premiere Pro CC ???
Where are you seeing this. I have been working in there and don't see this.
The only way to encode prores on windows it though FFmeg in a standalone program
like "Cinemartin" or 5DtoRGB (I use that for all my h264)
Yeah, it IS true! I'm running 7.2.1 (4).. the file formats are all different now.. I have a lot of image sequences for example.. and inside Quicktime MOV I have this:
https://www.dropbox.com/s/eicb6qmz1aya7pl/prores.jpg
But whenever I hit export I get a compiler error... maybe I screwed things up myself.. but it IS there!
Man... all I can say is thanks! This converts my .RAW files with lightning speed. I'm still not sure on the dead/cold pixels issue. I noticed they are on all my files whether shot in MLV or RAW. I'll try some other settings tomorrow. I'm shooting in MLV/RAW on a 60D with...
1280x720
16:9
fps override on 24fps
Global Draw off
Frame skipping off
Preview - Hacked No Prev
Extra hacks - On
The pixels appear in both MLV/RAW files. You can see here...
(http://www.dueymoore.com/pixels.jpg)
The other issue is with MLV files. Here is what my raw2cdng window and raw2cdng folder contents looks like...
(http://www.dueymoore.com/raw2cdng.jpg)
But if I try to drag an MLV file onto the window nothing happens... although I can see a DOS window flash briefly as soon as I drop the file in, but nothing happens then, the file does not drop onto the GUI and Win 7 x64 doesn't ask any questions. Not sure why I can't get the MLV files to work. Any ideas?
The RAW files work perfectly as far as I can tell, pull right up in ACR for batch DR correction, and edit smoothly in PPro CC 7.2.1 (4).
@duey101 i bet, the mlv_dump does show a "file not found" error, maybe of a longer path-tree or unusual symbols in pathnames.. Please try to rename the rawcdng-folder. All points to underlines.
[deadpixels] please try another converter (fi raw2dng) and examine the pictures with rawdigger, lightroom or something else. just to clear, if its sensordata or software-creation-failure.
regards chmee
I moved the folder to my root D:/ rive and now it works. Renamed the folder to raw2cdng only and that seems to have worked, takes a lot longer to convert MLV>RAW though! But the RAW to CDNG is really fast! :D
raw->dng will be faster next versions :) (with mlv, please forgive me, i take the advantages of the reference-implementation of g3gg0)
Here is some interesting stuff...
Convert with raw2dng, clean no weird pixels
(http://www.dueymoore.com/raw2dng.jpg)
Then convert with raw2CDNG... 12/16 bit Partymode On and Off same results, funky pixels.
(http://www.dueymoore.com/raw2cdng1.jpg)
And then I loaded the same file in PPro direct with Drastic's plugin installed again (after uninstall) and I get this... funky pixel hell!
(http://www.dueymoore.com/premiere.jpg)
Hmm....
the screenshots are all from premiere? resolve? photoshop? (fi acr uses algorithms to hide these pixels, my dng's and the drastic-importer have no "repair-sensels"algorithms..)
The first two are from Lightroom 5, I converted the raw's in raw2dng/raw2cdng and opened both files in LR5 and took screen caps. The last is from a single frame export from the PPro timeline.
my premiere doesn't even have quicktime mov option! i'm running latest version, i would love to export to prores in windows, i think you have something installed in your pc that activated those codecs.
Quote from: tonybeccar on February 10, 2014, 05:34:39 PM
Yeah, it IS true! I'm running 7.2.1 (4).. the file formats are all different now.. I have a lot of image sequences for example.. and inside Quicktime MOV I have this:
https://www.dropbox.com/s/eicb6qmz1aya7pl/prores.jpg
But whenever I hit export I get a compiler error... maybe I screwed things up myself.. but it IS there!
okj i got it, you can export from premiere cc in windows IF you install mediareactor, then a lot more codecs will appear, including prores, i've already donde this but actually i can export to prores with no problems! :)
Quote from: arturochu on February 10, 2014, 09:20:45 PM
okj i got it, you can export from premiere cc in windows IF you install mediareactor, then a lot more codecs will appear, including prores, i've already donde this but actually i can export to prores with no problems! :)
Lol, yes installed it and uninstalled it.. another victim of the mediareactor greatness..
Your converter is looking VERY NICE!!! After more testing today I have found that to convert MLV files with raw2cdng they need to be in the same directory as the raw2cdng program and that needs to be on a root drive for example, D:/raw2CDNG works for me. At least this seems to be the case for me. While processing the MLV files this window pops up...
(http://www.dueymoore.com/window2.jpg)
But the files finish converting (slowly) and then appear in the raw2cdng GUI, after that they convert really fast to CDNG. The mlv_dump process is the only bottle neck in this method of converting MLV files. Here are some more test images from today.
raw2dng, no issues. (4:1 magnification)
(http://www.dueymoore.com/0raw2dng.jpg)
raw2CDNG, very slight problem on a few pixels, or maybe blown highlights or something? Almost undetectable. (4:1 magnification)
(http://www.dueymoore.com/0raw2cdng.jpg)
MLV files converted through raw2cdng...
File 1
(http://www.dueymoore.com/mlvconverted1.jpg)
File 2
(http://www.dueymoore.com/mlvconverted2.jpg)
Both MLV conversions are crispy clean, no pixel problems. I completely uninstalled DMR and Preview, not sure if that had anything to do with it. But while it was installed I did some more conversions and got this clean raw2cdng file when opened in Light Room 5...
(http://www.dueymoore.com/raw2cdngLR5.jpg)
But then when I opened the exact same .dng file in PPro I go these pixels showing up...
(http://www.dueymoore.com/raw2cdngAPP.jpg)
Still testing...
@duey101 thx a lot for your testing parcour.
(/me) important toDo: native mlv with splitfile-support :D
* Are these corrupted Sensels static?
* did you tried rawdigger with both files? (raw2dng vs. raw2cdng)
* are you using a proper modeltag?
* if you want, send me the test-dngs to
[email protected]regards chmee
* Are these corrupted Sensels static?
Static as in do they move around when the video is played on the timeline? No they do not move but remain where they are. What does Sensel actually mean? (sorry, new to these terms)
* did you tried rawdigger with both files? (raw2dng vs. raw2cdng)
I never used that program but downloaded just now, have no idea what it does. LOL!!! When I open both files in Rawdigger the raw2CDNG file looks okay, the raw2DNG file looks pinkish.
* are you using a proper modeltag?
I'm not sure, this is how it looks when I convert...
(http://www.dueymoore.com/rawwindow.jpg)
I got that model name off of some list that I think was EXIF data for specific cameras, where do I get the correct name for a 60D?
In some of the RAW files I get these pink corrupted frames (2 or 3 per clip).
(http://www.dueymoore.com/pinky.jpg)
After converting several formats, H.264>>ProRes (with Bigasoft Converter) Cineform>> (GoPro Studio) and a few others I have tried, these CDNG files are the best and edit in real time seamlessly even with effects dropped on them and in full res. The native H.264 files are horrible to try and edit with and even ProRes files bog down in 1/2 and 1/4 res. The CDNG workflow is the way to go. It is really smooth in PPro editing wise so far. Glitch free scrubbing and playback at full res, on my machine anyway. You don't have to import all the clips in as a sequence or anything like that, just convert and drag into the project bin from the Media Browser in PPro.
You are on the right path, and close with this. The best thing is how FAST it converts the RAW files! Fast! MLV is another story because of that bottleneck, so yes, native MLV may be the way to go.
Oh... on some clips I get weird color banding in PPro like in this image (3X magnification) screen shot from PPro single frame export.
(http://www.dueymoore.com/colorband.jpg)
But doesn't appear if the same CDNG is opened in Photoshop. Only looks like that in Premiere.
thank you for the motivational words :)
[static sensels] The Name Sensel comes from the fact, its the "pixelated" part of a sensor, but not a pixel, because 2/3 of its destinated value is calculated. Wiki says, its a blend of "sensor" and "cell". Part of an array of sensors.
[Rawdigger] has one big advantage, its good to analyze raw data - so if there are real dead pixels, they should be present in both files, as rawdigger do not reconstructs and refines the picturedata.. it seems, its the case.
(http://www.upload-pictures.de/bild.php/50065,deadsenselsrawdigger5SGA0.jpg)
[banding][moiré] do you record in zoomed mode? these "problems on hard contrasty edges are logical because of the bayer-structure - in premiere more than in ACR/Lightroom (..better debayer-algorithms). BUT i've been asked messages before if its depending on my tool. Yes, it seems, my recalculation makes it more obvious - and No, its already there..
[corruptedFrames] Oh.. maybe there is an "coincidence" with the amount of splitted parts? 2 damaged Frames with 3 SplitFiles?
my 5 cents. regards chmee
Okay that's good to know and thanks for the education. When that dead pixel shows up in the rawdigger does that mean my sensor actually has a dead pixel in it?!!! :o Or is is that during the recording in raw a dead pixel happens? I hope my sensor isn't being damaged. Still new to all this technical stuff.
Those shots with the color banding were shot in 5X crop mode so I get how that would effect the image. Overall, when I shoot raw on my 60D I can get 400-420 frames in MLV mode and about 7-10 seconds in RAW mode @1280x720 which scales up quite well to 1920x1080.
As far as the pink corrupt frames not sure what the cause of that is.
Do a simple CR2-raw-Picture and load it into rawdigger. Look, if the deadpixels are present. And Yes, Dead or Hot-Pixels/Sensels are typical - but in your case its a lot ;) If its really that much, i would contact your distribution.. i won't believe, the local dealer will handle that, but another higher instance of canon.. as a gesture of goodwill maybe..
ahh, btw. these corrupted Frames are not caused by conversion, but seems to be a "problem" with too high recording-datarate.. MLV is a little bit slower than raw_rec.
http://www.magiclantern.fm/forum/index.php?topic=10257.0
I loaded a raw CR2 file and didn't see any deal pixels so what does that mean? Why do they show up in the RAW/MLV converted CDNG's? I don't get it. ???
just one more test.
Do they show up, if you're using the reference converter? fi. raw2dng, mlv2dng, mlv_dump?
Do they disappear, if you're using lower resolutions? (datarates)
I'm not soo into it, to decide now, where the failure's coming from.. sry.
I just ran Rawdigger and used a picture of a white wall (off white) and then blew it up in Rawdigger. Looks like this...
(http://www.dueymoore.com/pixeltest.jpg)
Other than these wht/grey colors there are no black/hot/weird looking pixels, they all look like this so I think my camera is fine, no dead pixels or other colored pixels at all.
Let me test the other options and get back to you.
Okay, after another battery of tests here is the result.
raw2dng - no issues
raw2CDNG convert raw file, no issues, convert mlv file no issues.
mlv2dng - no issues.
All the files are clean including the 10X crop. Now...
If I take the raw2CDNG file and open it in PPro, drag it to the project window, open it in the source monitor, blow it up to 400%... THEN I see the dead/black/sensel issues. Look here...
(http://www.dueymoore.com/ppro-deadpix.jpg)
But outside of PPro these pixels from the same DNG/CDNG files are not present. I sent you the converted files in email. I don't know with the mlv2dng or raw2dng files cause they don't open in PPro (because they are not Cdng I assume). EDIT: And if I output the sequence from PPro the black/dead/sensels show up in the output video, but oddly enough not in the files outside of PPro.
One more test, with the new version of MLVConverter 1.8.2....
http://www.magiclantern.fm/forum/index.php?topic=10198.0
In this test the output dng file is clean, but the tiff proxy file from the exact same conversion shows the same dead/cold/pixels/Sensels but ONLY in the proxy tiff files, not the dng file output. They are also red and not black. Any ideas on what is happening? Very odd indeed.
DNG output...
(http://www.dueymoore.com/same_dng.jpg)
TIFF output...
(http://www.dueymoore.com/same2_tif.jpg)
i dont have such problems
[caution speculation] i'd say, because the dng's are made by adobe, there is a badpixel-tag. i dont know what it does, but it seems havin a positive effect. the same i'd say from the cr2-files straight out the body, where canon sets this tag. duey, please send me a cr2-file from your cam, just for testing purposes..
first i will optimize the actual things ->mlv with splitfiles<-, then i'd take the time for the bad-pixels opcode.
thx & regards chmee
Hey chmee, one question. I was looking into the awesome workflow of cdngs in premiere.. and the people that have the bmcc import their sequences in premiere, just like us, BUT, in the project panel, when they right click on a cdng sequence and they go to Source Settings, it opens in ACR and they can tweak things in there, close it and it updates on Premiere. Dreamy! Now.. I cannot seem to do it with your cdngs, the option is greyed out (though I haven't tried with others anyway), but I have everything up to date, premiere, phosohop, adobe camera raw (8.1 though, couldnt find the 8.2). Do you have this feature? I saw it and it is truly amazing...
PS: Please command line command line! :D I was thinking, with your app, then this is the way to go.. so when you have the command line ready i'll focus more on the interface than anything else.. drop the common dngs, ffmpeg proxies, they are no use really, slow and low quality and tedious.. but instead just make a neat interface that interlaces with your magic software.. I can put my efforts on that...
Anyway, does the premiere feature work for you?
Cheers!!!
Quote from: chmee on February 12, 2014, 05:37:11 PM
it seems havin a positive effect
Yep, that's why I'm going to remove it from raw2dng ;)
Quote from: a1ex on January 12, 2014, 08:55:15 PM
In theory, the current solution (tagging the bad pixel in DNG files) is more elegant. In practice, only ACR reads these tags.
@tony could you give me a link to bmcc-files, where this sequence-settings are available? i've got some here, but its greyed out. this is a great finding, and would make a huge step for us.
[commandline] please (all) give me the time to primarly reach the magic milestone, raw/mlv/audio straight for premiere. then i'd recode all into a commandline.
@a1ex lOl. ok, i should examine your way with the banding correction and the stucked-sensel correction.
Quote from: chmee on February 12, 2014, 06:12:31 PM
@tony could you give me a link to bmcc-files, where this sequence-seetings are available? i've got some here, but its greyed out. this is a great finding, and would make a huge step for us.
[commandline] please (all) give me the time to primarly reach a first finish line, the magic milestone.. then i'd recode all into a commandline.
@a1ex lOl. ok, i should examine your way with the banding correction and the stucked-sensel correction.
Here I saw it in a video: http://www.youtube.com/watch?v=oALUwhZaK4s
I found sample dngs of the pocket camera here (my god how good it is!): https://www.dropbox.com/sh/44bnza1ooi34xqb/6KlnDHKWpy I wonder if they are useful.. gonna test them myself now! But yeah, that workflow is HEAVEN.
EDIT: http://philipbloom.net/2012/08/23/grading-the-blackmagic-cinema-camera-dngs-in-photoshop/
There you have BMCC sequences. For some reason I couldnt import the BMPCC single frames into Premiere.
Have you tested doing the post color in Speedgrade?
And please! Of course, first the MLV support! :D
Thanks so much and cheers mate!
ah. thats a video with the beta-cdng-importer from cs6 - this was crushed & buried long ago. i was wondering why/how the gpu-powered cdng-importer in CC actually should support ACR.. nope.
Quote from: chmee on February 12, 2014, 07:16:48 PM
ah. thats a video with the beta-cdng-importer from cs6 - this one was crushed long ago. i was wondering why/how the gpu-powered cdng-importer in CC actually should use ACR.. nope.
I know it's outdated.. but you think they removed that feature?? Is there a reason? It was a good feature.. :/
I just downloaded the BMCC shots and it's still the same, grayed out. I do not have ACR 8.2 though, only 8.1, I can try and search for it...
Also, I notice that the dngs from the BMCC play faster in premiere.. and also when I play them in 1/4 or 1/2, they don't get that "greenish" look they have with your cdngs.. On full quality they look ok, but as soon as you hit play and quality isn't full the colors shift...
Will research more on that one.. to see if it's possible.
Quote from: tonybeccar on February 12, 2014, 05:48:02 PM
when they right click on a cdng sequence and they go to Source Settings, it opens in ACR and they can tweak things in there, close it and it updates on Premiere. Dreamy! Now.. I cannot seem to do it with your cdngs, the option is greyed out
The same for me, this option is greyed out. You can see in this R3D tutorial http://vimeo.com/52037147 (http://vimeo.com/52037147) that he uses this option of source settings in PPro.
I don't get why the DNG's came out without bad pixels but the TIFF's did? I just ran this program http://www.starzen.com/imaging/deadpixeltest.htm (http://www.starzen.com/imaging/deadpixeltest.htm) on RAW files I took with the lens cap on, VF covered, ISO 100, and 1,2,5,15 second shots. Converted those to uncompressed TIFF's in LR5. Then tested those TIFF's with that Dead Pixel Test program at the default threshold setting of 60 and then halved that to 30. No issues, 0 dead pixels / 0 hot pixels.
Anyone know a better way to test for dead pixels? I don't see any in any images I shoot in RAW/JPG.
CR2's sent.
When this gets sorted out it will be amazing! Working with what I have right now from raw2cdng is very nice in PPro! Glitch free playback and scrubbing, very smooth on my Toshiba with 7200 rpm HD, 8 gigs ram, GEForce 540M GPU (which I tricked PPro into thinking it's a CUDA card). Very good work flow and fast.
Thanks to chmee and tonybeccar! :D
Quote from: duey101 on February 12, 2014, 08:10:13 PM
When this gets sorted out it will be amazing! Working with what I have right now from raw2cdng is very nice in PPro! Glitch free playback and scrubbing, very smooth on my Toshiba with 7200 rpm HD, 8 gigs ram, GEForce 540M GPU (which I tricked PPro into thinking it's a CUDA card). Very good work flow and fast.
Thanks to chmee and tonybeccar! :D
Lol... please, thank chmee ;)
Truthfully, I thank every individual on this forum. :D
Hi there, I don't know if I understood correctly but on the magic lantern facebook page it says :
"we've fix pink frame on the 5DIII and 60D"
Does it mean that RAW files coming from the 5DIII won't need the "partymode" anymore ?
I dont think so. I assume, its about erroneous files and/or frames, that can be restored..
http://www.magiclantern.fm/forum/index.php?topic=8237.0
for disabling the partymode, we have to debias the blacklevel and expand the values to the full-level of destinated bitdepth. means blacklevel = 0 whitelevel = (2^bitdepth-1). i doubt, its possible inside the body.
regards chmee
In the case with my 60D I think these corrupted pink frames as I posted earlier in the thread come from too slow of read/write speed for the camera itself so the frames get corrupted. This happens on 3-10 takes so far for me on 1280x720 and my max frames are 400-415 so far in MLV mode and up to 15 seconds in RAW mode, and that is probably the best the 60D can do. I'm going to have to save and get a 5D III!
I just discovered my sensor was filthy! So I gave it a good cleaning with Pec pads and Eclipse so now it is clean, I'll try to shoot some more RAW and see if that had anything to do with the results I got.
chmee: you get my RAW files?
yes, got it. but i'm now in geneva for work. will code here on raw2cdng as well, but hadnt the time to inspect your cr2's..sry
A couple of more questions... I cleaned my sensor, reshot RAW/MLV then took out my ML card and loaded a normal SD card (Sandisk 32gig 45mb/s') and then shot some plain old H.264 clips. I loaded the new RAW/MLV clips into the timeline in PPro and marked the offending pixels locations with yellow circles (zoomed in 400X). In all RAW/MLV files they appear in the same location.
(http://www.dueymoore.com/evilpix1.jpg)
(http://www.dueymoore.com/evilpix3.jpg)
If I load the same SOURCE MLV/RAW files into PPro with Drastic Media Reactor's plugin I get the same pattern but the pixels look entirely different (pluses and colored squares).
(http://www.dueymoore.com/evilpixdmr.jpg)
The same pattern appears in any RAW/MLV conversion or source MLV/RAW file dropped into PPro via CDNG conversion or DMR's plugin.
Now the part I don't understand. Next, I loaded the regular old Canon video H.264 clip and pixel peeped it at 400X also and I see no funky pixels at all, anywhere! Neither in the marked positions that the MLV/RAW files show them or anywhere else in the H.264 clips. My question is, if my camera has bad/dead/hot/other messed up pixels then why don't they show up in H.264 footage?
(http://www.dueymoore.com/evilpix2.jpg)
I don't think it is really sensor problem even though I don't know anything about it, but in my case I got those pixel bug on many of my MLV with sound footage I had to use ACR to make them got away and convert my dng in AE (but it didn't work out with all my footage).
The point is I never had those bad pixel with Raw_rec, so I'm sticking with it for now.
Thanks Malex. I get them with both, MLV and RAW.
Okay next test, I used raw2dng_escho.exe and got clean dng files (not cdng). As those won't open in PPro, they do open in After Effects so I loaded them up in AE. No funky pixels. My camera is fine I now understand. Keep on going, testing, winning!
Quote from: duey101 on February 14, 2014, 06:14:20 PM
Okay next test, I used raw2dng_escho.exe and got clean dng files (not cdng). As those won't open in PPro, they do open in After Effects so I loaded them up in AE. No funky pixels. My camera is fine I now understand. Keep on going, testing, winning!
raw2dng_escho is my go to option at the moment too.
thanks for the hint. here's the thread to the problem: http://www.magiclantern.fm/forum/index.php?topic=9700.0
i put it on my todo-list.
Hello people,
I have no problem with RAW2CDNG, as it is a really great app.
But I have still the problem with the pink highlights in Premiere CC 7.2.1 :-(
And strangely, only in Premiere! In After Effects, and in the ACR Pink higlights are not seen.
What must adjust in Premiere?
It does not matter what I set in RAW2CDNG, only In Premiere I have the Pink Horror. :-(
greeting
Quote from: Wartburg-Entertainment on February 15, 2014, 06:53:45 PM
Hello people,
I have no problem with RAW2CDNG, as it is a really great app.
But I have still the problem with the pink highlights in Premiere CC 7.2.1 :-(
And strangely, only in Premiere! In After Effects, and in the ACR Pink higlights are not seen.
What must adjust in Premiere?
It does not matter what I set in RAW2CDNG, only In Premiere I have the Pink Horror. :-(
greeting
its not premiere problem. magic lantern problem. alex have been working hard to fix it. he succesfully fix for 5d mark iii. im sory i got u now. make sure you check the partymode to get rid of it.
@Wartburg-Entertainment
What camera, RAW2CDNG settings are you using? (It helps to put all your specs in your user profile signature). I had that same issue in 1.4.8. In 1.4.9 I do not. Latest version is... http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.zip (http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.zip)
My settings are like this...
(http://www.dueymoore.com/raw2cdng_window.jpg)
I get no pinkish output but still the hot/sensels problem exists.
Hi guys,
I use on my 7d currently still Tragiclantern because MagicLantern is not stable.
And I have version 1:49 of RAW2CDNG.
And believe me, I've got all the settings through 10,12, 16bit - with and without Tiff Model.
Only Premiere shows this pink highlights!!!
Quote from: Wartburg-Entertainment on February 16, 2014, 07:44:29 AM
Hi guys,
I use on my 7d currently still Tragiclantern because MagicLantern is not stable.
And I have version 1:49 of RAW2CDNG.
And believe me, I've got all the settings through 10,12, 16bit - with and without Tiff Model.
Only Premiere shows this pink highlights!!!
as i said make sure you check partymode
CDNG's Without Party Mode? Really?
Edit:no Partymode - brings no Improvement.....
Make sure you have Gamma at 1.0, check the box for partymode and then try moving any files you want to encode into the same folder as raw2cdng, I think that is what I did, might be worth a try. Also use the latest nightly build. I was having that same problem and I checked/unchecked partymode and tried 10,12,16 bit and all that but I think it was when I set the Gamma to 1.0 is when it fixed it. Make sure about the Gamma setting.
I've always gamma 1 and level the problem only in Premiere.
Even in the media browser, the small preview files are all pink.
Will now test the build of Magic Lantern nighly, let's see
what it is so new insights there.
regards
for some reason I can not make it work by dropping MLV file even from another partition/disk .
I can see the Command line window for a moment but nothing happens.
What am I doing wrong ?
Thanks
I have the same problem. Windows 8.1
The contents of your folder should look like this...
(http://www.dueymoore.com/raw2cdngfolder.jpg)
I named my folder raw2CDNG, and it looks as above. I dragged one MLV and one RAW file in there and it started working from then. You need to have mlv_dump.exe (http://upload.g3gg0.de/pub_files/fb196456f6a6389763cab895fb6067f8/mlv_dump.exe (http://upload.g3gg0.de/pub_files/fb196456f6a6389763cab895fb6067f8/mlv_dump.exe)) in the same folder otherwise you'll just get that brief command window popup. In my testing you need to have everything all in the same folder including the MLV/RAW files...
Set your settings (as in the previous picture, then with all the contents of your folder looking as above, drag the MLV file onto the raw2cdng window, it should pop open the command window and look like this...
(http://www.dueymoore.com/processingraw2cdng.jpg)
See if that helps.
no success
and it doesnt even show the file name in the Batch list
(the Op system is Win 7 64bit)
Quote from: gideonplus on February 16, 2014, 03:48:38 PM
no success
and it doesnt even show the file name in the Batch list
(the Op system is Win 7 64bit)
copy and paste the mlv file to another drive and try drag and drop from that drive... you dont have to copy everything. copy only the mlv file
I did try that and it doesnt work
Okay, well it certainly works and I am on the same OS (Windows 7 Ultimate x64 SP1). Here is a video screen capture to show usage...
So just keep slugging away and you'll get it to work. Your missing something somewhere but I don't know where. Can you drop a .RAW file into the GUI? Are the Output path and Tif Model tag check boxes checked? Look at the video and duplicate what you see and it should work as you can see it does.
1. Download raw2cdng 1.4.9.zip http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.zip (http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.zip)
2. Extract it into a folder called raw2CDNG. You will get 2 files in the zip, mlv_dump.exe and raw2cdng.1.4.9.exe.
3. Copy one MLV and one RAW file into the same directory as the installed files.
4. Double click on the raw2cdng.exe file, the GUI opens.
5. Drag your MLV file from the same folder as where the program is onto the open raw2cdng GUI (program window).
6. The mlv_dump.exe command box should open and start extracting the MLV and then when it is finished you will see the RAW file in the GUI. Then all you need to do is export it where you want to.
I can drag and drop files from the desktop, SD card, C drive, D drive, it all works.
Yes it works OK only with the RAW file
Watch the video. ;)
Works on a different disk. Sorry.)
please describe it with some details, so i can fix it. on my computer i tested from different places, and it worked all the time, no matter if desktop, root or some other directories.
* filesystem (ntfs? fat32? static or dynamic hard drive/partition?)
* language?
* os?
* whole directory-chain?
if its not too intimate :)
OK its Win 7 64bit ntfs static same/different drives
* language English US
Command line window just for amoment and no change on UI
example Log file:
14.02.16 19:34:29 -- ----------- Application started ------------
14.02.16 19:34:29 -- - gamma loaded and set: 1
14.02.16 19:34:29 -- outputPath: E:\Gideons Pictures\2013_12_31
14.02.16 19:34:29 -- uniqueName: ML CDNG1.4.6 PHR
14.02.16 19:34:29 -- loading LogLists for 12bit
14.02.16 19:34:29 -- setting arrays for 12bit-Loglists
14.02.16 19:34:29 -- found and enabled mlv_dump
14.02.16 19:34:29 -- loaded GUI 1.4.9
14.02.16 19:35:20 -- **-- filesList_DragDrop started
14.02.16 19:35:20 -- * dragdropped File E:\Gideons Pictures\2013_12_31\M15-0911.MLV
14.02.16 19:35:20 -- is mlv-File and will be converted E:\Gideons Pictures\2013_12_31\M15-0911.MLV
14.02.16 19:35:20 -- converted to raw
14.02.16 19:35:20 -- is raw file
14.02.16 19:35:20 -- getting RAW-Attributes
14.02.16 19:35:49 -- **-- filesList_DragDrop started
14.02.16 19:35:49 -- * dragdropped File E:\Gideons Pictures\2013_12_31\M15-0918.MLV
14.02.16 19:35:49 -- is mlv-File and will be converted E:\Gideons Pictures\2013_12_31\M15-0918.MLV
14.02.16 19:35:49 -- converted to raw
14.02.16 19:35:49 -- is raw file
14.02.16 19:35:49 -- getting RAW-Attributes
Finaly NOTHING is done in output
..mainly "file not found error" is the problem, caused by white spaces (or other "not so fine" characters) in directory- or filename.
* small fix - raw2cdng 1.4.9.2
- mlv-convert-process-output written into debuglog
- put filenames into quotationmarks in commandline-process
http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.2.zip
i assume the space in
E:\Gideons Pictures\2013_12_31\M15-0918.MLV
will be the problem.
@chmee I have been doing lots of testing and also testing for pink frames on my 60D and now it seems to have been fixed. Did some more experimenting with conversions. Still getting the odd pixels/sensels in raw2cdng but if I convert with raw2dng_escho.exe the files are perfect. Here is a raw2dng_escho file for your reference (Canon 60 D / MZ 5X mode / mlv_rec.mo @ 1280x720)
http://www.dueymoore.com/000000.dng (right click/click save link as)
OK you have nailed the problem (the fix doent help it by the way but I have renamed the Dirctories)
some new thing:
in Premiere my futage the was 25fps was showing like it is 24fps
so I have checked the DMG metha data and in the FrameRate Field
it was 24fps which is wrong and should be 25fps
Another question that I have is:
Does in Resolve 10, the dng-sequence and audio file will be recognised automatically and in synch , As Well ?
Thanks
Quote from: gideonplus on February 17, 2014, 12:13:54 PM
OK you have nailed the problem (the fix doent help it by the way but I have renamed the Dirctories)
some new thing:
in Premiere my futage the was 25fps was showing like it is 24fps
so I have checked the DMG metha data and in the FrameRate Field
it was 24fps which is wrong and should be 25fps
Another question that I have is:
Does in Resolve 10, the dng-sequence and audio file will be recognised automatically and in synch , As Well ?
Thanks
no it wont. only c2dng syncs that too only in premiere pro. c2dng is the best and i always use in all my projects since u can cut directly in premiere pro.its even better now by syncing audio which makes my workflow much easier and fast
@gideonplus thats correct, i nailed it to static 24fps, because there've been more important things to solve - and changing framerate is the less difficult problem in fi premiere.. its the same with timecode - i could use "timeofday" but by now i start simply at 01:00:00:00
by the way: could you show a debuglog with mlv_dump-error with version 1.4.9.2? i thought, this could solve the problem :)
regards chmee
@ chmee
Can you please help me with my Premiere CC CDNG problem.
So the MLV files all come from a 7D with last Tragic Lantern.
I can open the DNG in Premiere, but the settings of ACR are not included in Premiere.
And now the main problem, as already described here I have only completely pink in the highlights and in the over-exposure.
Today I again several MLV transformed with the new 1.4.9.2. - No improvement. Is it Windows 8.1 or Tragic Lantern or or ...?
There no ACR support in premiere pro cc right now, dynamic link to A.E. or speedgrade
is still there , but time consuming. Better to use A.E. first save the project xml import that
by dynamic link in to Premiere Pro CC . :)
@wartburg-entertainmant
i feel a liitle bit sorry.. 7D is another workplace (i'm using and testing 5DIII), and as you stated, tragic-lantern-mod is much older, so you cant get all benefits from actual versions, it can hit you right there.. simply, i cant help you.
@a1ex you stated in another thread i'm neither giving source open (via fi github) nor i respect gpl-copyrights. i worked on that, solved that and gave all informations, to code fi premiere support more than on time. finally, its a free software. that's not really fair..
@chmee: Thank you anyway, even if it is not good news.
@reddeercity: Thanks for the idea. I also think that it is currently the best workflow.
@chmee: it wasn't directed at you, it was a doubt about all the other closed-source apps from this forum that do (more or less) the same thing as raw2dng.
This is an open source community, we share the knowledge with everyone else, so I think it's fair that everybody who uses our software and expands upon it should share their knowledge instead of hiding it in binary blobs.
I understand you are not using any of my code, so there are no GPL issues. But what about the fundamental aspect of open source, collaboration?
The part about raw2cdng was that a user asked me to fix a bad pixel problem in your program, I told him your program does not appear to be open source, so there's nothing I can do.
Sorry for the misunderstanding.
Quotefinally, its a free software
Since we do not have the freedom to see your source code, to study it or to reuse bits from it, your program does not count as free software (http://www.gnu.org/philosophy/free-sw.html), sorry. And the same is true about all the other closed-source programs (freeware != free software).
@a1ex ok. understandin' - so let me reorganize this codechaos and then..
Thanks, no need to hurry. That's also a practical aspect of open source - since you upload it in the open, you tend to be a bit more careful about code quality, readability and so on (so it's a good idea if you are trying to sharpen your skills).
raw2cdng 1.4.9.3 online
* second attempt fixing this directory-filenotfound-thing
* modeltag and uniquemodel are separated (@Andy600 told about problems in cinelog (http://www.magiclantern.fm/forum/index.php?topic=10151.0))
* optimizing some things (fi tiff-dng-tag-creation)
http://dl.phreekz.de/raw2cdng/raw2cdng.1.4.9.3.zip
regards chmee
Hi,
Thanks chmee for great work....
Im totaly not familiar with how to make apps,..
Just a question ... in future ... will it be available for OSX also?
Thanks,
Peter
@peterbrne
actually not from my side, i m Mr ignoramus in terms of mac-world:) so, just hope for the day, i will opensource the code and others will use that for cross-compilations. :)
hi.. may i please have the link/instructions of the latest tutorial on resolve premiere pro roundtrip? or the new premiere pro cdng to resolve workflow? thanks..
there are so many ways to do that. import cdng in premiere pro.cut n export xml. import xml in resolve. do your grading and export xml for premiere pro and now you can add titles, effects etc
hey thanks.. but xml could not be imported in resolve "the file M17-1805.000000.dng" can't be found" pops up... i should do some renaming. Please explain that stuff so that resolve can import the xml from ppro...
no. you dont have to rename it. open your project. click go to media page and click "davinci resolve" on top right. click preference. click "+" and add your folder where your cdng is located. there are so many basic tutorials on youtube
has any faced problem with the following workflow? CDNG to resolve and edit all the footage in resolve itself, color correct and export it to premiere pro (with timeline xml) for perfect trim, transition effects and do mastering from premiere..
since the beginning this was one of the favoured workflows.
* first we manipulated canon-raw to 16bit to work in resolve9
* we found the right switches for xml-roundtrip
* then resolve 9.15 came with native support of 14bit-dng's
the drawback was and is the complete import and recode of the footage, just to work with it in other software. time and space-consuming.
Quote from: reddeercity on February 18, 2014, 12:25:17 AM
There no ACR support in premiere pro cc right now, dynamic link to A.E. or speedgrade
is still there , but time consuming. Better to use A.E. first save the project xml import that
by dynamic link in to Premiere Pro CC . :)
Could you please explain your workflow a little more?
Quote from: kaco on February 25, 2014, 07:45:58 PM
Could you please explain your workflow a little more?
Sure no problem its very simple, open Cdng's and or dng's image sequence
in After effects ,now you have access to the ACR panel save your project as cs6 xml
import A.E. Project in to your premiere pro project.
I posted a short video tutorial , link below
http://www.magiclantern.fm/forum/index.php?topic=8177.msg86395#msg86395
Hope this helps :)
Quote from: reddeercity on February 25, 2014, 09:31:10 PM
Sure no problem its very simple, open Cdng's and or dng's image sequence
in After effects ,now you have access to the ACR panel save your project as cs6 xml
import A.E. Project in to your premiere pro project.
I posted a short video tutorial , link below
http://www.magiclantern.fm/forum/index.php?topic=8177.msg86395#msg86395
Hope this helps :)
big thanks
@chmee
Question: Is the mlv2cdng app going to work in the same way as the raw2cdng app? What I mean is will we still have to convert from MLV>RAW or will it be drop MLV files straight into the GUI and convert to CDNG. The one thing I like about raw2cdng is the SPEED! It is blazing fast on my system.
If i implement mlv, it will be native.. And also the speed should stay equal. I'm thinking about a processing module, for cs2x2 for example, but it would lower the Speed massively
Would you consider posting the LUT or the formula used to convert 12-bit linear RAW to 10-bit log ?
I am still trying to figure this out.
Cheers!.
@zcream
i convert from 14bit to 10bit/log. you find the discussion about lin/log in the early posts inside this thread, IliasG helped me out.
http://www.magiclantern.fm/forum/index.php?topic=5618.msg47480#msg47480
beside that, try it with a simple lin/log-conversion as described in a lot of math/physics-forums..
http://www.physicsforums.com/showthread.php?t=304633
here are the old calculations by IliasG
14lin10log_gamma_rec709.zip (http://www.file-upload.net/download-8689468/14lin10log_gamma_rec709.zip.html)
@all
mlv/audio native is on the way.
regards chmee
Quote from: chmee on March 07, 2014, 12:47:35 PM
@all
mlv/audio native is on the way.
Sounds good =))
@all
mlv/audio native is on the way.
that wil be great. for me i stil wil have to convert to raw first to remove pink dots since i use 650D now.
the pink dot removing costs a lot of speed - but i still think about a optional button for that.
Quote from: chmee on March 07, 2014, 02:03:59 PM
the pink dot removing costs a lot of speed - but i still think about a optional button for that.
oh wow!! that wil be fantastic if there is optiional button for removing pink dot. no other application wil be needed in the workflow if you do so. drag and drop all mlv files in raw2cdg. remove pink dots and convert to cdng. this wil be the best application for sure
@all
mlv/audio native is on the way.
This look promising, +1 for the pink dot button.
Just a little bug report (sorry if this has already been posted):
On Windows (8, 64-bit), the drag-and-drop functionality breaks if the path of the dragged file contains spaces (mlv_dump gives error).
E.g.:
C:/this/path/works/video.mlv
C:/this/path/does not/work/video.mlv
In the latter case, mlv_dump would say that "C:/this/path/does" does not exist.
@CrW What Version you're using? Please try 1.4.9.3 and tell if its still apparent..
http://www.magiclantern.fm/forum/index.php?topic=5618.msg102531#msg102531
Quote from: chmee on March 09, 2014, 12:14:36 AM
@CrW What Version you're using? Please try 1.4.9.3 and tell if its still apparent..
http://www.magiclantern.fm/forum/index.php?topic=5618.msg102531#msg102531
I was about to report same bug. With 1.4.9.3 it works for me.
Quote from: chmee on March 07, 2014, 12:47:35 PM
@all
mlv/audio native is on the way.
When developing native wav extract, could you please test importing the final CDNG into Davinci Resolve? For me the DNG sequence and the WAV file is always being imported as two separate files. :-/
Quote from: chmee on March 09, 2014, 12:14:36 AM
@CrW What Version you're using? Please try 1.4.9.3 and tell if its still apparent..
http://www.magiclantern.fm/forum/index.php?topic=5618.msg102531#msg102531
Using 64bit win 8.1 here, no response when attempting to drag and drop a MLV file on version 1.4.9.3 of the converter, am I missing something? :'(
Edit:
I can convert from MLV to RAW first then use raw2cdng and it works - sometimes (all crash logs saved) - but there seems to be a huge loss in quality when importing to premiere for editing, compared to the other option of converting MLV's to DNG's with Mystic then importing to After Effects, Any ideas? ;/
@AD DP
The difference is the DNG-Wrapper. Premiere uses another code than AFX. Because i dont have Win8.1 i cant check, whats wrong there. Please wait some more days for native MLV-Support, then give it another try.
Quote from: AD DP on March 10, 2014, 05:39:25 AM
Using 64bit win 8.1 here, no response when attempting to drag and drop a MLV file on version 1.4.9.3 of the converter, am I missing something? :'(
Edit:
I can convert from MLV to RAW first then use raw2cdng and it works - sometimes (all crash logs saved) - but there seems to be a huge loss in quality when importing to premiere for editing, compared to the other option of converting MLV's to DNG's with Mystic then importing to After Effects, Any ideas? ;/
I'm also on Win 8.1
The 1.4.9. version was not working with spaces in directories. With 1.4.9.3 it works for me (although the windows with mlv_dump will stay black during the extraction process).
Quote from: chmee on March 10, 2014, 09:18:00 AM
Please what some more days for native MLV-Support...
Wonderful =))
Quote from: chmee on March 10, 2014, 09:18:00 AM
@AD DP
The difference is the DNG-Wrapper. Premiere uses another code than AFX. Because i dont have Win8.1 i cant check, whats wrong there. Please wait some more days for native MLV-Support, then give it another try.
@chmee
That's Great! I did have success converting to RAW and then using the program to make CDNG's, just not with the MLV's. Cant wait for the updates! Thank you for all your hard work. :)
raw2cdng v1.4.9.5 is online
* native mlv+audio
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
PLEASE! i wont believe, that all is working fine. so, test and tell all bugs, glitches, problems. my wish is to get a stable 1.5 version..
I'm a total newbie so my questions may be quite simple but have a few doubts about the raw2cdng workflow and quality implications:
1. To edit directly in PPro (with no pink cast) I just extract the cdngs setting bitdepth at 12/linear as I hve read in chmee's blog. Everything goes smoothly but my question is: is there any quality loss vs extracting at 16 bit and then grading in resolve and sending back to premiere via xml for edit and back to resolve for export (or using the 16 bit extract at ACR)? (I'm not a pro in any way, so the quality I get with the 12/linear export in PPro is more than enough for me...but just wanted to know...)
2. When I open the CDNG sequence in premiere for color grading I use the speedgrade direct link but when I've triedto color in resolve (via xml from PPro) I get the problem that the CDNGs have embeded a 24fps frame rate. In premiere I just interpret footage as whatever it actually is (i.e. 25fps) and sequence is adjusted accordingly. But if I export that 25fps sequence as xml, Resolve just does not import the clips as it says the clips do not match the sequence timecode (I guess the dngs have a fixed 24 fps timecode embedded). ANything I'm doing wrong? If not, how can I change that "internal" fixed 24 fps setting?
Thanks
Quote from: chmee on March 12, 2014, 11:25:57 PM
raw2cdng v1.4.9.5 is online
* native mlv+audio
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
PLEASE! i wont believe, that all is working fine. so, test and tell all bugs, glitches, problems. my wish is to get a stable 1.5 version..
link to 12.03.2014 – v1.4.9.5 – native mlv-support is downloading v1.4.9.3 first glitch.. :P
just ve seen it :) fixed.
@melimelo
(1) i didnt recognised any real disadvantages on 12bit versus 16bit. primarily i left the 16bit because of acr and resolve-users. and some saying, its a disadvantage by numbers :)
(2) will look deeper on that.
Quote from: chmee on March 12, 2014, 11:33:10 PM
just ve seen it :) fixed.
@melimelo
(1) i didnt recognised any disadvantages on 12bit versus 16bit. primarily i left the 16bit because of acr and resolve-users. and some, saying, its a disadvantage by numbers :)
(2) will look deeper on that.
Thanks for your extremely quick reply...have downloaded the new version, I guess this native feature must be a big improvement (as I've read you've been working on this for quite long) but could you elaborate for "illiterates" like me what are the main improvements/advantages of this version of this vs previous versions? Thanks!
PS: the only thing I may add is that in the new version the default tif model tage still says version 1.4.9.3...not much help but that is the most I can contribute ;-)
:) main advantage is the native mlv+audio support. before that i had native raw-support and mlv+audio was done thru chaining to external app (mlv_dump). it was a double copy/recode-process. i just wanted to do an app, you're able to transcode straight from the card, without copying and merging splitted files, with batch support (dropping more than one raw/mlv-file) - support in premiere, because thats my favoured edit-app - and finally something with a kind of speed. i dont like the timeconsuming process with acr/photoshop/lightroom/aftereffects, nonetheless its still the way to get the best quality.
Quotestill says version 1.4.9.3...not much help but that is the most I can contribute
thx.
so is there a option for 23.976? or its just 24.000fps
Quote from: chmee on March 12, 2014, 11:25:57 PM
raw2cdng v1.4.9.5 is online
* native mlv+audio
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
PLEASE! i wont believe, that all is working fine. so, test and tell all bugs, glitches, problems. my wish is to get a stable 1.5 version..
@chmee
I think there is a loss in quality when going the CDNG route, but it may also be my inexperience with utilizing speed grade and not really being able to pull all the detail back out. Here are the results I was able to get using the previous version 1.4.9.3 by going the MLV>RAW>CDNG then importing into Premier for my regular workflow:
https://vimeo.com/88723935 (https://vimeo.com/88723935)
Shot on the t2i with the most recent nightly build [v2.3 March] at 1152x482 with a 2.39 aspect ratio, also at 1080p / 24 frames sec selected in the canon menus. Stretched just abit (14%) to fit. Not amazing quality, but eh for the t2i its pretty good! and its leagues better than the h264. :)
Will continue some testing [this time on my 5DIII] and post results. Excited to see the new version, will definitely give it a shot with my next test!
Quote from: chmee on March 12, 2014, 11:25:57 PM
raw2cdng v1.4.9.5 is online
* native mlv+audio
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
PLEASE! i wont believe, that all is working fine. so, test and tell all bugs, glitches, problems. my wish is to get a stable 1.5 version..
Great work, thanks!
In my case i am having problems with FPS detection (the MLV was shot as 23.976, in older versions it worked OK).
(http://i59.tinypic.com/2dlyx4.jpg)
Next problem is that the resolve is importing this CDNG as 24p (bud I can change this via Clip attributes). But the audio will always import as separate file and it is few seconds longer.
I assume this is some MLV stuff and not a problem of CDNG?
(http://i60.tinypic.com/n5ovuh.jpg)
Hi everyone ! :)
Yes, I have the same problem than Kaco with the fps detection. Also: how do you activate the chroma smoothing function ? (Because it's "disabled")
Thanks
(1) fps-read from mlv. on the list. (something since yesterday while coding :))
(2) according framerate (also timecode) into cdng. on the list.
(3) chroma smoothing - not useable now, on the list for 1.5
(4) audio length. please check, if theres really async or if the audiopart is just longer.
edit:
6074 frames / 24fps = 4min 13sec 2frames
6074 frames / 23.976fps = 4min 13sec 8frames + ~6 frames audio often recorded by mlv (seen in another thread)
(5) audio/video resolve - maybe a nameproblem (according to the path/filename-thing found entries ago)
regards chmee
Quote from: chmee on March 13, 2014, 02:34:48 PM
(5) audio/video resolve - maybe a nameproblem (according to the path/filename-thing found entries ago)
probably a timecode problem.
OK, I found I way how to sync audio and video in Resolve. You can do this in Media Pool by syncing the clips viac timecode.
Read page 122 from this document: http://software.blackmagicdesign.com/DaVinciResolve/Manual/DaVinci_Resolve_Colorist_Reference_Manual_Nov_2013.pdf
Now the problem is, that all my clips from CDNG has Start timecode 01:00:00:00. I have to manually change clip attributes to 00:00:00:00 otherwise the sync won't work. I can't change this for all clips in one click, I have to do this for every DNG sequence.
@chmee: Could we somehow change the start TC in raw2cdng?
You can see the wrong start TC column here:
(http://i60.tinypic.com/n5ovuh.jpg)
(5a) :D thy will be done. static sync-TC set to 00:00:00:00 (because no one ever said something about it)
and/or
(5b) button switch for TimeOfDay?
and/or
(5c) instead of wav -> bwf (wav with timecode) timecode in bwf is strange thing.. no.
regards chmee
Quote from: chmee on March 13, 2014, 03:36:06 PM
(5a) :D thy will be done. static sync-TC set to 00:00:00:00 (because no one ever said something about it)
and/or
(5b) button switch for TimeOfDay?
regards chmee
Tested it with bigger project. Manually changed 50+ clips to 00:00:00:00 and applied Resolve's "Auto-sync Audio Based on Timecode" function and it doesn't work very well. It links wrong audio and video files from time to time. So, maybe the ML does not add correct (same) timecode to audio and video? Or is this raw2cdng problem?
I know it works good in Premiere (probably based on filename), but for many the Resolve is more important.
For now the only working solution for audio/video sync in Resolve is to manually select every audio/video and click chain icon to link the files:
(http://i58.tinypic.com/2uny14z.jpg)
(quite annoying if you have 100+ clips)
because plain wav cant hold timecode, its fixed to 00:00:00:00. so, the problem should be managed in other ways. it could be the naming-definitions for path and mediafiles inside. will look on that. has something to do with the clipNo, that should be in directory and filename. (will investigate)
Directory : M021234_C0001
Pictures : M021234_C0001.[0000-9999].dng
Audio maybe : M021234_C0001.wav
regards chmee
Quote from: chmee on March 13, 2014, 03:58:58 PM
because plain wav cant hold timecode, its fixed to 00:00:00:00. so, the problem should be managed in other ways. it could be the naming-definitions for path and mediafiles inside. will look on that. has something to do with the clipNo, that should be in directory and filename. (will investigate)
Directory : M021234_C0001
Pictures : M021234_C0001.[0000-9999].dng
Audio maybe : M021234_C0001.wav
regards chmee
Thanks! I can provide you working BlackMagicDesign CDNG if you like.
We're getting there!
For me it didnt work at all.. the frames, fps and camera model tags are wrong. The fps seems like it adds the previous value, so you end up with a huge number in the end. Fps shows 1.00.. maye that's the multiplier factor in mlv metadata? And when I hitted start, "raw2cdng stopped working". Oh also, i dragged and dropped 62 mlv files.. short ones, 9 seconds each.. and it took very long to add. Also, it added some wav files that were hanging around because of mlrawviewer.. it'll be wiser to filter it only for mlv's (and raw's).
But anyway, simple mistakes.. it's looking very promising!!!
Myself I'm coding at the moment.. making an interface update, to include dual iso, black image substraction, full mlv support and much much more.. please tell me you didnt forget about the command line? ;) Cause then it will be perfect!!!
Cheers chmee, thanks so much for the hard work! :)
Tony.
it seems like i cant add for than one mlv. its works if i add only one but if i do add two or more mlv and hit start "raw2cdng" stop working
ok, thanks for reports.
(1) framesCount reset . todo.
(2) Fps-show 1.00 - fixed
(3) right framerate in cdng - fixed
(4) modeltags are not taken from mlv, by now. todo.
(5) chaos-wav's - cant get the problem. think, will be fixed with other filename-definition.
(6) 69 mlv are a lot - every mlv has to be opened, properties have to be read and written into dngheader into ram while dragdrop-action. so, no great wonder.
(7) raw's dont have any wavs - because there is no mlv_dump-action anymore.
(8 multiple mlv's and raw. todo.
(9) command line maybe in 1.5
regards chmee
@g3gg0 and @a1ex
its strange, it doesnt fit to the definition of droppedframes: you write 23976/1000 into Nom/Denominator for 23.976fps. but by specification it should be 24000/1001. honestly saying, both leads (in my case) in premiere to 24fps. (page 10 in cdng-specs 1.1 (http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/devnet/cinemadng/pdfs/CinemaDNG_Format_Specification_v1_1.pdf))
(fixed: 0x0a instead of 0x05 for C764)
regards chmee
chmee,
I tested v 1.4.9.5 but it keeps crashing after outputing a folder with 4-5 frames. Same files I tested extracted ok using raw to dng converter by tonybeccar.
I tried doing it one by one. Sometimes it would convert a file and in another it would crash.
I'm trying to test how long it would take to covert files from a single 64gig cf card comparing it to raw converter 1.9 to dngs.
For the Dng converter it took me 29 min. I'm trying to see if converting via raw2cdng would be faster.
edited:
additional info,
I am using 7D and all files are MLV.
no new bug to report but additional evidence for crashes commented on posts above:
1. unable to extract several mlvs together, always crashes
2. individual files, sometimes works, sometimes crashes (when it stops, it has already created the extract folder with the first 4-5 DNGs already extracted)
I'm on windows 8 (not 8.1) with mlvs coming from 5D2
pink in highlights as in previous versions (at least for me), only visible in premiere pro, not in ACR or Resolve
thanks for your work
Hey chmee, strange.. in my converter I also look for like 6 fields of metadata while adding, and it's extremely fast.. maybe it's the method you're looking for the bytes? I always look for an exact byte, I know there's a command that searchs all bytes in the files which takes forever.. anyway, you're an awesome programmer so I might just be talking nonsense.. hehe
tony, thats not the point. i#m not an awesome coder, i just have ideas and try to realize them, often with a lot of oldschool-sequential-code :) i decided to read the whole file-blocks (VIDF/AUDF-Count) while drag/drop-action - just to get my properties right, when they are in the list. if its that important i'd do them into the convert-part..
btw. 1.4.9.6 is on the way, maybe today, maybe tomorrow.
short info: testing all output-formats on several apps. think, today 1.4.9.6
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
16.03.2014 – v1.4.9.6 – bug fixing
* timecode set to 00:00:00:00
* framerate in dng fixed
* mlv-batchmode fixed
* framesCount in GUI fixed
* 16bit and 12bit linear partymode are ok (in photoshop, acr, resolve and premiere, i think speedgrade as well, not tested)
* 12bit log/bmcc and 10bit are not really useable by now. investigating.
* linking video/audio in resolve via rightbutton link. searching for the according filename-conventions to get it automatically.
Today tried to convert mlv to dng directly with raw2cdng v1.4.9.6 with a canon 5dmk3 and problem continues with pink highlights in Premiere CC 7.2.1
Partymode enabled with gamma 1.0 bitdeph 16 linear
Tiff model tag Canon EOS 5D Mark III
The view in ACR looks OK and audio is in sync
Magic Lantern version v2.3.NEXT.2014Mar11.5D3113
I don't know what i'm doing wrong. Please let me know
The only problem i have is the pink highlights in Premiere cc 7.2.1\
sorry for my bad english
Thanks
Yes, there are still pink highlights in Premiere. the dng-properties seems ok, because in acr, photoshop, afx its all ok.. by now im fiddling about the way to avoid them in premiere, but not to cut too much highlight-data.
all infos & hints about this are highly appreciated.
regards chmee
Quote from: chmee on March 16, 2014, 03:40:20 PM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
16.03.2014 – v1.4.9.6 – bug fixing
* timecode set to 00:00:00:00
* framerate in dng fixed
* mlv-batchmode fixed
* framesCount in GUI fixed
* 16bit and 12bit linear partymode are ok (in photoshop, acr, resolve and premiere, i think speedgrade as well, not tested)
* 12bit log/bmcc and 10bit are not really useable by now. investigating.
* linking video/audio in resolve via rightbutton link. searching for the according filename-conventions to get it automatically.
awesome, thanks chmee!
your tool has been by far my favorite.
i'll be sure to donate to your cause!
btw, would it be possible to import the same cdng footage back into raw2cdng and compress it down even more?
hi chmee, whenever raw2cdng crashes or even when i extract cdngs, there are additional empty folders with somewhat similar name styles created in the same directory. Example: M121643J_C0001_B16LINMAX (when software crashed while extracting .raw file), M121644K_C0001_B16LINNOR (empty folder1 after mlvs were extracted), M121643J_C0001_B16LINNOR (empty folder2, after mlvs were extracted)... the actual folder's name M121653K_C0001_B16LINNOR (contains all the cdngs from mlv file).. am i doing something wrong?
:) because i didnt tested the actual version with RAW, this could be a silly bug. thanks for info.
did you enabled the logfile? please link/show it via pastebin.
are you using a newer firmware with audio-support?
regards chmee
Quote from: chmee on March 16, 2014, 03:40:20 PM
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
16.03.2014 – v1.4.9.6 – bug fixing
* timecode set to 00:00:00:00
* framerate in dng fixed
* mlv-batchmode fixed
* framesCount in GUI fixed
* 16bit and 12bit linear partymode are ok (in photoshop, acr, resolve and premiere, i think speedgrade as well, not tested)
* 12bit log/bmcc and 10bit are not really useable by now. investigating.
* linking video/audio in resolve via rightbutton link. searching for the according filename-conventions to get it automatically.
Not sure if I'm doing something wrong, but I downloaded this version and placed them into my raw folder. I checked "Partymode", 1.0 for gamma, changed output path, and drag my raw files onto the Batchlist. When I click on convert to CDNG, a pop-up appears that says 'raw2cdng has stopped working'. I tried doing it again and change bitdepth and such, but still getting the same pop-up. Any suggestions?
Thanks
@nang
Quotedid you enabled the logfile? please link/show it via pastebin.
Did You used RAW or MLV-Files?
Quotebecause i didnt tested the actual version with RAW, this could be a silly bug.
Quoteare you using a newer firmware with audio-support?
My Debug Log for the instant crash on 1.4.9.6
Used with raw_rec, no audio.
And I have one feature request:
Can we have an option to change the output folder and filenaming?
Something like "Name folder and files after the original .raw" would be perfect.
14.03.19 11:40:01 --
----------- Application started ------------
14.03.19 11:40:01 -- - gamma loaded and set: 0
14.03.19 11:40:01 -- outputPath: M:\140318_Day_15\152
14.03.19 11:40:01 -- uniqueName: ML CDNG1.4.8 PHR
14.03.19 11:40:01 -- loading LogLists for 12bit
14.03.19 11:40:01 -- setting arrays for 12bit-Loglists
14.03.19 11:40:01 -- loaded GUI 1.4.9.6
14.03.19 11:42:14 -- **-- filesList_DragDrop started
14.03.19 11:42:14 -- * dragdropped File X:\140315_Day_13\138\138_M0001.RAW
14.03.19 11:42:14 -- is raw file
14.03.19 11:42:14 -- getting RAW-Attributes
14.03.19 11:42:14 -- ready. got all RAW-Attributes
14.03.19 11:42:14 -- set FileNameShort: 138M0001
14.03.19 11:42:14 -- added to fileData - batchlist
14.03.19 11:42:14 -- found Splitfiles: 1
14.03.19 11:42:14 -- new ListViewItem: 1920x1080 - 2016 frames - 25,00fps - 2047 - 15000 - Canon EOS 5D Mark III
14.03.19 11:42:14 -- added ListViewItems to GUI
14.03.19 11:42:14 -- **-- filesList_DragDrop Ended
14.03.19 11:42:18 -- **-- clicked startConvert and starting Workerthread _doWork
14.03.19 11:42:18 -- **-- _doWork Workerthread started
14.03.19 11:42:18 -- yes, there are Files in the list: 1
14.03.19 11:42:18 -- counting Frames
14.03.19 11:42:18 -- counted AllFrames: 2016
14.03.19 11:42:18 -- changing GUI because of work :)
14.03.19 11:42:18 -- * converting File: 138_M0001
14.03.19 11:42:18 -- changing bitsPerSample
14.03.19 11:42:18 -- bitdepth set to : 16
14.03.19 11:42:18 -- blackLevel Old - New : 2047 - 2047
14.03.19 11:42:18 -- whiteLevel Old - New : 15000 - 15000
14.03.19 11:42:18 -- set and change DNG-Values
14.03.19 11:42:18 -- changed uniqueName: ML CDNG1.4.8 PHR
14.03.19 11:42:18 -- byte Array DNGHeader written
14.03.19 11:42:18 -- set and changed OutputPath: M:\0000_Test\1
14.03.19 11:42:18 -- * Opening File and setting *delta for Splits*
14.03.19 11:42:18 -- - File 138_M0001.RAW split No 0
14.03.19 11:42:18 -- - delta Begin: 0
14.03.19 11:42:18 -- - delta End: 192
14.03.19 11:42:18 -- - framepos: 2016
14.03.19 11:42:18 -- * delta SplitValues done
14.03.19 11:42:18 -- created OutputPath: M:\0000_Test\1\138M0001_C0001_B16LINNOR
14.03.19 11:42:18 -- created helperClass parameter for Threading
14.03.19 11:42:18 -- set MaxThreads to 4
14.03.19 11:42:18 -- started CountdownEvent. starting threadloop for frame-converting.
[raw] ok, thanks a lot. i'll look on the raw-converter-modules - there will be a bug inside.
Quote..Name folder and files after the original .raw..
just give me some days. by now there's on the list "pink highlights" (nearly done) and "raw".
regards chmee
Quote from: chmee on March 19, 2014, 07:19:54 AM
@nangDid You used RAW or MLV-Files?
I tried on RAW files and I'm using August 13th build. Maybe that's the reason why? I haven't updated because everything's been working perfectly since August.
it should work with this older version as well - for sure its a bug in my code for raw-data - but, yours is really old :)
Quote from: chmee on March 19, 2014, 02:09:06 PM
it should work with this older version as well - for sure its a bug in my code for raw-data - but, yours is really old :)
I'll look into updating to a more recent build. My concern is if something goes wrong. The August build has been working flawlessly thus far, but maybe it's time to update. Might give mkv a shot, but I record my audio externally so I never really had a need for this.
I tried version 1.4.9 and for the most part it works, but it crashes randomly.
raw2cdng1.4.9.7 is online pink highlights fix
* this is a testversion, if pinkhighlight-fix works everywhere
* logarithmic approaches freezed
* partymode is now on by default, no click. gamma changes not needed as well.
* raw-bug fixed (fps.denominator was the bad boy)
* downside: it has lost much of its speed. (but: theres space to optimize a lot)
short tests with premiere, acr/photoshop and resolve looked ok. (5dIII)
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
regards chmee
chmee: "because i didnt tested the actual version with RAW, this could be a silly bug. thanks for info.
did you enabled the logfile? please link/show it via pastebin.
are you using a newer firmware with audio-support?
regards chmee"
I am sorry, but what is logfile and how to enable it?
I used .raw files. Also i request for fixing bigger (greater than 6GB) raw files getting crashed while converting.
i am using the latest firmware, both in ML and raw2cdng. I am using raw and not mlv, so no audio. By the way, which is preferable for the converter, .raw or .mlv?? thanks
(http://dl.phreekz.de/raw2cdng/raw2cdng_desc.jpg)
No 7
regards chmee
Hello Chmee,
since the last two versions of the app does not create more folders.
It used to be that I got MLV per file, create a folder.
Now creates a folder and it contained only by a MLV the CDNG.
The others are generated, they are not found. (overwritten)
greeting
?written so complicated..? (try german?!) do you mean the batch-convert is broken?
Yes deutsch, :)
in den letzten Versionen war es immer so, dass ich pro MLV einen Ordner angelegt bekam. Das war super.
Seit 1.4.9.6 und 7 entsteht nur ein Ordner von einem MLV File.
Das Programm arbeitet aber munter weiter und scheint die anderen CDNG's zu erzeugen, aber im Ordner sind nur die Files von einer MLV-Datei.
Habe noch nicht gecheckt von welcher - erste oder letzte in der Batchliste.
Windows 8!!
Gruß aus Thüringen
EDIT: Es sind die Dateien vom letzten MLV-File in der Batchliste!! Es werden die vorgehenden überschrieben. Deshalb kann ich sie nicht finden!
Ok danke. Schau ich mir heute abend an.
tested latest with several MLV files. Seem to work well so far.
version 1.4.9.7 works well with .raw files.. i tried with files greater than 7 GB and multiple files.. great Chmee... thanks
anybody has an idea why the cdngs ignore the white balance I chose?
I get horrible warm images in Premiere independent from which Kelvin I record with.
because i set static values. its a code-module before mlv, that now has whitebalance-data.. long time ago (v1.3.0) i wrote a whitebalance-importer, that takes the values from a cr2-photo. its still inside raw2cdng. but, thanks for the hint, seems, its time to change that.
@wartburg-entertainment
please send me a logfile from a multiple file-convert-process. raw2cdng[at]phreekz.de or use pastebin.com
Quote from: chmee on March 21, 2014, 11:12:04 PM
raw2cdng1.4.9.7 is online pink highlights fix
* this is a testversion, if pinkhighlight-fix works everywhere
* logarithmic approaches freezed
* partymode is now on by default, no click. gamma changes not needed as well.
* raw-bug fixed (fps.denominator was the bad boy)
* downside: it has lost much of its speed. (but: theres space to optimize a lot)
short tests with premiere, acr/photoshop and resolve looked ok. (5dIII)
http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/
regards chmee
Hey Chmee,
Went through another short project using your raw2cdng, no problems here! I had some pink banding on afew frames - but I believe this happened during recording the MLV's. I dropped the affected frames.
Here is the video: https://vimeo.com/89770585 (https://vimeo.com/89770585)
Will be migrating to the 5DIII from here on out. :)
Thanks!
@AD DP
thanks for sharing (and using raw2cdng). if you use lower exposure and control highlights below a point, they havent be seen in earlier version as well. i made tests with dimming lights to see, where the pink highlights begun, and finally tried to investigate where this level is. on this point green is clipped, but red and blue are going further. but to say it once more clear, its the fault of the cheap cdng-code in premiere. its horrible. with this step we have lost highlight recovery. in v1.5 there will be back the "old" 16bit transcoding (beside the highlights-hack). and finally i hope, adobe will do something clever: code the importer as dng was founded, an open format with descriptive metadata inside it.
tested 1.4.9.7 to convert 3 mlv files recorded at 60P and 2 of those files caused the app to crash. I noticed that selecting 12 bit causes the crash. 16bit no problems.
I went back to 1.4.9.6 and all 3 converted fine using 12 bit.
OMG! Where the hell have I been! I just found this today! I just wanted to give a big Thank You!
I've been trying to find some kind of exit strategy for the 19lights GingerHDR workflow I've been using, which just shut down (drag and drop with ML Raw http://19lights.com/wp/tutorials/magic-lantern-raw-video/ was a great solution for a bit, fast workflow with direct import into PP with very few drawbacks)... After that, First option was the ACR workflow with 8bit output and that just wasn't going to cut it (for bigger jobs)... but, now I know I can get solid 12bit cDNG on my timeline (with a pro NEC monitor) and I couldn't be happier. Awesome deal. Great Work man! :D
@ted ramasola
thanks for information. did it converted some of the pictures or does it crash right in the beginning? could you send me first ~10mb of the failing file? (use for example Hxd for cutting that into another file)
@kirklasalle
you're welcome. tell me/us a little bit about the workflow with your nec monitor, primarly about your real advantages. thats a interesting thing. and of course, how the cdng-files behave..
regards chmee
Hey guys, Using 1.4.9.6, I have been encoding with 16bit and 1.0000 gamma, i did some test of encoding with and without 1.0 gamma and it seemed the same, not sure what benefit I am getting from that..
Also when I export as 12bt LOG for resolve, all my files come out pure white, but 16bit linear works perfect. I thought with resolve I am meant to be exporting them as LOG files for my lut's to work but doesnt seem to be working :) (sorry noob post but have searched online for an hour or so and can't find a solution)
Quote from: chmee on March 23, 2014, 10:28:11 PM
@kirklasalle
you're welcome. tell me/us a little bit about the workflow with your nec monitor, primarily;yy about your real advantages. that's a interesting thing. and of course, how the cdng-files behave..
regards chmee
The cDNG's I get are drag and drop which is great... workflow is what you'd expect, import the folder from prelude (when needed, which auto-pulls up media encoder and allows transcode for offline, so you'll need some space and some time, again, IF NEEDED, I say if needed when working between web project @720p and feature length project @ full color gamut... otherwise, just direct drag N' drop folder import into PP CC) all the way through to a final edit and color. [EDIT: btw - only first pass on color in PP, 2nd and final polish in Davinci]
Really the monitor (http://www.bhphotovideo.com/c/product/880645-REG/NEC_p232w_bk_23_Widescreen_Professional_Graphics.html) not that big of a deal, it just allow me to do 10bit color with a wide AdobeRGB gamut and has some function for a basic LUT in the field... the cool thing is your software allows me to choose 12bit , that's what I really got excited about (16bit crashes on me, I'll pull logs if you need) so, I have two 7Ds that I can keep now (no buying a new camera) and get some kickin RAW out of them.
[EDIT: The thing is, right now, for the price, (especially since I already own two 7D's) you can't beat 12bit RAW @ 1080p with the 7D, it saves me on a monthly rental... you deserve a huge (and more of a) thanks! Please don't stop! this is some of the best ML covert software I've seen so far]
I'll donate... I promise that! [EDIT: Although, I need an English version... there's no translate with paypal I guess]
Quote from: chmee on March 23, 2014, 10:28:11 PM
@ted ramasola
thanks for information. did it converted some of the pictures or does it crash right in the beginning? could you send me first ~10mb of the failing file? (use for example Hxd for cutting that into another file)
Its hard to replicate the crash, but I've also made comparison with 1.4.9.6 and 1.4.9.7 and it seems the .6 version produces similar quality with 14bit DNGs side by side in resolve. When I use 1.0 Gamma and NO Party mode. The CDNGs from .7 version is darker and I need to push up exposure in resolve. What is the default gamma for .7 ?
the gamma is 1.0 as well, but i have to lower the values by (assume) ~1ELV to fix the highlights for premiere. in next versions will be back the normal convert without partymode - for usage in speedgrade, acr and resolve.
regards chmee
I am back on 1.4.9.
The new CDNG naming convention interferes with my workflow. Which is to bad since I really liked not having to change the FPS every time I imported into Resolve.
hi, i am trying to open .cdng files in Nuke. With J_Ops plugin, I could only open a single frame of .cdng file in Nuke 7.0. The sequence could not be opened as usual, like in .exr pics. Pls help me out. How do i import the entire .cdng sequences? thanks
anandkamal: Why do you want directly open cdng in Nuke ? ... In the cinema industry: the digital compositors work their shots a lot with the dpx 10 bits log and Exr for the 3d sequences. ;)
I read in the past even with J_Ops plugin, Nuke is not able to read dng.
thanks Olivier57.. that would be a good alternate. Also Nuke is good with .exr. I thought because of quality degradation, converting cdng to dpx10 or exr would be lossy. Anyways, that being a standard, I will have to do that. ;)
hi.. sometimes my footage does not play in MLRawViewer, the screen is fully black, I thought its a mistake and proceeded to convert the .raw to cdngs. The converter shows "0 FPS" and crashes. Is that footage a shooting mistake? i don't think so bcos other footage is good. Am I wrong at something? Why would that happen? (the take I okay-ed is unfortunately showing this error :-[)
@chmee: Any news about the vertical banding (fixed pattern noise)? I've shot a lot 50p footage and the banding is pretty bad. So I have to convert MLV's to RAW's, then export dngs with raw2dng and then have to edit the exif camera model with irfanview batch for ACR -- very slow and complicated process. It would help a lot if you'll have a time to implement FPN correction. 0:-)
..because i went to an x/y-view to the data (instead of simple array rewriting) theres a good chance to implement anti-banding and cs.
@chmee,
does the new fix in raw2dng have any effect on your raw2cdg app?
A1ex fixed a bug posted here;
http://www.magiclantern.fm/forum/index.php?topic=5404.msg109330#msg109330
can someone explain why in after effects is so slower playback of dng?
in davinci i got around 10-15 fps.
in adobe 0.5 i think i even can't scrub trough
apart from the fact that after effects in general is slower (its a compositing software, not a clip-based edit tool), dng is imported by a slow, but high-quality onTheFly-converter (acr).
please be patient, i'm gathering all experiences, fixing and optimizing raw2cdng. there was work as well (echo2014 award) https://www.flickr.com/photos/chmee/
@ted will have a look on it. i never saw this kind of artifacts..
regards chmee
Nice pic's Chmee :)
Is there a way to get rid of vertical noise with this app? I dont get vertical noise in raw2dng but I do in raw2cdng
yeah I'm also having those vertical banding issues.
There's another app (mystic) which has a fix for it, just check the function and bang, problem solved. Perhaps something like it on raw2cdng?
Thanks chmee for the lovely job on this software!
i described it above. to solve vertical banding or deadpixels you have to look on it as a picture (x/y-pixels) not as an array of bytes. i'm on it, but i guarantee, the conversion will slow down. thus it will be an optional button..
(still recoding) regards chmee
by the way. we'll see what advantages the next CC-Version will bring (NAB2014) - maybe the whole work is not needed anymore :)
http://www.cinema5d.com/news/?p=24528
Quote from: chmee on April 03, 2014, 10:48:08 AM
by the way. we'll see what advantages the next CC-Version will bring (NAB2014) - maybe the whole work is not needed anymore :)
http://www.cinema5d.com/news/?p=24528
Still lot of people use CS6 and I think this is the Canon raw from C500, not ML. But we'll see :)
i do not count on the canonRAW-entry in the video, but on optimizing/opening the raw-module. thats all i want :) (but yes, i think about converting ml_raw to arriraw or canonraw, but these tests will come after the recode of raw2cdng). After that (middle of 2017) think i'll find the time to read the CC-sdk to write an native importer :D
Quote from: chmee on April 03, 2014, 10:48:08 AM
by the way. we'll see what advantages the next CC-Version will bring (NAB2014) - maybe the whole work is not needed anymore :)
http://www.cinema5d.com/news/?p=24528
The forthcoming release of Premiere Pro CC and Media Encoder CC will support *some* types of ML-generated CinemaDNG footage but not all, and they have to be real cDNG frames not just a folder of 'stills'. I can't give a detailed matrix yet as the code is still in flux, for example the build I'm testing now has problems with stuff from the 7D and won't read 14-bit files.
... by the way, if folks want me to run a test on a particular 'flavor' of cDNGs, upload them someplace and ping me a link. Don't need more than a second's worth of frames.
Quote from: DFM on April 03, 2014, 01:07:36 PM
... by the way, if folks want me to run a test on a particular 'flavor' of cDNGs, upload them someplace and ping me a link. Don't need more than a second's worth of frames.
Hi DFM,
I use MlRawViewer to export cdng, it opens really good when i import to pp cc and use direct link to speedgrade. but when i import straight to speedgrade it has heavy green cast.
Does the new version has solution to this issue?
For Speedgrade you should build your own Matrix or LUT - i made a Matrix-example in raw2cdng 1.4.5. Use this one or tweak the Matrix, and write them into the file i provided. thats not a real issue.
btw. did @baldand solved the pink-highlight-problems?
Sg in the current version of CC expects slightly different things in footage; DNGs in the other apps previously went through Camera Raw but in the forthcoming version they import natively, and you don't get the ACR dialog if you open the cDNG footage's "source settings" - instead there's a Lumetri quick-adjust tool. That will impact on workflows that use the ACR engine to do heavy lifting (lens corrections, noise reduction, etc).
Here's the dialog being fed a 16-bit linear cDNG from chmee's app:
(http://media.uvsar.com/AdobeCC/PPCC-Lumetri-dialog.jpg)
Quote from: chmee on April 03, 2014, 02:04:11 PM
For Speedgrade you should build your own LUT - i made an LUT-example in raw2cdng 1.4.5. Use this one or tweak the Matrix, and write them into the file i provided. thats not a real issue.
btw. did @baldand solved the pink-highlight-problems?
Thanks chmee i will look for your example, even when i fix the green cast it looks like there is no highlight information to work with.
Regard baldand MlrawViewer, on pp cc only saw them once at high contrast areas.
chmee your matrix made my day!
Quote from: DFM on April 03, 2014, 02:36:04 PM
Sg in the current version of CC expects slightly different things in footage; DNGs in the other apps previously went through Camera Raw but in the forthcoming version they import natively, and you don't get the ACR dialog if you open the cDNG footage's "source settings" - instead there's a Lumetri quick-adjust tool. That will impact on workflows that use the ACR engine to do heavy lifting (lens corrections, noise reduction, etc).
Here's the dialog being fed a 16-bit linear cDNG from chmee's app:
(http://media.uvsar.com/AdobeCC/PPCC-Lumetri-dialog.jpg)
Looks good. It's a start... i guess.
@DFM thanks for the insight - looks like the arriraw-settings window :)
@DavidSh you're welcome
@baldand had an issue entry on bitbucket about pink highlights. i emphasize his opinion, we have to wait for the next CC-Version. my actual fix is not the ideal solution.
https://bitbucket.org/baldand/mlrawviewer/issue/55/premiere-renders-dng-highlights-pink
Quote from: chmee on April 03, 2014, 02:04:11 PM
btw. did @baldand solved the pink-highlight-problems?
Unfortunately not. I'm still hoping Adobe solve this in the next version of PP since it's clearly their issue.
Hi all , there a new update to premiere pro today at least to me. Version CC 7.2.2(33)
I guess with NAB 2014 now in full swing there was bound to be some updates .
Anyways I was trying some cdngs and at least to me it the closes I have seen yet.
I used raw2cdng Version 1.4.9 with 12bit party mode gamma1 it looks right.
With 16bit its still pink hi-lights, so looks like improvement for 12bit , if someone could verify
this the would be great . :)
With the forthcoming versions, 'party mode' isn't required anymore. The screenshot I posted earlier isn't of a 'party mode' file.
@DFM good to hear.
btw. raw2cdng 1.5 is on the way, one or two days to finalize coding and then there will be a beta.
ok. please see that really as a beta. testing, telling. all stuff you remark, problems, ideas and so on.
raw2cdng 1.5.0BETA
* PLEASE before the first start rename/delete the old settings-file in user/[user]/appdata/roaming ->raw2cdng.settings.json
* mlv and raw native (quite fast :))
* audio of course
* changed GUI to wpf (just saw a design-glitch on another pc)
* mouseover on preview ;) the preview is not accurate, its just a preview if the footage is the right one.
* the checkbuttons in the list are not implemented yet.
* options are disabled by now excepted pink highlights (but it doesnt work well by now :) i assume only in 16bit)
* there is a possibly empty field under the bitdepth-buttons. mouseover. you can refine your filename to your own needs..
* only for testing general purposes!
* this version is only linked here
http://dl.phreekz.de/raw2cdng/raw2cdng.1.5.0.beta.zip
regards chmee
how about the ev levels chmee? is it default like 1.4.9.6 ? or lowered like 1.4.9.7 ?
the ev levels just lowered if you're using pinkhighlight-fix, as i assume. by now i wait for the next adobe update - and i hope we dont need any pinkhighlight-fix :) last days i didnt really tested all circumstances.
Had a lots of App Crashes, worked for about 10 file (loaded same file but change settings)
now it just crashes when I try to convert. Delete the old settings-file in user/[user]/appdata/roaming ->raw2cdng.settings.json
to get it to work after it crash, but now it just stop working, the preview work ok and the metadata was visible but no go.
Of what I did get to work I found Cdng 16bit maximized to work the best no pink, but very flat looking but editable.
12Bit maximized worked also but with a gamma shift up and all high lights were pink and "delta pink high lights" didn't make any difference one way or another.
But it wasn't flat it had good contrast.
what's the differences between maximized and non and what doses "delta pink high lights" do ?
Note * it crashes on Raw & MLV. Deleted the .json file again and use 1.4.9 and back to normal for raw.
Windows 7 Ultimate , Toshiba i7 laptop 8GB Ram, SSD. the only error messages I could get are as follows:
"C:\Users\i7 Toshiba\AppData\Local\Temp\WERDA.tmp.WERInternalMetadata.xml
C:\Users\i7 Toshiba\AppData\Local\Temp\WER3784.tmp.appcompat.txt
C:\Users\i7 Toshiba\AppData\Local\Temp\WER3795.tmp.mdmp"
I do like the new look very nice job, the preview very fast. I though came to mind as I'm writing this, the preview stop working on the last 3 files
of MLV so I did not click to preview I just converted but then after a few more it stop working all together.
I also have dual video cards on my laptop (Intel HD400 & NvidiaGeforce GT 630M) Could this be a problem with debayering in preview ?
Good job! :)
:o geez.. ok, i have to code the debuglog into the new version :) it seems, it has to do with the IO-read/write-routines. did you used some heavy subpath-things?
please try it one more time with one path from root and put into the filenamegenerator [S]_ (for shortfilename). just to hear if it helps..
(btw. weeks ago i searched for some filename definitions from other filmcameras, i will optimize the filenamegenerator to this, but its quite possible to write the same filenames with raw2cdng)
http://pastebin.com/bE5jsBUY
Arri like filenaming
[CardNo][ClipNo]_[Date]_[Base36SerialNo]
A004C010_20100610_R1JL.mov
could be in raw2cdng
A001C001_[D]_[S]
regards and thanks a lot chmee
Quote from: baldand on April 03, 2014, 06:17:27 PM
Unfortunately not. I'm still hoping Adobe solve this in the next version of PP since it's clearly their issue.
Wish you would have been at the CDNG NAB meeting on Wednesday. Some good info was being given. Frustratingly though, Adobe failed to send any of their people over. There are some good things potentially coming to the CDNG standard.
i had been busy shooting so couldnt test the new version and finally i hav tested today. it keeps on crashing and i am unable to use the application at all. however i wil love this new application once all the check buttons are implemented. this new version will save a lot of time. i have been using the old version v 1.4.9.3 because i have to convert to raw first and remove pink dots.
please give me some more informations.
* when does it crash? right in the beginning? after drag/drop? while showing preview? when starting to convert?
* maybe once again a path/naming problem?
@ted ramasola [pinkhighlight]
i just made my thoughts about the light-loss because of the highlight-workaround. finally it does not corrupt the SNR, because the values are only mathematically lowered (in 12/16bit). so if you brighten the frame, it should not have more SNR than before.. it "only" loses kind of highlight recovery ability, so it does not affect the simple premiere workflow, because there is no way to recover highlights..
i can drag and drop. after that i cant click anywhere as it would crash if i do so
@andy_kh
would you quote the path, your mlv/raw are saved? i'll do the test here now as well with a bigger directory-tree.
@chmme
i have tried different paths but it dint work and i cant even click the check box to preview. it keeps on crashing
i think (reddeercity) has mention the same problem above
if you dragdrop, its imported into the list and the values are right? resolution, frames/framerate?
but, nonethelesse, i will recode the debug-logging, to get faster informations.
Quote from: chmee on April 14, 2014, 02:53:48 PM
if you dragdrop, its imported into the list and the values are right? resolution, frames/framerate?
but, nonethelesse, i will recode the debug-logging, to get faster informations.
yeah values are right
Amazing changes!!!! Interface is a beauty!! As anotherone said here.. a lot of crashes with the app, that for me have to do with the interface. Most reproducable when adding multiple files.. after that when I scroll, or tick a box, or click a column, crash. The preview is out of this worldddd!!! It'll be just nuts to have the function like premiere, to scrub with the mouse when you move to the right, hehe. One last thing I think this program is missing that is vitally important, is multifolder support, subfolders I mean. When you are on a big shoot you always have your footage organized into folders. Oh and also, the naming convention, very nice! But is it possible to have the option to have the filename untouched? The shortened filename is weird and I never figured out what the last 'random' letter means!
Anyway, amazing stuff!!!
Cheers and thanks chmee!!!!!
thanks.
* Yeah, wpf is a new world for me, i have to figure out, what event is doin the crash.
* i thought about scrubbing over the picture (left frame 0, right last frame), we'll see :)
* subfolders.. hmm. means, you'd change the subfolders per sequence. in this case, you put the mlv/raw files into according named subfolders and use the "source-path alias" inside the namegenerator.. it wont generate subfolders, but it will ease the sorting by name (maybe i find an intuitive way to implement it)
* realFilename comes to the alias list. thx.
* the shortened filename was born out of the fact, EDL-Names should be 8 chars long - and the mlv-name [a-z0-9] is just 7 - the eighth char is calculated from the first 7, means its not random, but not really useable :)
but to be honest. i wont code a commandline - if this ones will work, i'll start coding a premiere native importer-plugin - after some longer tests i have to admit, that the transcoding process is time consuming as well - and in the end not needed..
if someone's got crashes, please pastebin the saved crashreports (not the minidump of course :)) they will be in the temp-path with some strange name.xml. (see above, entry from reddeercity)
regards chmee
@chmee: we have discussed the wav+dng import into resolve, so I know you are working on this. Have you tested this in Adobe AE? When I drop the CDNG folder into AE it imports just the video, not audio in my case. Maybe I'm doing smth wrong or it is not working? But it would save a lot of time if this will work.
i promise i will look on that :)
btw. i just made some small boring tests. recording, transcoding (16bit maximized, no pinkhighlight-fix) and straight using in premiere.
* the transcoding-process is (for me) much too long. as i stated theres only one solution. writing a mlv-importer for AE/premiere, since its the same SDK-Base
* importing the filesequences is a bit annoying, because you have to mark sequence by sequence. have to figure out, if premiere can cope with kind of containers, marking all pathes, thats it.. (would be solved with the importer :))
* but finally premiere works well with the data, i had no pink highlights, maybe i didnt overexposured :)
(vimeo) 1280*448 at 50fps - down to 23.976fps
(youtube) 1920*1080 at 23.976fps
@chmee, it is very slow and complicated process (AE or resolve), but it will give you more production options. For me, the basic color grading tools in Premiere CC are not enough (I think they are more color correction than color grading). So even the process is slow and complicated, it is sometimes worth it.
I usually export with raw2cdng (or MLV converter if I need to fix vertical bandings). Then depending on the type of a project:
1. Create proxies in Resolve (faster then realtime). Import into Premiere. Make my edit, export XML to resolve, create final grade.
2. Create proxies in Resolve. Import into Premiere. Make my edit. Then duplicate the edit in the another layer. Nest every clip. Replace with AE composition (Adobe dynamic link) and import CDNG sequence. Then I make color grading on the first frame with Lightroom, save metadata and reload footage in AE. It will also update the clip in Premiere. Very slow, but higher quality debayer process then in Resolve.
3. Grade with Lightroom, import all into AE and directly export Prores/DNxHD -- all clips graded.
The second workflow is highest quality and with ability to render just the clips used in the final edit with ACR. But very annoying that I have to manually nest and relink every clip to AE.
Still though, I have problems with audio :D You should really look on that. If you have 1TB project with 250 shots, I spend day or two linking just the audio. :D
Also, using Premiere CC to grade raw makes no sense to me when you can use Resolve which is for free. The process is not so much more complicated, isn't it?
@kako
i dont understand why dont you guys import cdng in cc and start your cut? you guys neednt sync audio manaully in premiere pro thats the reason why i love raw2cdng. after your cut you can grade in resolve or after effects without having any problem
@kaco
with cdng editing raw is as easy as editing h264. i wil do a tutorial very soon to show you what i mean
Quote from: andy kh on April 15, 2014, 07:19:26 PM
@kako
i dont understand why dont you guys import cdng in cc and start your cut? you guys neednt sync audio manaully in premiere pro thats the reason why i love raw2cdng. after your cut you can grade in resolve or after effects without having any problem
It's because I am still on CS6. If there would be ACR involved into Premiere CC I would definetely upgrade. But just because of CDNG "proxies" and then also round tripping to Resolve it's not worth the money for me.
If the CDNG audio works with Premiere, I think it might somehow work in AE and Resolve too. If it works with Blackmagic CDNG files. I would say that the Premiere just checks the filenames while AE and Resolve also timecode or what -- that's why the audio and video imports as two separate files.
Quote from: kaco on April 15, 2014, 07:35:12 PM
If the CDNG audio works with Premiere, I think it might somehow work in AE and Resolve too. If it works with Blackmagic CDNG files. I would say that the Premiere just checks the filenames while AE and Resolve also timecode or what -- that's why the audio and video imports as two separate files.
it doesnt work in resolve
Quote from: chmee on April 14, 2014, 07:06:31 PM
thanks.
* Yeah, wpf is a new world for me, i have to figure out, what event is doin the crash.
* i thought about scrubbing over the picture (left frame 0, right last frame), we'll see :)
* subfolders.. hmm. means, you'd change the subfolders per sequence. in this case, you put the mlv/raw files into according named subfolders and use the "source-path alias" inside the namegenerator.. it wont generate subfolders, but it will ease the sorting by name (maybe i find an intuitive way to implement it)
* realFilename comes to the alias list. thx.
* the shortened filename was born out of the fact, EDL-Names should be 8 chars long - and the mlv-name [a-z0-9] is just 7 - the eighth char is calculated from the first 7, means its not random, but not really useable :)
but to be honest. i wont code a commandline - if this ones will work, i'll start coding a premiere native importer-plugin - after some longer tests i have to admit, that the transcoding process is time consuming as well - and in the end not needed..
if someone's got crashes, please pastebin the saved crashreports (not the minidump of course :)) they will be in the temp-path with some strange name.xml. (see above, entry from reddeercity)
regards chmee
Awesome awesome awesome!!!
Wow, an MLV importer would be like... mmhhh there is no real world comparison, lol! Regarding the subfolders system, take a look at how I've done it in my mlv converter... it's pretty straightforward, you maintain alphabetical order and everything. I do not show ONLY the filename when it's multifolder but that info I guess can be stored in a property value somewhere...
Anyway, amazing work man!! Thanks so much!
Cheers!!!
Quote from: chmee on April 15, 2014, 10:45:59 AM
btw. i just made some small boring tests. recording, transcoding (16bit maximized, no pinkhighlight-fix) and straight using in premiere.
* the transcoding-process is (for me) much too long. as i stated theres only one solution. writing a mlv-importer for AE/premiere, since its the same SDK-Base
* importing the filesequences is a bit annoying, because you have to mark sequence by sequence. have to figure out, if premiere can cope with kind of containers, marking all pathes, thats it.. (would be solved with the importer :))
* but finally premiere works well with the data, i had no pink highlights, maybe i didnt overexposured :)
(vimeo) 1280*448 at 50fps - down to 23.976fps
(youtube) 1920*1080 at 23.976fps
Exquisite quality! But what about those green edges?
Quote from: andy kh on April 15, 2014, 07:39:14 PM
it doesnt work in resolve
CinemaDNG + wav from Blackmagic Design cameras work. So it's probably just a question of correct filename convetions and maybe some metadata.
@chmee -- I can provide you some BMD footage for the "reverse engineering" if you are interested.
@kaco (for sure) you can help with the BMD naming conventions:
Pathname
+--->dng-filenaming
+--->wav-filenaming
ill try to rewrite/implement the filenamegenerator, as every can construct this filenaming-convention for himself :)
@tony in both videos? please tell video/timecode..
ahh. PLEASE DONT FORGET TO GIVE ME SOME INFORMATIONS ABOUT THE CRASHES. (pastebin the xml-files) its difficult to find this bloody bug. i tried it on 4 pc's here at home, it does not crash :)
The app opens, I then proceed to drag in a .RAW file... I click convert, the filename turns RED and the app crashes with below log
Here is my Crashlog:
Problem Event Name: CLR20r3
Problem Signature 01: raw2cdng.1.4.9.7-2.exe
Problem Signature 02: 1.0.0.0
Problem Signature 03: 532cb395
Problem Signature 04: raw2cdng
Problem Signature 05: 1.0.0.0
Problem Signature 06: 532cb395
Problem Signature 07: e1
Problem Signature 08: 352
Problem Signature 09: System.NullReferenceException
OS Version: 6.1.7601.2.1.0.256.1
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789
for this version (1.4.9.7) enable the debuglog and link it via pastebin. please do a test with 1.5.0.beta (linked some entries before) - its important to pastebin the crash-xml-files from the temp-path (if it crashes :) )
Gave it a go again to see if I could get it work, still crashes but now when click on the batch window header bar it crashed right away with out any files.
Try it for my desktop, My downloads folder & even tried from my 2nd SSD drive on the root in my laptop. Crashed when I click on the batch window header bar, but
then I tried it off my USB 3.0 4TB raid "0" box and it fired up loaded a file I could see the preview play the file just like you should but crashed when I tried to convert it.
The "outputPath":"D:\\New folder. So I inputted a unique name this time and it work, :o "MLV 5D2" , So do you Need to input a name first of it will crash ? I guess so
I would have thought it would just use the file input name. The App & test file was on USB 3.0 drive in a root folder & output path was to the 2nd SSD drive on the root folder. Just run it from my 2nd SSD drive and ok now a long as you have a unique name in that field, but it will crash if there is now file in the batch window and you click on any thing on that side. Well at least it work just need to get a few bugs out. :D
Edit:
- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
<Provider Name="Windows Error Reporting" />
<EventID Qualifiers="0">1001</EventID>
<Level>4</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2014-04-16T02:27:51.000000000Z" />
<EventRecordID>27556</EventRecordID>
<Channel>Application</Channel>
<Computer>i7Toshiba</Computer>
<Security />
</System>
- <EventData>
<Data />
<Data>0</Data>
<Data>CLR20r3</Data>
<Data>Not available</Data>
<Data>0</Data>
<Data>raw2cdng.1.5.0.beta.exe</Data>
<Data>1.0.0.0</Data>
<Data>534871ee</Data>
<Data>mscorlib</Data>
<Data>4.0.30319.18444</Data>
<Data>52717edc</Data>
<Data>24d</Data>
<Data>18</Data>
<Data>System.ArgumentOutOfRange</Data>
<Data />
<Data>C:\Users\i7 Toshiba\AppData\Local\Temp\WER4359.tmp.WERInternalMetadata.xml</Data>
<Data>C:\Users\i7 Toshiba\AppData\Local\Microsoft\Windows\WER\ReportArchive\AppCrash_raw2cdng.1.5.0.b_bef85dffdf135db157af789b841bb5ef8296687_291a4a4c</Data>
<Data />
<Data>0</Data>
<Data>b37f3eb0-c50e-11e3-8dcb-6036ddb07e2f</Data>
<Data>1</Data>
</EventData>
</Event>
XML error reporting also getting "System.NullReferenceException" so times
Win 7 Ultimate SP1, hope this helps
Problem signature:
Problem Event Name: CLR20r3
Problem Signature 01: raw2cdng.1.5.0.beta.exe
Problem Signature 02: 1.0.0.0
Problem Signature 03: 534871ee
Problem Signature 04: mscorlib
Problem Signature 05: 4.0.30319.18063
Problem Signature 06: 526766b5
Problem Signature 07: 24d
Problem Signature 08: 18
Problem Signature 09: System.ArgumentOutOfRange
OS Version: 6.1.7601.2.1.0.768.3
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789
I import the new cdngs 16bit maximize to my Mac and they have a heavy green over cast , but the 16bit liner look normal in the mac preview.
Tried a direct import in SpeedGrade on the Mac and heavy green cast which I seem not be able to cleanup, but with dynamic link from Premiere Pro it look ok.
A.E. look ok either way but with a gamma shift up on the 16bit maximize, but in Apple Products - Motions, Compressor, FCPX the 16bit maximize file option Will not work as a Heavy Green Cast over the image. So in Apple products use "non 16bit maximize". Never tried the 12bit thou so may be better I will have to test that later.
Edit: 16bit maximize look Ok in Photoshop on the Mac.
thanks a lot. thats something i can work with.
btw. with speedgrade there was a greencast before, you have to use/tweak a .look file with values for the colormatrix (example in 1.4.5 zip). anyway, thanks, i will look on that as well.
regards chmee
Hi chmee,
I've been away on holiday for a while, just got my 5D Mark III and I am happy to report that all previous issues I had in the past are now gone. No hot/bad sensels or anything else. MLV/RAW files import perfectly into PPro CC iwth no color cast issues so far. I only shot a couple of things in the last day or two since I got my camera but so far it all looks good. I'll do some more testing and report back.
The interface it great and it work a treat, fast too. Great work!
i've seen on my today's tests, the pink highlights are still there in 16bit maximized, but the pink-highlight-button does his job quite well, the general brightness does not change. so, avoid massive highlights while recording :) (and we're waiting for the adobe-update)
(coding on beta2 1.5.0, found some silly bugs, leading to these crashes i think - today or tomorrow)
regards chmee
raw2cdng 1.5.0.beta2 online
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
(http://dl.phreekz.de/raw2cdng/raw2cdng.1.5.0.beta2.jpg)
test and tell. if this beta2 crashes on first start, try it a second time (on first start it writes the settings-file, i think, on todo-list)
(some buttons are light transparent, because not yet implemented)
i drag and drop 25 mlv files and converted all 25 files succesfully. in your older version when convertion is taking plce the particular file is highlighted with green color which makes easier for us to know how many files are converted and how many files are stil left. it would be great if you can stil add that feature but not a big deal we can open the save path folder and see. i wil do more test and report if i find any problem. i cant wait to test the chroma smoothing as well. this application is simply amazing
:) ah. yes. yesterday i still thought about it :) on todo list.
Oh my god, amazing!!!! Working smoothly importing 30 mlv's. Filenaming is amazing now! Just don't click any item on the list while converting, it will crash.. hehe. Maybe disabling the interface while converting? The preview is amaaaazing! Working flawlessly now!!!!
Just, WOW.
lol, oh yeah, disabling gui-events while converting.. on todo-list. thx.
the new interface is perfect. great job
Looks so good, too bad i am on a mac :(
Thanks for your efforts Chmee :)
I just tested beta2 and am still seeing the vertical lines in the DNG files.
Currently RAWanizer 0.6.1 (the one I've been using up until now) exports without the vertical lines, but I'm limited to shooting RAW v1 in camera and it doesn't create CinemaDNG files (which are awesome by the way - thanks).
So hopefully you can figure out the issue because it would make your program the number one choice for ML RAW conversion!
Cheers from Tokyo,
Paul :)
vertical banding and chroma smoothing are under gpl-license. i have to talk with @g3gg0/@a1ex/@1% about using their code, because in this case i'm too lazy to solve it myself.. but, they're mostly recoded and inside :)
@DavidSh As i've read, ThomasWorth is preparing his new RawMagic-Version.
I'm OK with you using my code for free software (http://www.gnu.org/philosophy/free-sw.html) under the terms of GPL (nothing to do with price).
Maybe you can tell Thomas Worth too; he is currently using my GPL'ed code in his proprietary app.
This is great chmee! (1.5 beta) Superb program and interface, no problems here just good CDNG files to import direct to PPro CC, no vertical banding or pink highlights, thank you.
Quote from: chmee on April 17, 2014, 12:46:32 PM
vertical banding and chroma smoothing are under gpl-license. i have to talk with @g3gg0/@a1ex/@1% about using their code, because in this case i'm too lazy to solve it myself.. but, they're mostly recoded and inside :)
@DavidSh As i've read, ThomasWorth is preparing his new RawMagic-Version.
Quote from: a1ex on April 17, 2014, 01:31:23 PM
I'm OK with you using my code for free software (http://www.gnu.org/philosophy/free-sw.html) under the terms of GPL (nothing to do with price).
Maybe you can tell Thomas Worth too; he is currently using my GPL'ed code in his proprietary app.
If I understand this correctly, does this mean that we won't have vertical banding correction soon?
nope, it means, if i "free" my software, i can use other gpl-licensed sourcecode as well. (so give me one or two days)
regards chmee
it will be great that we will be using only one application after implementing vertical bending and chroma smoothing. for now i m using atleast three applications in my workflow
1.convert to raw
2.remove pink dots
2.raw2cdng
so. i've done a git with the sourcecode. therefore its now free software (this software will never cost a dime, its up to you to donate me some cents). i hope, some of you will help me to optimize code, to put some new options into it and to still let me learn the next stage of coding :) tried to comment most points.
https://bitbucket.org/chmee/raw2cdng
(i'm now a little bit tired, have to take my time for my children and for two other projects)
regards chmee
Quote from: chmee on April 18, 2014, 09:59:13 PM
so. i've done a git with the sourcecode. therefore its now free software (this software will never cost a dime, its up to you to donate me some cents). i hope, some of you will help me to optimize code, to put some new options into it and to still let me learn the next stage of coding :) tried to comment most points.
https://bitbucket.org/chmee/raw2cdng
(i'm now a little bit tired, have to take my time for my children and for two other projects)
regards chmee
hey chmee,
Is it possible for you to integrate a pink dot removing feature? I love the fact that cdng files can be directly imported into premiere pro thus simplifying the workflow. But i can't benefit from that because my 650D procuses raw footage with pink dots so i have to use g3gg0's raw2dng to convert from raw to dng and then import into AE and then into premiere... That takes too long and some quality lost in the process.
In this thread: http://www.magiclantern.fm/forum/index.php?topic=5404.0
g3gg0 released a raw2dng executable that does chroma smoothing which removes pink dots as well.
please read some more thread entries. the last 10 will give you the answer.
Quote from: chmee on April 18, 2014, 11:37:00 PM
please read some more thread entries. the last 10 will give you the answer.
ok my bad. i hope you solve license related issues soon.
Quote from: Nautilus on April 19, 2014, 12:36:57 AM
ok my bad. i hope you solve license related issues soon.
If you read the previous posts, it actually said there are non. :)
Quote from: ted ramasola on April 19, 2014, 12:40:59 AM
If you read the previous posts, it actually said there are non. :)
Sorry i couldn't see that part. He says he has to ask alex, g3gg0 and others to use their codes. And Alex replied it's ok.
Quotenope, it means, if i "free" my software, i can use other gpl-licensed sourcecode as well. (so give me one or two days)
:) you quoted the interesting part..
Quote..so. i've done a git with the sourcecode. therefore its now free software ..
Quote from: chmee on April 18, 2014, 09:59:13 PM
so. i've done a git with the sourcecode. therefore its now free software (this software will never cost a dime, its up to you to donate me some cents). i hope, some of you will help me to optimize code, to put some new options into it and to still let me learn the next stage of coding :) tried to comment most points.
https://bitbucket.org/chmee/raw2cdng
(i'm now a little bit tired, have to take my time for my children and for two other projects)
regards chmee
Bravo! Thanks a lot! I hope will see the vertical banding checkbox soon! :))
i dropped 4 .raw files, each 6-11gb. The converter crashed. something needs to fixed? thanks
better informations lead to better solutions.
WHEN does it crash? while dragdrop? while preview? while convert? Does it affect one file? did you tested, if maybe one's corrupt?
Chmee, here is an issue I have encountered with my RAW video. Looks to be aliasing? Note: other shots do not see this.
Details:
50d
Lexar 1000x (new)
1568x882
Processed individually
Running CDNG version 1.4.9.7
Directly imported into Premier
(http://www.upload.ee/image/4013824/M201109F_C0001_B16._000000-000246_.dng.Still001.jpg)
Here's the h.264 equivalent if you are at all interested.
(http://www.upload.ee/image/4013827/M201109F_C0001_B16._000000-000246_.dng.Still002.jpg)
@painya,
DSLR video is subject to aliasing due to line-skipping. RAW video does not negate this issue. The only way you can avoid it is to use crop mode.
Quote from: Frank7D on April 21, 2014, 12:56:12 AM
@painya,
DSLR video is subject to aliasing due to line-skipping. RAW video does not negate this issue. The only way you can avoid it is to use crop mode.
Right! Thank you very much. I forgot that it had to be in crop mode to fix that.
Silly me.
Quote from: painya on April 21, 2014, 12:57:58 AM
Right! Thank you very much. I forgot that it had to be in crop mode to fix that.
Silly me.
Aliasing and moire are much more aggresive because you're shooting at a lower resolution than it's max also!
I was shooting at the 50d's max no-crop resolution for RAW.
chmee, it crashed while converting the .raw files to cdng.. i did nothing, did not even try to preview. Just drag and dropped, clicked convert, it processed for a few seconds and got crashed, the 1.5.0 beta 2 version
The application crashed with new mlv nightly builds (Canon 5d Mk3 113). Tried to convert a old mlv from March 15 and worked fine. The new MLV that i recorded with recently nightly builds the application crashed with only drag and drop.
Though I was able to convert all my 48 mlvs to cdngs heres my notes on v1.5.0beta2
1. Took me 2 times to initially start the app. 1st it crashed.
2. I was able to drag n drop all 48 files but while converting I was moving my mouse over in active buttons and it crashed at the 45th file.
3. I repeated the process this time I left it alone, it successfully converted. But it has a new name structure which presents a problem for me in resolve.
with v 1.4.9.6 the files can alternate dng and wav in the media bin in resolve making it easy to click the audio link button to sync dng and wav files. This version however will not display in alternate with dng and wav.
I look forward that you will finally be able to link the wav and dngs with your upcoming version.
Best of luck!
minutes ago i put a beta3 (http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/) on the server.
done:
* wav files have now an according timecode for resolve fi
* some more infos in debug-log
* some more variables for the filenamegenerator. [D2] is interesting for resolve filenames
[C]_1_[D2]_C0000(_)
found a bug seconds ago:
* an "Argument out of Range" exception leads to crash. investigating right now..
* inconsistency with raw-files - if they re wrong shown in preview, they will be converted wrong.
* (fixed. will be in beta4 - if there are more than one .raw, all .raw's take the blocklistdata of the last one, silly bug..)
Chmee!!!!
AUDIO AUTO SYNC NOW WORKS!!!! AAARRRGGGHHHHH!!!
Thank you!
Quote from: ted ramasola on April 23, 2014, 09:34:28 PM
Chmee!!!!
AUDIO AUTO SYNC NOW WORKS!!!! AAARRRGGGHHHHH!!!
Thank you!
how you doing this lol ?
Quote from: ted ramasola on April 23, 2014, 09:34:28 PM
AUDIO AUTO SYNC NOW WORKS!!!!
Confirmed and replicated :-)
Great job, chmee! Now if only I could make it work under Wine.
Quote from: Jakobmen on April 23, 2014, 09:51:10 PM
how you doing this lol ?
In Resolve, Autosync the files in media bin
What's the workflow? We talking about resolve?
after the cdng extraction using beta3, open resolve.
Drag the folder containing all the extracted dngs and wavs into the Media Pool.
In the Media Pool right click on Master folder, or if you crated a New Bin right click the Folder containing the footage files.
Choose AutoSync based on timecode.
In Edit Tab, Create New Timeline, uncheck create empty timeline, all your footage with linked audio shows up on new timeline.
Sit back a while and smile.
Go back online and click donate button on chmee's site. :P
@dubzeebass
[linux, wine] do you have some info for me? i dont have any linux based pc here..
when using Black Magic camera CDNGs, the audio snaps automaticly
and any addional action is not requaierd.
is that the case for MLV DNGs when using v1.5.0beta3 ?
nope, i didnt have found the key for that. but it s a big step into the right direction, it syncs accordingly, with just one or two more clicks. and i assume, its something about the wav-file, because, if you delete the wav file, it changes its behaviour in the mediabrowser and the path changes into a mediafile.
please link some more BM-Mediasequences with audio, so i can analyze it.
I see this app also fixes the pink cast in premiere cc. Is there no mac version of this program?
you'll have to wait for
RawMagic-Update
and/or
the next adobe update (things will be better)
@chmee have you abandoned proxies? I know Premiere is switching to better support for dng, but even a top end workstation is going to struggle with projects over an hour in length and it would be great to have proxies for editing. The only codec I can find that does not require previews in Premiere is .m2v, which I can create in AE but that means going through ACR with every clip to get a proxy...
would be
* jpg filesequence
* with half resolution
* not color accurate
ok?
Test out the beta3 , every thing work perfectly not crashes . The best results for Premiere Pro yet with 16bit maximized Cdng's , Hi-Light were handled very will &
with the "Auto Level" filter it look dam good ! I would say at this point I can go mobile with my i7 PC Laptop and a small USB 3.0 raid 0 box and rough edit on the road or small project with only a few hours for a turn around (e.g. new stories current events etc...) , being I still use FCPX as my main NLE , but now I have a very good second choice.
At least now I can use more that A.E.cc and get my money worth out of my subscription from adobe :D
Thanks chmee :)
had a crash with a large amount of files being imported.
The debug doesn't say much, just that where it stops. It can't seem to load a file that is 11gb. It just doesnt do anything when I try to add it to the batch on its own.
**Update* But it loaded a 5 file one just fine. I wonder if that file is corrupt now that I'm looking
I'm on 1.5.0 beta3 btw
There is gold every day on each topic of this forum ! I'm going crazy !
But wait, what's the difference between DNG and CDNG ?
How can cdng sequences playback be so smooth in premier pro cc ?
@chmee, you make the whole process much more easier. No offline editing and intermediate rendering any more.
There is some bugs of course for a beta :
Crashes during extraction
sometimes the destination path is not working and no way to know where the dngs are extracted. I'll try a long extraction with many MLV.
CDNG 16 bit make premiere pro cc crash during playback. CDNG 16 bit maximized are all right.
EDIT : no crashes at all with .MLV files queue. Drag and drop multiple RAW not working. Drag and drop multiple MLV working. When drag and drop multiple file, the loading time seems a bit long, but maybe it's normal.
Thanks !
win 8.1 x64, premiere pro cc, updated.
ml 63b2f145cb3b 2014/02/14.
audio not syncing in resolve as well
@chmee JPEGs still produce a yellow bar for me, so I don't want to use them for proxies for the same point Reddeercity raised: Premiere doesn't need to create previews for .m2v (MPEG2 format in output module of AE), so I can edit anywhere on a basic laptop with Premiere installed. All I need is a way to get from MLV to MPEG2. Thanks all the same though, perhaps others would be happy with JPEGs.
I've tried the last few versions of your app and kept getting crashes. Beta3 is a lot faster and seems to be using most of my computers raw power, but I can't drag the folders into AE like I always have. Tried with the same MLV file ran through RAW Video Converter 1.9.1 and had no problem.
mates, thank you for testing and response.
* RAW-Error is fixed, will be in beta4, found it days ago.
* Resolve-Sync - i tried to realize it, it seems to work. @evanbuzzell please check, if both mediafiles have the same timecode - this was my first intention - the second one was, to change the path/files behaviour into the regular path-as-mediafile. i didnt found the key for that, i need more samples, i said it before..
* @RedRocks. ok. in beta4 there will be a proxy-jpeg-output, as i stated, they are not color accurate, have a wrong gamma, but for simple cutting it should do. and: ffmpeg has implemented mlv, this could be your way (maybe mine too for other codecs).
* ah, btw. do you use mercury-playback on gpu? my yellow marked sequences play quite fluid, even if i use gpu-accelerated effects.
* clearly saying, my intention was/is to get the user as fast as possible editing his stuff
* @piloui "sometimes destinationpath not working". only under raw or with mlv as well?
regards chmee
MLV workflow seems to be quiet stable indeed. No crash, no destination path bug. Actually the only limitation i see comes from adobe ::)
Cannot drag n drop multiple files in the bin like in resolve. For 100 shots, you have to do 100 drag n drops. Can't belive such obvious thing are missing.
Great job chmee. Any tips concerning loading time when dropping several MLV in raw2cdng ?
Some ideas :
it would be great to have some highlight of the current mlv extracted, it's purely visual.
Maybe some button to pause, stop, resume.
And a great thing you never find in batch processing software : the magic check box "turn off computer when finished"
Just some ideas, the tool as it is is already a gift !
;)
thanks a lot.
* the loading time on drag/drop is because of reading all attributes of every file, how many frames are there really, is there audio, creating frame-lists for the preview, lensdata and so on.
ideas:
* highlight active converting-file - its already on the todo-list - as it was already in the older versions..
* "turn off computer when finished" really? ok.
Thanks Chmee, I'll try the JPEGs and see how that works out. I do have gpu enabled for PP & AE, running through a 3 GB GTX 660 Ti. When I first open a project yellows are fine, then I get 20 mins down the timeline and things get laggy. A project of .m2v seems to run smoother than any other codecs I can spit out of AE. Like you say, ffmpeg will hopefully give us more options.
Files naming tool is great too. date of creation is a really good way to differentiate shots. However, date of creation, under window, changes when you copy the file. This means, if you shoot let say 10 MLV, on a 3 days period, they will all have, on CF card, a different creation date.
BUT
If you copy all of them at once ( as everybody do i hope ^^ ) they will all get an new and similar creation date with a little difference.
(http://www.cinemersion.fr/images/divers/mlv_date.jpg)
The MODIFICATION DATE always stays the same. Weird, for me, creation date would be the shooting time, and modification would be, copy, export etc ... Try figure :/ windows world !
Anyway, maybe an other tag option like [DM] "modification date" would be cool.
Thanks
;)
Quote from: piloui on April 26, 2014, 12:34:06 AM
Actually the only limitation i see comes from adobe ::)
Cannot drag n drop multiple files in the bin like in resolve. For 100 shots, you have to do 100 drag n drops. Can't belive such obvious thing are missing.
There is a simple solution for several cdng sequences to drop in premiere pro in one move :
Go where your DNGs are excracted. You should have all your folders containing all your DNGs. In the search bar, top right corner, type *000.dng
This should list all but only your first DNGs from all your sequences.
Select all of them, drag n drop and bim bam, all your sequences in premiere in one drop.
OK. Another, no doubt often asked question (I couldn't find anything in a search, though). I have many hundreds of DNG files from the last few months. Now that CHMEE's lovely CDNG tool is the latest answer to my latest prayers, I was wondering how to convert these DNGs to CDNGs, as CDNGs are more compatible with a wider range of video apps and I'd like to use them as my standard workflow thingies. I've deleted all my MLVs, and before them all my RAWs so I can't go back to the source.
Any ideas?
@dubzeebass
QuoteGreat job, chmee! Now if only I could make it work under Wine.
here you go: http://pastebin.com/SUHCXm8X
@chmee
Quote[linux, wine] do you have some info for me?
Plenty of infos about raw and linux here (http://www.magiclantern.fm/forum/index.php?topic=9335) collected by me..
also how to run your program via wine...
WINE is acutally and opensource implementation of the windows api, thus a layer of windows libraries upon linux. That has the advantage to be able to run stuff in native speed, as it's not emulated. The disadvantage against a crossplatform raw2cdng aparrt from the obvious is that it needs plenty of space, 600+mb with dependecies, and then in wineprefix ( a bottled environment where a windows system is presented to the application) is annother 500mb, with dontnet4 installed etc, just for running a little program like yours; also, it is not guaranteed that programs will work at all or with every version of wine, or possible that changes you make to your program would then also break it in wine.
One can ship around this by using a dedicated wine version from playonlinux, just for one app, and set it up so that everything a windows program needs to run is selfcontained in one folder, but it's quite a job to do that.
Hence my question - Would it be possible to go crossplatform with raw2cdng, if it's just to build against mono? I have not looked at your code if it's that easy..
QuoteI dont have any Linux based pc here..
In terms of developing, I guess you don't have to...
If you would go for crossplatform and build raw2cdng against mono, the .exe build for windows should run on Linux/Mac, too
g3gg0 with MLVViewSharp e.g. does it that way.
Normally, a virtual machine would suffice, e.g Virtualbox as portable solution:
http://www.vbox.me/ (portable has the advantage that you are not polluting your system at all)
and a ready to use image:
http://downloads.sourceforge.net/virtualboximage/xubuntu_1204.7z
(username/password): xubuntu/reverse
Thats only one solution, as Linux also runs form usb-stick, and it even saves the changes you do on the stick then (e.g install mono, in terminal, enter "sudo apt-get install mono-runtime, libmono-corlib2.0-cil libmono-winforms2.0-cil" ) ...
If you want to try that, I would recommend unetbootin, quite bulletproof thing (check persistence)
@revast
thanks a lot for the short introduction. its time to get on this ship because of coding for raspberry and other linuxbased things, this sounds the right way. i'll try my best..
please some patience for beta4, next days..
Hi
This weekend I have a short that I want film with magic lantern mlv format. Recently i have problems with latest nightly builds Canon 5D MK3 1.1.3 using raw2cdng 1.5 b3 with constant application crashes.
Maybe this is a question for chmee very grateful and congrats for your great job with the raw2cdng tool but i want to know what nighly build you test that is working better or more stable with mlv format using raw2cdng 1.5 b3.
Please let me know
Thanks
i had to look into my download-directory :) it was a version from april 14. but, i m no reference for that, i ve got a bunch of mlv's i recorded some day and i use them to check my code..
@piloui thanks, will implement it, because its the more useful date :)
@jayzed sorry, didnt thought about it, and by now i wont. lack of time.
friends I could not render out the footage correctly from Resolve. The audio gets jumbled and only one out of two different footage is rendered. I shot using .mlv with audio enabled, beta3 converter, 5dIII, march 2014 update. The footage preview was good, had syncing audio in mlrawviewer. I also ticked "autosync based on timecode" in resolve... anything wrong that i ve done? also the last audio file is slightly lengthier than the video file.. thanks
Quote from: ted ramasola on April 24, 2014, 02:17:26 AM
Go back online and click donate button on chmee's site. :P
Alredy done:) Chmee really cool!!!
[F] mode of naming had the problems i previously posted. But changing the naming mode to [F]_[D] cleared the problem now...
@chmee.. thanks for the great application. Could you also please check whether the flickering issue in Resolve is due to the converter? because 1.4.9.7 version had no flickering issues after rendering the color corrected output, however there were audio syncing complaints which was rectified in beta 3. But, in beta 3, I get the flickering issues in resolve once again. Audio syncing is perfect. Roundtrip is good except that i have to manually link the footage, as usual. Because the flickering was absent when i used cdngs from 1.4.9.7 and present when i use beta 3, i think there is an association. Pls see to that friend... thanks
hi again, I used the naming code [F]_[D] in beta 3 converter. Selected cdng 16bit maximized. Did a roundtrip in resolve and premiere cc. Worked fine, there was no flickering issue in the color corrected outputs. Thanks.
@anandkamal good to hear.
Quote from: chmee on April 25, 2014, 06:32:11 PM
mates, thank you for testing and response.
* RAW-Error is fixed, will be in beta4, found it days ago.
* Resolve-Sync - i tried to realize it, it seems to work. @evanbuzzell please check, if both mediafiles have the same timecode - this was my first intention - the second one was, to change the path/files behaviour into the regular path-as-mediafile. i didnt found the key for that, i need more samples, i said it before..
* @RedRocks. ok. in beta4 there will be a proxy-jpeg-output, as i stated, they are not color accurate, have a wrong gamma, but for simple cutting it should do. and: ffmpeg has implemented mlv, this could be your way (maybe mine too for other codecs).
* ah, btw. do you use mercury-playback on gpu? my yellow marked sequences play quite fluid, even if i use gpu-accelerated effects.
* clearly saying, my intention was/is to get the user as fast as possible editing his stuff
* @piloui "sometimes destinationpath not working". only under raw or with mlv as well?
regards chmee
So the timecode for the audio and the video are different. I could have sworn that the audio files were synced in Premiere just a few days ago and now I'm not seeing any audio sync in premiere or resolve. This just made this project a million times harder aye! I've tried changing the file names to something different when exporting, any other ideas?
@evanbuzzell
re-check: what's the framerate for this videos? please write some video/audio timecode pairs, if theres maybe a correlation.. is the audiofile playing fine in any audio/mediaplayer?
Quote from: chmee on May 02, 2014, 12:18:26 AM
@evanbuzzell
re-check: what's the framerate for this videos? please write some video/audio timecode pairs, if theres maybe a correlation.. is the audiofile playing fine in any audio/mediaplayer?
23.976 on all the clips.
Video length is 02:18:10, audio is 02:17:38400
V: 01:12:01, A: 01:11:38400
V: 01:43:07, A: 01:42:19200
V: 02:13:11, A: 02:13:00000
Different day, More Recent than most:
V: 00:23:06, A: 23:09600
V: 02:12:18, A: 02:12:00000
V: 03:13:07, A: 03:12:09600
V: 00:46:12, A: 46:28800
Most recent build:
V:00:21:16, A: 21:28800
V: 00:39:07, A: 39:19200
V: 05:43:03, A: 4:25:28800
V: 02:08:05, A: 02:07:38400
older one
V: 00:28:23, A: 28:19200
V: 00:56:17, A: 57:00000
V: 00:51:23, A: 51:19200
The audiofile plays fine for the most part, it is the sync that is the issue. 5d2 is the camera I'm using apr 23, 14
the timecode (first frame) is made of the creationtime of the file (lastWrite is the better attribute, its coded already, will be in beta4). does one of the both timecodes match to the filetime? could you send me examplefiles to
[email protected]? (just one dng and the audiofile) i have to code some more debug and helper-things for those problems.
regards chmee
Hello,
how can it be that my 1.5.beta3 makes no CDNG,
but nut DNG?
At least recognizes Premiere not the files.
greeding
@wartburg no, they should 100% work. maybe you're looking into the wrong tab/window in premiere :)
chmee, i was telling about the flickering issue in resolve. Now when i turn "delta pink highlights" in raw2cdng and process further, i get flickering issues during roundtrip between resolve and premiere. When i turn the delta pink highlights off, the flickering issue solves... Just wanted to say the converter influences everything (an advantage of course) ;) With [f]_[d] naming, 16bit maximized and everything else turned off, the roundtrip is good without flickering in resolve. Of course i had to link the media pool manually (problem between adobe and resolve :()
the pink highlight-button is ONLY for direct editing in Premiere. you dont need it (i think) if you start and end in resolve. the premiere roundtrip is finally only for editpoints.
but: the interesting point is, if its flickering: where does it flicker? in resolve as well? in premiere? what data/recodring mode you're using?
I use .mlv files for processing. Latest build 5d3 113. Beta 3 converter.
Flickering is caused by Resolve. Flickering is like lightning occurring on the whole frame (not at the top or bottom, but as a whole). The color correction does not apply to certain frames and those frames show, among the other colored frames, like flickerings or lightning. This is seen in the rendered output and obviously when roundtrip-ped back into premiere.
irfanview is able to render dng's, please look into the folder, if you find those bad pictures. send it with another functional dng to
[email protected]
thanks, with irfanview, cdngs are fine. But the flickering issue is NOT while importing the cdngs. The problem arises after I do color corrections (in playbacks & rendered outs). The color changes won't apply to certain frames. For ex: among 700 frames, the erroneous frames are around 300 or something. They keep on blinking indicating that the color corrections have not been worked on those 300 frames. I found that it has to do something with raw2cdng. When certain parameters are changed in raw2cdng, the flickering reappears. Every time (during updates or whatever), I optimize the roundtrip - resolve is a headache :(
The flickering is a very common problem in resolve (i googled it) and the below link shows the same problem that I face:
https://www.youtube.com/watch?v=Zq_Nhkoz0KU
There are so many reasons for flickering. Mainly, it is related to frames and frame rates, i think. With older versions of raw2cdng, when I faced problems in 23.976 conversion, there was flickering. Now in beta 3, everything is fine, unless I stick to certain keys, like naming in [f]_[d] and 16bit maximized options.
please take acount into the timecode of "broken" frames. are they regular in distance or on a certain timecode/frame?
Quote from: chmee on May 04, 2014, 09:09:17 PM
please take acount into the timecode of "broken" frames. are they regular in distance or on a certain timecode/frame?
Hey, was going to test this converter.. but I can't seem to import .raw files :(
The beta3 and beta2 open fine, but when i drag drop a .raw file it gives this (http://www.oxygenxml.com/doc/ug-editor/img/cursor_unavailable.png) thing :(
I ll notice the timecode next time when it flickers. I think those are random flickerings based on the amount of color change that we do. I do not get flickering, what so ever, if I use LUTs and do nothing else. I face the problem only when i manually color correct the footage.
@ramon this symbol should only appear if you drop on wrong position. strange.. maybe thats a common problem with dragdrop in windows sometimes? try other app-position? or start app as admin..
does anyone else have issues when trying to convert mlv files with numerous .001, etc (split at 4GBs due to using FAT32). I found beta3 isn't recognizing my media past around 3 files , any larger and the program doesnt see the clip at all ?? The application essentially did not recognize when I dropped the first MLV of the sequence of 8 files. However, If i drop a single MLV file or and MLV file with a single .001 file, it reads and converts no problem. Otherwise, it essentially does not see when I drop files with more "splits"? Any suggestions? Tried beta 2 and beta3.
Also, very excited for the new mlvtocdng :)
Hello Chmee! I test beta3 crashes with big files. I shot files from 250 frames to 6000 (4 minutes with 25 fps) and all OK! But with 8 minutes (12000 frames) program crash :( One time on 2900 frame, and one time on 3900 frame... this is very strange, because 4 minutes shot with 6000 frames convert ok... Somebody test beta3 with big files 8 minutes and more.
thanks for info. please forgive, busy in jobs. the next beta will have a debug-button for better insight into the things happening..
Just shot 376gb in dual-iso mode, ready to test new beta.
Hmmm... with bild magiclantern-Nightly.2014May07.5D3123 and raw2cdng 1.5beta3 pink come back...
(http://i7.pixs.ru/thumbs/7/2/6/M071324140_7884203_12020726.jpg) (http://pixs.ru/showimage/M071324140_7884203_12020726.png)
Then i use MLVViewSharp http://www.magiclantern.fm/forum/index.php?topic=8447.0 to convert to DNG and no pink appeare... Strange
@artyg
please send me a short MLV example to
[email protected]. actually i'm a bit confused because of the actual changes regarding the blacklevel. i didn't saw those problems with earlier ml-builds..
https://bitbucket.org/hudson/magic-lantern/pull-request/484/black-level-fix/diff
please @allthe actual ml-builds are quite new (especially regarding the blacklevel-things), some of them (1.2.3 for 5DIII) are bleeding edge.
dont change your system if you want to work productive. if you need them, use an older build, something between ~feb09(mlv-audio) and ~apr27.. i am still testing my code with a 5DIII 1.1.3 build from ~apr20, because its workin' and stable. you are not bound to use the newest builds!
The black level bug was introduced in February 11 (http://www.magiclantern.fm/forum/index.php?topic=10443). I am currently waiting for code review for the fix before merging it.
If April builds are any better than May ones (especially regarding black level), please show exactly where the difference is (bug report).
dose this problem refer to 5D3 Only ? as there has never been a problem with 5d2
but I'm still on Feb16th NB, no problems.
There are more causes for this bug.
The one I've talked about earlier only affects 5D3 and 60D (it appears when changing video modes and recording right away, and I could reproduce it and fix it).
It can also appear if a pink (corrupted) frame appears right when ML is computing the black level (though this one is quite hard to reproduce).
It can also appear because of some Canon code overwriting the raw buffer - remember that raw mode is just a debugging flag written by Canon, but they didn't design it for everyday usage. For example, in 500D, if you take a picture in LiveView and hold the shutter button for a little longer, the raw buffer will be overwritten by LiveView data and the raw zebras will be wrong. This has nothing to do with raw video, but it's a test case that can be reproduced (very useful for testing the fix).
It may also appear because of race conditions (I've tried to avoid it and make the raw backend thread-safe, but I'm not an expert in multithreaded programming, so I may have missed something).
The workaround from MLV (which is enabled by default - not by me (https://bitbucket.org/hudson/magic-lantern/commits/02c9af698af933dfea31212713d5216cb0626ff8)) is incorrect for 5D2 and many other cameras. The pull request removes the workaround and fixes the raw backend properly IMO. It checks the optical black area, so it should also detect bad black levels caused by a pink frame (though not with 100% chances).
On the pull request page, I've listed the relevant tests needed to make sure the fix is portable, and got zero feedback on these for about one week. I simply didn't have the time to do these early tests myself, and without code review and early feedback, I'm not going to include the fix in the nightly builds.
Aside from the black level issue, I cant get the application to process MLVs from my 7D without it crashing. I've tried several sets of MLV files recorded at different times, on different cards, with different version of ML\TL. Is there anyway I can provide a log of source of diagnostic info for this? I have no other alternative mlv2cdng workflow, as this has worked so well
Many thanks!
@jbowdach please be patient for 1 or 2 days, i will a provide a beta4 with more debug-infos (and some bugfixes)
Quote from: chmee on May 10, 2014, 10:18:33 PM
@jbowdach please be patient for 1 or 2 days, i will a provide a beta4 with more debug-infos (and some bugfixes)
@chmee many thanks, you are awesome! Please let me know how I can help provide any debug info
first: @a1ex it was not my intention to leave bad words on testing and generally on your work - understanding bugs in my code with a glitch on ml-side makes it very difficult to find the problem. so please @all it was no call-up to not test new builds.. your tests are of course necessary.
beta4 is online
* raw-bug (i hope) fixed
* more templates for filename generating
* proxy jpegs
* more debug infos in log (debug-button on top)
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
regards chmee
Quote from: chmee on May 11, 2014, 10:52:53 PM
first: @a1ex it was not my intention to leave bad words on testing and generally on your work - understanding bugs in my code with a glitch on ml-side makes it very difficult to find the problem. so please @all it was no call-up to not test new builds.. your tests are of course necessary.
beta4 is online
* raw-bug (i hope) fixed
* more templates for filename generating
* proxy jpegs
* more debug infos in log (debug-button on top)
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
regards chmee
Thank very much, good sir! I am already testing it now to see if it fixes any of the crashing issues I had w beta3 and I have debug enabled so I can provide you debug info. Your work is VERY much appreciated as it's easily the best method for MLV -> DNG + WAV at the moment.
EDIT: Please find my debug log on pastebin below and issue files on bitbucket. It crashed as I was converted MLV files to 16bit CNDNGs (trying to avoid extra options so not using 12bit downconversion or any other options yet) -
Debug Log - http://bit.ly/1jw3160 (http://bit.ly/1jw3160)
Thank chmee, Just downloaded the new version (1.5.0 beta 4) and with give it a work out :)
One question thou, Is there a way of getting the normal 16bit Cdng as in Version 1.4.9 ?
I can't edit the Cdngs or use them in any Apple produces, as there are all pink cast (for Adobe)
but I like the new version with the file name generator etc.. . I can use the older version it fine but just wondering . :D
Quote from: reddeercity on May 11, 2014, 11:24:05 PM
Thank chmee, Just downloaded the new version (1.5.0 beta 4) and with give it a work out :)
One question thou, Is there a way of getting the normal 16bit Cdng as in Version 1.4.9 ?
I can't edit the Cdngs or use them in any Apple produces, as there are all pink cast (for Adobe)
but I like the new version with the file name generator etc.. . I can use the older version it fine but just wondering . :D
off topic real quick: thanks for your videos online. Helped me get started w the workflow when VERY few mlv videos working w sync sound seem to be online
@reddeercity :o huh. didnt noticed such a glitch. please send me a wrong and a working dng. i'll do my best to get you into the position using the new gui :)
@jbowdach you as well, send me a short ( short! :) ) not functional mlv from your 7d to analyze it on my pc.
-->
[email protected]regards chmee
Quote from: chmee on May 11, 2014, 11:59:01 PM
@reddeercity :o huh. didnt noticed such a glitch. please send me a wrong and a working dng. i'll do my best to get you into the position using the new gui :)
@jbowdach you as well, send me a short ( short! :) ) not functional mlv from your 7d to analyze it on my pc.
--> [email protected]
regards chmee
hmmm....short..let me see what i can find that errors out thats not huge.
EDIT: Okay, I have NOTHING thats not massive. GOnna shoot a test tonight thats not too big for you to analyze . Many thanks :)
EDIT 2: Now of course, all my very short clips are working. 5-13 seconds. Possibly something with large files and split M01s, etc?
Testing Beta4, Had crashes with jpeg proxy enabled & 16bit output or when I previewed then try to convert without proxy enabled crashed also
will test some more.
@chmee here is a screen shot of FCPX Right hand side is from the Beta 4 Raw2cdng & on the left is the 1.4.9 ver. raw2cdng.
http://500px.com/photo/70187941/fcpxca~-by-david-miazga?from=user_library
Tried with 10 separate clips and couldn't get them to crash, OF COURSE....dont want to upload 4GB MLVs, so will try again in or day or so
@jbowdach
my longest testfile is a 3strip-sequence. so, i will record a longer one today and do some tests. looking, if there are "out of memory" or some IO-exceptions doin this.
thanks! Ive been trying to shoot full cards, so I 59GBs of MLVs + M001, M002, etcs. Seems be errors on various clips tho, not just REALLY long ones. my specific tests for you were all below 20 seconds tho
EDIT: Posted another log to bitbucket
Quote from: chmee on May 09, 2014, 01:14:46 PM
@artyg
please send me a short MLV example to [email protected]. actually i'm a bit confused because of the actual changes regarding the blacklevel. i didn't saw those problems with earlier ml-builds..
https://bitbucket.org/hudson/magic-lantern/pull-request/484/black-level-fix/diff
please @all
the actual ml-builds are quite new (especially regarding the blacklevel-things), some of them (1.2.3 for 5DIII) are bleeding edge. dont change your system if you want to work productive. if you need them, use an older build, something between ~feb09(mlv-audio) and ~apr27.. i am still testing my code with a 5DIII 1.1.3 build from ~apr20, because its workin' and stable. you are not bound to use the newest builds!
I downgrade my 5D3 firmware to 1.1.3 and install ML 24Apr. And pink highlites stay... Then I run beta4 in CDNG 12bit (not maximezed) mode and programm work correct! No pink. I test my old files, and they all convert correct! So, I think 1.2.3 firmware work correct too... But some strange things with 1.2.3 happen. Sometimes 5D3 does not turn on!.. I decided to stay in 1.1.3 - this firmware more stable. Thanks, chmee!
So, one bug i find. In 12bit maximized mode pink highlights appear... (post on bitbucket)
please @all i need more testfiles from other bodies than mine (5DIII 1.1.3).
(*) do a short (max 100MB) mlv or raw record, 10 frames
(*) with high dynamics, it should have overexposed and dark areas as well
(*) some greyish things and some colored (colorchecker, if you have)
(*) some kind of testpattern (search via google)?
(*) write down build date (checksum?) and body with canon-firmwareversion
(*) the files will be collected and released/listed on a site
if you want, you can send them to me (
[email protected])
regards chmee
@artyg 12bit pinkhighlights, yes i know. thats the reason i ask for more videofiles from other bodies.
Do you need a mkII file?
What setting should the BLACK FIX be ON or OFF ?
yes, 5DII as well. best is to deliver both and name them.
Sent you 4 mlvs from 5D2 and 7D to
[email protected]2 settings for each cam. With black fix ON and OFF.
Used ML nightly may 14 build for both cams.
i'm already downloading. thanks.
raw2cdng 1.5.0 beta5 online
* stabilizing code (memory leak if bigger files fixed, memory leak if preview was used fixed)
* proxy jpgs now with gamma 2.0
* busy-progressbar while reading data from drag/dropped files
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
now there are two important things for me:
* @reddeercity please send me (as written above) these two different dng's - want to fix that.
* 12bit output isnt fine at all - have to fix that.
regards chmee
many thanks, will test and report back
@chmee I will get those file to you later on today when I wakeup (eight hours from now) :D
@chmee tested out beta5 and def much improved. Still eventually ran into a crash, but got much further along. Posted as a new issue ( https://bitbucket.org/chmee/raw2cdng/issue/16/crashes-when-converting-to-16bit-cdng-from (https://bitbucket.org/chmee/raw2cdng/issue/16/crashes-when-converting-to-16bit-cdng-from)) on bitbucket for your review when you have a chance. Ive been using the same files for testing so I will shoot some fresh test footage to future testing to eliminate that as an issue
EDIT: 2 crashes. but eventually success!!
@jbowdach
looked into the debug.. it crashes while/right after audiofile-creation. it converted 5146 Frames. does the converter show this amount of frames if you dragdrop it? open the taskmanager and look while converting on the resources (memory) of the process raw2cdng. does it come near 1.5GB?
Quote from: chmee on May 19, 2014, 10:06:31 AM
@jbowdach
looked into the debug.. it crashes while/right after audiofile-creation. it converted 5146 Frames. does the converter show this amount of frames if you dragdrop it? open the taskmanager and look while converting on the resources (memory) of the process raw2cdng. does it come near 1.5GB?
Let me try another conversion and see.
Quote from: chmee on May 19, 2014, 10:06:31 AM
@jbowdach
looked into the debug.. it crashes while/right after audiofile-creation. it converted 5146 Frames. does the converter show this amount of frames if you dragdrop it? open the taskmanager and look while converting on the resources (memory) of the process raw2cdng. does it come near 1.5GB?
Doesnt appear to be coming near, its hovering at 40-100MBs. It just crashed with mem usage around 73 MBs. Newest log uploaded to bitbucket. Heading to sleep for the night, but let me know anything else you need and Ill get it for you tomorrow. THanks for the beta5, def making progress! Ur hard work is much appreciated :)
@jbowdach try to find a tool for us (maybe i code it). raw2cdng first reads the file, all blocks inside, and somewhere frame 5200 it does not take the real framenumber from the block, but a zero, some are doublets..
please use mlv_dump to write an indexfile (idx) and link to that file.
http://www.magiclantern.fm/forum/index.php?topic=7122.0
./mlv_dump -x in.mlv
i want to see if the blocks inside the mlv's are ok.
regards chmee
Chmee,
tested the latest beta5. 42 clips totalling 8+ minutes.
In resolve when I use autosync based on timecode some wav files will link with the wrong video files.
This will be indicated by wavforms with wrong displays, showing blank spaces.
Aligning them manually and the wav files are ok.
(https://scontent-b-dfw.xx.fbcdn.net/hphotos-frc3/t31.0-8/10344292_480990862031703_6166731265983228991_o.jpg)
@ted please do a second try with changed filename-structure. it seems you use
[F]
-->
/M16-2015
M16-201500000.dng
M16-2015.wav
try
[F](_)
-->
/M16-2015
M16-2015_00000.dng
M16-2015.wav
remember, Resolve wants idealy a filename like that:
[F]_[D]_001_C0000(_)
regards chmee
Quote from: chmee on May 19, 2014, 06:29:33 PM
@jbowdach try to find a tool for us (maybe i code it). raw2cdng first reads the file, all blocks inside, and somewhere frame 5200 it does not take the real framenumber from the block, but a zero, some are doublets..
please use mlv_dump to write an indexfile (idx) and link to that file.
http://www.magiclantern.fm/forum/index.php?topic=7122.0
./mlv_dump -x in.mlv
i want to see if the blocks inside the mlv's are ok.
regards chmee
Does it mention which filename?
filename of your recording, fi
mlv_dump -x M16-2018.MLV
should output a
M16-2018.idx
in the same folder.
regards chmee
Quote from: chmee on May 20, 2014, 08:50:35 AM
try
[F](_)
Tried this and the wav files are still not linked correctly. Strange thing is that some files are ok, some are not.
as i mentioned entries before, look on the files in the mediabrowser. you should see the embedded timecode. please tell me both tc-values. did you recorded nondrop or drop?
Using [f](_) convention and testing using just 2 files still will not link properly using autosync.
resulting files is named as:
WAV file = M16-2036.wav
DNG file(1stframe) = M16-2036_00000.dng
yeah, i think, there are some circumstances, where tc of video and audio do not match. and by now i cant catch these moments..
Just a small bug report:
raw2cdng.1.5.0.beta5.exe crashes while drag&drop a "early build" RAW-File (i guess very first build for 550D from May 2013) while v1.3.3.1 is doing fine.
I'm on Vista 64.
Beschreibung:
Stopped working
Problemsignatur:
Problemereignisname: CLR20r3
Problemsignatur 01: raw2cdng.1.5.0.beta5.exe
Problemsignatur 02: 1.5.0.0
Problemsignatur 03: 53790dc6
Problemsignatur 04: raw2cdng_v2
Problemsignatur 05: 1.5.0.0
Problemsignatur 06: 53790dc6
Problemsignatur 07: 5d
Problemsignatur 08: 0
Problemsignatur 09: System.DivideByZeroException
Betriebsystemversion: 6.0.6002.2.2.0.256.1
Gebietsschema-ID: 1031
The Raw file is over 900 MB big so i can't just attach it :)
@glubber are you crazy? :D confronting me with a nearly 1 year old build on a body i had no samples yet :) when does it happen? right while dragdrop? while converting? so, it shows a "DivideByZeroException", some values - assume - couldnt be read. please do me a favour, do a short record as its described here (http://www.magiclantern.fm/forum/index.php?topic=11899.0) and send it to me - i would use it for the samples gallery as well.
regards chmee
Yep right while dropping into raw2cdng window.
I'll try to dig out that old built and give You a short sample.
Quote from: ted ramasola on May 20, 2014, 07:48:59 AM
Chmee,
tested the latest beta5. 42 clips totalling 8+ minutes.
In resolve when I use autosync based on timecode some wav files will link with the wrong video files.
This will be indicated by wavforms with wrong displays, showing blank spaces.
Aligning them manually and the wav files are ok.
Hmmm, I test autosinc today and all work OK...
yeah, often its ok (on my footage i hadnt problems as well), but here were two guys with problems. the Timecode is taken from datetime of file. for the wav file i have to recalculate to audiosamples AND adjust with nondrop or dropframe - sometimes this adjusting does a wrong thing. by now i cant grab that bug (i should do some more debug infos :))
@chmee, just sent you 2 Cdng's converted by raw2cdng v1.4.9 & v1.50 beta5.
As I said v1.4.9 display right but v1.5.0 beta5 has a pink cast in MAC Finder or preview app.
with 16bit linear .
Edit: ML build feb16 NB, 1872x1012. Extracted with MLVBrowseSharp to Raw v1.0 then
converted with v1.4.9 but not on beta5 just drop in your app and converted.
Note* the only difference I notice in ACR in v1.4.9 has a WB of 4950-k and a Locked embedded camera profile
while v1.5.0 beta5 has a WB 5000-k and the Camera profile default Adobe Standard but other profiles are selectable.
sent you my thoughts on your examples :) the finder-result does not interest me, honestly saying. assume, its because of the linearizationtable, thats inside to work with premiere..
the reason, you couldnt use profiles in acr is, you changed the name to "5D Mark2 Cdng 1.4.9". "Canon EOS 5D Mark II" works.
sent you a screen capture of finder with the Cdng's
jepp. in this case (app-world and apps) i cant really help - just to say it officially once again: i assume its about the linearizationtable inside the dng's. will think about it (much later) if i just dont use them, when only linear 16bit dng's are wanted.
Ok, I use PC because it's faster then mac in converting raw and better tools!
I can use the older version I just wanted the new name generator feature on the new one that's all. :)
Hey Chmee,
i was able to repoduce the behavior of 1.5.0beta5 vs. 1.3.3 with a RAW file of Luke Neumanns sheeps :D
http://neumannfilms.net/2013/05/15/magic-lantern-raw-video-workflow-tutorial-downloadable-files/ (http://neumannfilms.net/2013/05/15/magic-lantern-raw-video-workflow-tutorial-downloadable-files/)
Nevertheless I'll try my best to upload my 550D samples the next 2-3 days.
Below a screenshot of the crash showing the progressbar while dropping in the file.
(http://img213.imagevenue.com/loc1145/th_68007_raw2CDNG1_5_crash_LNsheep_122_1145lo.jpg) (http://img213.imagevenue.com/img.php?image=68007_raw2CDNG1_5_crash_LNsheep_122_1145lo.jpg)
i just got a file with the same behaviour (strange i hadnt that on other files, the problem is so obvious.) when raw's have no framerate (did they ever had this metadata?) , it will crash while calculating the duration of the video. if framerate is zero, i set it now to 24fps. today i will put a beta6 on the server.
raw2cdng 1.5.0 beta 6 online
* 12bit maximized should work again (with pinkhighlight-fix as well)
* RAW crash fix (divideByZero-Exception)
* more debugdata
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
Will 12 bit regular return? What's the major difference between maximized that requires the huge increase in processing involved to create them?
. never could find that file you mentioned but ill see if i have any future crashes or issues
Thanks as always! Giving your suggestions regarding filenames using f(_) a try for audio sync in Resolve, as I'm having some issues auto syncing.
Hi Chmee,
is it intentional that that the headline of the app shows "rawcdng 1.5" instead of "raw2cdng 1.5"?
@jbowdach the main difference normal versus maximized is i dont use the original values from canon ( example 5DIII: 2048-15000 from max 16384) but i spread them to the maximum values (0-65535 with 16bit). this is needed for premiere and gives no drawback on other apps. with 16bit we have no problem with changing valuerange, because we're not lowering it. in 12bit we do and every bit we have we should use.
ahh btw. with beta6 i write more from the audioprocess into the debuglog. i want to find this bug for autosync in resolve as well :)
@glubber thanks, will change that :)
Quote from: chmee on May 23, 2014, 12:27:35 AM
@jbowdach the main difference normal versus maximized is i dont use the original values from canon ( example 5DIII: 2048-15000 from max 16384) but i spread them to the maximum values (0-65535 with 16bit). this is needed for premiere and gives no drawback on other apps. with 16bit we have no problem with shortening valuerange, because there isnt. in 12bit we're lowering the bitdepth and every bit we have we should use.
@glubber thanks, will change that :)
Makes sense thanks for the explanation. Seems to save approx 20% space compared to 16 bit conversions based on my tests last night. Will know of sync worked when the get home and can check the render output.
Does premiere not accept 12 bit non maximized cdngs sequences? Haven't played w native cdngs in premiere pro yet
premiere does not work with biased files (zero is not zero). and because premiere in the actual version only accepts bmcc-like cdng-files, i (we?!) had to code this pinkhighlightfix. BUT it works well with 12/16bit maximized pinkhighlight-fixed files.
make the test for yourself, its a huge difference.
(i promise, in the next beta there will be the vertical banding-button, its no1 on todolist)
@a1ex is vertical banding an individual static property of every body? if yes, would it be an idea to write a module to calculate it for the body and save it as a block into the mlv?
From the same body, it seems to differ from clip to clip, but inside one clip it stays pretty much the same.
My heuristic (for speed reasons) is to compute it from the first frame only. This works fine most of the time, but if you start recording with lens cap on for example, it will fail. A better approach would be to take say 5 or 10 frames, evenly spread across the footage, and compute from there. Most likely, averaging the coefficients or taking the median value should do the trick.
I don't know exactly how it behaves with long recordings though, but I assumed the current approach works fine, since there were no relevant bugs reported for this use case.
thx. it was only my assume, the banding is kind of fingerprint per body-electronics. so it was a simple idea to implement the calculations into the body, calculate it once and saving the array of floatingpoint on the body. but it's not i know now.
hi chmee, with beta 6, audio sync in resolve works with certain footage and some others are wrongly synced. Same audio synced twice to different footage.
Just another note which i wanted to share is that, i used to optimize the flickering problem in resolve by changing file name codes in raw2cdng. I had to work with .raw files and .mlv files together for a project. A single timeline in resolve having both raw's cdng and mlv's cdng really threatened me regarding flickering issues. However, i went through resolve's manual and now one can overcome the flickering issue by checking the "Output single field when paused" option from "General Options" in resolve. It worked for me (but I also changed file naming in converter). Hope the solution works and has nothing to do with file naming... thanks
Hi guys,
This could be an old question, but I really need some help:
The only raw2cdng version I can launch are 1.2.0 or olders. With newest: "raw2cdng has stopped working" appear. I´ve tried run as adm, compatibility mode, etc, and nothing.
Do you know why?
OS win7, 64bits
Thanks!
Hmm. It seems, my app cant write some data. It writes two Files:
* debug.log - since 1.4.6, into the same path as the app
* raw2cdng.settings.json - since 1.2.1, into Users/[User]/Appdata/roaming
* Dont use programs or windows as path.
* maybe it has something to do with special chars in your username?
* try a start from root or other harddisk.
* under file-properties, try to set admin static.
* wait a moment, it should show a some debug info, there will be some files in a temp-directory. the small xml-file should show the reason.
* No debug.log file
* No raw2cdng.settings.json file
I found this xml files:
-C:\Users\Dario\AppData\Local\ElevatedDiagnostics\733862231\2014052901.000\PCW.0.debugreport.xml:
miércoles, 28 de mayo de 2014 22:19:02true6.1Clientamd641.5Microsoft WindowsHaga que los programas antiguos se ejecuten en esta versión de Windows.03740x0C:\Windows\diagnostics\system\PCW0x00000000006B5C40<Answers Version="1.0"> <Interaction ID="IT_LaunchMethod"> <Value>ContextMenu</Value> </Interaction> <Interaction ID="IT_SelectProgram"> <Value>NotListed</Value> </Interaction> <Interaction ID="IT_BrowseForFile"> <Value>C:\Users\Dario\Desktop\magic lantern\raw2cdng.1.5.0.beta6.exe</Value> </Interaction> </Answers>50380x803C0100
-C:\Users\Dario\AppData\Local\Temp\WER5B1F.tmp.WERInternalMetadata.xml:
<?xml version="1.0" encoding="UTF-16"?>
-<WERReportMetadata>-<OSVersionInformation><WindowsNTVersion>6.1</WindowsNTVersion><Build>7601 Service Pack 1</Build><Product>(0x30): Windows 7 Professional</Product><Edition>Professional</Edition><BuildString>7601.18113.amd64fre.win7sp1_gdr.130318-1533</BuildString><Revision>1130</Revision><Flavor>Multiprocessor Free</Flavor><Architecture>X64</Architecture><LCID>3082</LCID></OSVersionInformation>-<ProblemSignatures><EventType>CLR20r3</EventType><Parameter0>raw2cdng.1.4.9.exe</Parameter0><Parameter1>1.0.0.0</Parameter1><Parameter2>52f7c322</Parameter2><Parameter3>raw2cdng</Parameter3><Parameter4>1.0.0.0</Parameter4><Parameter5>52f7c322</Parameter5><Parameter6>82</Parameter6><Parameter7>c8</Parameter7><Parameter8>System.IO.FileNotFoundException</Parameter8></ProblemSignatures>-<DynamicSignatures><Parameter1>6.1.7601.2.1.0.256.48</Parameter1><Parameter2>11274</Parameter2><Parameter22>0a9e</Parameter22><Parameter23>0a9e372d3b4ad19135b953a78882e789</Parameter23><Parameter24>0a9e</Parameter24><Parameter25>0a9e372d3b4ad19135b953a78882e789</Parameter25></DynamicSignatures>-<SystemInformation><MID>8C63604C-DCF0-4276-B73A-601CDEFC9D85</MID><SystemManufacturer>System manufacturer</SystemManufacturer><SystemProductName>System Product Name</SystemProductName><BIOSVersion>2102</BIOSVersion></SystemInformation></WERReportMetadata>
- and this txt:
<?xml version="1.0" encoding="UTF-16"?>
<DATABASE>
<EXE NAME="raw2cdng.1.4.9.exe" FILTER="CMI_FILTER_PRIVACY">
<MATCHING_FILE NAME="mlv_dump.exe" SIZE="271511" CHECKSUM="0x89C7DAE7" MODULE_TYPE="WIN32" PE_CHECKSUM="0x48851" LINKER_VERSION="0x10000" LINK_DATE="02/06/2014 20:42:55" UPTO_LINK_DATE="02/06/2014 20:42:55" EXE_WRAPPER="0x0" FILE_ID="000019729fc7e9e575a55adfed708247462ec458211f" PROGRAM_ID="00034d1e3b779acee3608899521a05a279ae0000ffff" />
<MATCHING_FILE NAME="raw2cdng.1.4.9.exe" SIZE="553472" CHECKSUM="0x276FD127" BIN_FILE_VERSION="1.0.0.0" BIN_PRODUCT_VERSION="1.0.0.0" PRODUCT_VERSION="1.0.0.0" FILE_DESCRIPTION="raw2cdng" PRODUCT_NAME="raw2cdng" FILE_VERSION="1.0.0.0" ORIGINAL_FILENAME="raw2cdng.exe" INTERNAL_NAME="raw2cdng.exe" LEGAL_COPYRIGHT="Copyright © 2013" VERDATEHI="0x0" VERDATELO="0x0" VERFILEOS="0x4" VERFILETYPE="0x1" MODULE_TYPE="WIN32" PE_CHECKSUM="0x0" LINKER_VERSION="0x0" UPTO_BIN_FILE_VERSION="1.0.0.0" UPTO_BIN_PRODUCT_VERSION="1.0.0.0" LINK_DATE="02/09/2014 18:04:18" UPTO_LINK_DATE="02/09/2014 18:04:18" VER_LANGUAGE="Independiente del idioma [0x0]" EXE_WRAPPER="0x0" FILE_ID="0000e0cfb8a8373eb6a02cd0171346989ea56ad09f07" PROGRAM_ID="000391263d3901eb726be10e0256222aeca100000000" />
</EXE>
<EXE NAME="KERNELBASE.dll" FILTER="CMI_FILTER_THISFILEONLY">
<MATCHING_FILE NAME="KernelBase.dll" SIZE="274944" CHECKSUM="0xE4148384" BIN_FILE_VERSION="6.1.7601.18015" BIN_PRODUCT_VERSION="6.1.7601.18015" PRODUCT_VERSION="6.1.7601.18015" FILE_DESCRIPTION="Archivo DLL de cliente API BASE de Windows NT" COMPANY_NAME="Microsoft Corporation" PRODUCT_NAME="Sistema operativo Microsoft® Windows®" FILE_VERSION="6.1.7601.18015 (win7sp1_gdr.121129-1432)" ORIGINAL_FILENAME="Kernelbase" INTERNAL_NAME="Kernelbase" LEGAL_COPYRIGHT="© Microsoft Corporation. Reservados todos los derechos." VERDATEHI="0x0" VERDATELO="0x0" VERFILEOS="0x40004" VERFILETYPE="0x2" MODULE_TYPE="WIN32" PE_CHECKSUM="0x4AA27" LINKER_VERSION="0x60001" UPTO_BIN_FILE_VERSION="6.1.7601.18015" UPTO_BIN_PRODUCT_VERSION="6.1.7601.18015" LINK_DATE="11/30/2012 04:56:42" UPTO_LINK_DATE="11/30/2012 04:56:42" EXPORT_NAME="KERNELBASE.dll" VER_LANGUAGE="Español (España, internacional) [0xc0a]" EXE_WRAPPER="0x0" FILE_ID="000090b069cbc93c701394b47d4e830cdd7d384f5d5c" PROGRAM_ID="0000f519feec486de87ed73cb92d3cac802400000000" />
</EXE>
<EXE NAME="kernel32.dll" FILTER="CMI_FILTER_THISFILEONLY">
<MATCHING_FILE NAME="kernel32.dll" SIZE="1114112" CHECKSUM="0x9725986B" BIN_FILE_VERSION="6.1.7601.18015" BIN_PRODUCT_VERSION="6.1.7601.18015" PRODUCT_VERSION="6.1.7601.18015" FILE_DESCRIPTION="Archivo DLL de cliente API BASE de Windows NT" COMPANY_NAME="Microsoft Corporation" PRODUCT_NAME="Sistema operativo Microsoft® Windows®" FILE_VERSION="6.1.7601.18015 (win7sp1_gdr.121129-1432)" ORIGINAL_FILENAME="kernel32" INTERNAL_NAME="kernel32" LEGAL_COPYRIGHT="© Microsoft Corporation. Reservados todos los derechos." VERDATEHI="0x0" VERDATELO="0x0" VERFILEOS="0x40004" VERFILETYPE="0x2" MODULE_TYPE="WIN32" PE_CHECKSUM="0x11B7A6" LINKER_VERSION="0x60001" UPTO_BIN_FILE_VERSION="6.1.7601.18015" UPTO_BIN_PRODUCT_VERSION="6.1.7601.18015" LINK_DATE="11/30/2012 04:56:41" UPTO_LINK_DATE="11/30/2012 04:56:41" EXPORT_NAME="KERNEL32.dll" VER_LANGUAGE="Español (España, internacional) [0xc0a]" EXE_WRAPPER="0x0" FILE_ID="0000f9ac70c562ba70dd7917e99b00ed6878531de66f" PROGRAM_ID="0000f519feec486de87ed73cb92d3cac802400000000" />
</EXE>
</DATABASE>
* I´ve tried in an old xp and it works!
Any idea?
Is .net4.0 installed?
It´s installed.
A while ago I made some changes in the regional language settings for another program compatibility (pro tools), but today I restored to default and nothing happened.
Could be something there, maybe?
Thanks for your time chmee
try to save an empty textfile into the roaming-path i described before, then name it raw2cdng.settings.json or (>=1.5.0) raw2cdng.2.settings.json.
Done, but nothing has changed.
What about updating to netframework 4.5? It´s safe? Vulnerability risk or something?
someone's said, it needs 4.5 - didnt found the parts i used 4.5-code.. so. the main thing is, 4.5 is NOT backward compatible to 4.0 - 4.5 is NOT an update from 4.0. if you wanna try and tell me if it works after installing 4.5, would be great. THEN i would analyze my code once again for this dotnet-thing.
I just wanted to let you know that I used 1.5 Beta 6 and did not have any issues on my Windows 8.1, GeForce GTX 780, and pulling the footage into Premiere CC with audio synced. No pink highlights or green frames. Even when viewing the DNGs in Explorer, the images were scaled correctly and the colors were represented properly. Great work building this from the bottom up! Excited to see more to come and what Adobe does with the rumors of more DNG support.
Hey chmee, I wanted to offer you some testing results regarding the timecode issue of the WAVs auto-sync in Resolve, I've read that your software uses the datetime of the file to generate the timecode. I've tested this and I found why Resolve it's not syncing some of the files properly. I used two files M03-1753 and M03-1755.
The timecode generated for M03-1713 and its WAV was 20:15:59:00 to 20:16:28:12 (6 more for the WAV)
The timecode generated for M03-1715 was 20:15:28:00 - 20:16:05:18 (5 more for the WAV)
I don't think is a problem of timecode generation. Its consistent for video and audio, and they seem to match but, as you see, the timecodes for the two files are not unique and overlap, therefore, when Resolve syncs them it goes to the first file it can find that has some of that timecode (the first file when shorted by name that has some of that time interval, it defaults to that and ignores the reelname - because audio doesn't catch the reelname from the folder).
The timecode match the date of creation of the file on the hard drive, that means when the .MLV was copied to the hard drive, which obviously makes files overlap because the creation times are usually close.
I believe there can be a quick workaround for this (the proper fix would be emulating the BMCC WAVs) taking the timecode start from the name of the file i.e: "1713". As I mention this would be just a workaround because in the case of having files created at the "same time" in different days the timecode would overlap, also when using different cameras, and, Resolve would go crazy again. But this is solvable. When put in different folders of the Media Pool in Resolve (Day 1, Day 2 / Camera A, Camera B...) And selecting the clips in those folders, then clicking the auto-sync, the two files sync properly and only with the WAVs that have the same timecode in that folder. If timecodes did not overlap in one folder, we should be just fine. It's not perfect, but it works.
Well, in any case, I hope some of this testing offers some insight and helps in solving the problem.
Tech details:
- Camera: 5D Mark III
- raw2cdng 5.0beta6
- MLV / Nightly ML build of February 14th 2014 (I know, it's an old build, but I'm not aware of any changes on the timecode/name/file creation process that can influence the issue reported)
Create clean cDNG´s from MLV would be so great! Cant wait!
Quote from: Johannes on June 22, 2014, 10:04:19 PM
Create clean cDNG´s from MLV would be so great! Cant wait!
Raw2CDNG 1.5.0 beta 6 is what I use and it does break MLV files apart into wav and CDNG..
Quote from: ChadMuffin on June 07, 2014, 02:54:27 PM
I just wanted to let you know that I used 1.5 Beta 6 and did not have any issues on my Windows 8.1, GeForce GTX 780, and pulling the footage into Premiere CC with audio synced. No pink highlights or green frames. Even when viewing the DNGs in Explorer, the images were scaled correctly and the colors were represented properly. Great work building this from the bottom up! Excited to see more to come and what Adobe does with the rumors of more DNG support.
Chad - ditto here. I use 1.5.0 beta 6 with Adobe Premiere CC 2014 on Windows 8.1, GeForce GTX 670M, and have no issues.
thanks. so, next days i will turn back on coding. i'll try to implement vertical banding and chroma smoothing. as i've seen, the new dng-import-module in premiere 8.0 does pink highlights as well. hummm, and they#re working on a fix. do some tests today. i hoped, i can delete/forget the highlightfix.
@sergiocamara93
thanks for tests. it wonders me, why it behaves this way. its not unusual, that there are more files with nearly the same timecode (multicam).
Quote..proper fix would be emulating the BMCC WAVs..
by now i tried to do my best to implement all things, that are in bmcc-wav files.
http://www.magiclantern.fm/forum/index.php?topic=11520.0
by the way. i told days ago i wrote a diagnostic tool for mlv-files. here it is.
mlvdiag 1.0.0 - download here (http://dl.phreekz.de/raw2cdng/mlvdiag.1.0.0.zip)
whats the idea behind it?
* analytic look into the blockdata saved inside the mlv
* important for me: saving a minimal set of blocks into a new file for app-problems and debugging
-> it writes all metadata, one videoframe and one audiochunk into a new file, found on same place as original.
-> if you have some problems with raw2cdng-crashes this minimal mlv can help me to find those bugs.
* its not perfect, but it works :) ('til now)
(http://dl.phreekz.de/raw2cdng/mlvdiag.jpg)
regards chmee
Oh, Sweet! Thanks, chmee! :3
THis looks like a fantastic diagnostic tool to help resolve issues w problematic MLV files. THanks so much! Looking forward to the next version of raw2cdng.
Please let us know how we can use it to either fix files, and\or use it to provide you with useful diagnostic information
In beta 6, I still cannot scroll through the files without the app crashing
Just a thought for a request. I have seen in the MLV to DNG Batch Converter v1.9 there is an option to use an ACR profile when exporting the DNGs. The idea behind it is to apply an ACR preset that is a flat profile or one that you would create for what ever you have in mind. Then, you could either bring those into AE to export to ProRes or what ever codec you want OR bring those right into Premiere or Resolve with that preset applied. That would save a ton of clicks, especially with larger projects with bringing them with a flat preset applied already. However, I do not know if Premiere or Resolve will read that applied preset since cDNG is a different process than just the normal DNG (to my knowledge).
Quote from: ChadMuffin on June 28, 2014, 03:28:53 AM
However, I do not know if Premiere or Resolve will read that applied preset since cDNG is a different process than just the normal DNG (to my knowledge).
No, they won't. PP uses Speedgrade framework which doesn't read ACR and the same goes for Resolve (where you can apply a flat profile - BMPCC - and then a LUT of your choice to all your footage. Plus it encodes faster).
Forgot to mention that MLV to DNG Batch Converter v.1.9 generates DNG, not CDNG. So PP won't read them at all, for Resolve it's about the same between the two formats right now (reads both equally).
@sergiocamara93
Thanks for the info! Never mind about my request then. heh.
Hi, from my 2TB shooting I have a trouble opening one of my file. The loader bar is still loading, both raw2cdng and mlvdiag.
What i did -- downloaded all the mlv's, deleted those 1 bytes and batch renamed all the mlv's (because of Resolve round-trip method).
The debug info:
[drop] ------
[drop] -- file K:\xxx\002_M0336.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in K:\xxx\002_M0336.MLV
My theory is that the end of this file was splitted into those 1byte chunks and sice I deleted them, raw2cdng don't know where to find them? The progress bar is still active, but the disc activity is zero.
EDIT: But I deleted many 1 byte files and from 399 MLV's only with this I am getting error. The file exctracts with MLV Converter 1.9.2 fine. :(
@kaco
can you open it with mlvdiag (https://secure.magiclantern.fm/forum/index.php?topic=5618.msg119737#msg119737)? Oh Sorry, see now, you tried. ok. will change the reader module in mlvdiag/raw2cdng to allow open/read "corrupted"/strange blocks.
@chmee: now tested with mlv convertor 1.9.2 ... the files open, but won't extract... since it's a 80gb of data I think there's something important in it.. :( oh, god...
Do you think there is a chance to get the data from the file?
tested it also with mlv2dng:
mlv2dng.exe 002_M0336.MLV
Exception! Header not recognized: ''
Exception! Header not recognized: ''
mlv_dump:
mlv_dump.exe 002_M0336.MLV
MLV Dumper v1.0
-----------------
Mode of operation:
- Input MLV file: '002_M0336.MLV'
- Verify file structure
File 002_M0336.MLV opened
File 002_M0336.M00 not existing.
Processing...
[ERROR] Missing file header
Processed 0 video frames
Done
Do you have a Hex-File-Editor like HXD (http://mh-nexus.de/de/hxd/)? Open the file with it, copy the first 1MB (or 5 or 10) to a new file and send it to me.
[email protected] (i will code such a button into mlvdiag).. Ahh, please write down the original and ml-firmware-version and body, maybe its a hint for @a1ex on a bug. maybe..
as you assumed in your pm, this file is empty, full with zeros.
regards chmee
Chmee, I've got visual studio 2012 and have been playing with the source a bit, added some exception code due to warnings, etc. I'd be happy to help out on the code base, I'm trying to get my C# skills up, I've got a class on it this semester, so any tinkering I can do will help me. Also, I can help try and add some more try/catch blocks for some of the I/O type errors we've been seeing, but wanted to make sure you're needing help with the program.
~Phil
@Phil
of course, lets start a workflow with bitbucket.
hey chmee,
here are a few logs from attempting to decode MLV files from my latest vacation clips. These are all shot on a 7D. Tried numerous times, and it seems some of these files just dont want to decode, they just crash the application. Only several files out of almost a TB of video tho, so not horrible. Just wish I knew what was causing these crashes. Please let me know if I can provide any further info about any specific file.
Smaller log1 error:https://www.dropbox.com/s/3szc3mruyhop361/raw2cdng.2.debug_crash630_alaska.log (https://www.dropbox.com/s/3szc3mruyhop361/raw2cdng.2.debug_crash630_alaska.log)
Log 2 error log: https://www.dropbox.com/s/pschwkg4derpi1e/raw2cdng.2.debug_crash2_630alaska.log (https://www.dropbox.com/s/pschwkg4derpi1e/raw2cdng.2.debug_crash2_630alaska.log)
Log 3 Errors: https://www.dropbox.com/s/ts3z31ejyunnn8q/raw2cdng.2.debug_crash3_630alaska.log (https://www.dropbox.com/s/ts3z31ejyunnn8q/raw2cdng.2.debug_crash3_630alaska.log)
BIG log file: https://www.dropbox.com/s/houu1koi7ff3yw1/raw2cdng.2.debug.log (https://www.dropbox.com/s/houu1koi7ff3yw1/raw2cdng.2.debug.log)
chmee what happened to the development of this program? not a single update in two months? bummer.
+1
guys. sorry, really.. it just has been full o' work, and it is still. next days i'll publish a beta7 with some fixes but *SORRY* still no chroma and vertical banding. on the end of this month i'll start my holiday-time with more time for this project again.
to be honest, i didnt found the trick with chroma and vertical banding - i just spent far too little time to understand the sourcecode - and i think about changing the whole code to byte-arrays (instead of uint16) to be on a better track for that. cant say, if it changes the transcoding time. on the todolist is a c#-dll-library for mlv-things as well, cause i use nearly the same code on raw2cdng and mlvdiag. makes sense :) but positive news: a german guy mailed, he wants to help on raw2cdng, maybe we get all the transcoding-code in c++, hoping it speeds up the thing.
regards chmee
Glad to see more things coming and expanding the team! Excited for more developments.
+1
short info on some work: as someone stated i didnt used the WBAL-data from MLV (https://docs.google.com/spreadsheet/ccc?key=0AgQ2MOkAZTFHdHJraTVTOEpmNEIwTVlKd0dHVi1ULUE#gid=0). the values are kind of tricky. why? The values inside WBAL are only usable under some circumstances:
these are (by now) the interesting data-fields in WBAL
Quote
wb_mode - from 0 to 6 and 9
kelvin - only accurate if wb_mode = 9
wbgain_r ,_g and _b - only accurate if wb_mode = 6
If you use wb-Presets (tungsten, shade, cloudy etc), i only see the preset, but neither kelvin nor wb_gain is right (there will be old values from the moment you used them last time). If you use KELVIN (manual setting), only kelvin will be right, the wbgain-values are not refreshed. as well, wb_mode MANUAL leads to accurate wbgain-values, but kelvin wont be refreshed.
(by the way)
Quote
wbs_gm - is the known green-magenta-shift, i assume, its a simple value shift green versus red/blue
wbs_ab - assume its a YPbPr-Shift (Lab)
Why do i write this? it seems, it makes sense to write down all necessary fields/values and save them as lists into the main ml-modules or into the converter. If you look into a cr2-file from your body, you ll find the presets. more: to get the most informations we should save the values from all possible Kelvin-Steps.
(1) i did pictures with all kelvin-steps (5DIII 2.500-10.000).
(2) saved them into a folder
(3) then i used the great exiftool to extract the wbgain-fields.
save this code into a file named wb_manual.fmt into this folder:
$ColorTemperature,${WB_RGGBLevelsAsShot;tr/ /,/}
and run this via command-line inside the folder you saved the cr2-files.
exiftool -p wb_manual.fmt *.CR2 > list.csv
voila, you ve got a commaseparated list - kelvin to rggb-values. here you see it inside excel:
(http://dl.phreekz.de/raw2cdng/wb_5DIII_113.jpg)
then do this format-file (name it wb_presets.fmt) on only one cr2-file to save the presets:
Daylight,$ColorTempDaylight,${WB_RGGBLevelsDaylight;tr/ /,/}
Shade,$ColorTempShade,${WB_RGGBLevelsShade;tr/ /,/}
Cloudy,$ColorTempCloudy,${WB_RGGBLevelsCloudy;tr/ /,/}
Tungsten,$ColorTempTungsten,${WB_RGGBLevelsTungsten;tr/ /,/}
Fluorescent,$ColorTempFluorescent,${WB_RGGBLevelsFluorescent;tr/ /,/}
Flash,$ColorTempFlash,${WB_RGGBLevelsFlash;tr/ /,/}call it:
exiftool -p wb_presets.fmt [CR2-FILENAME] > wb_presets.csvafter that we've got the rggb-values for the wb_mode's 0-5. for 5DIII fw 1.1.3 these are the values:
Daylight,5200,2052,1024,1024,1678
Shade,7000,2362,1024,1024,1431
Cloudy,6000,2208,1024,1024,1544
Tungsten,3200,1479,1024,1024,2497
Fluorescent,3714,1796,1024,1024,2399
Flash,6320,2284,1024,1024,1520
so, now i've got all values to give the cinemadngs (nearly) accurate rggb-values to simplify the workflow inside the editing tools.
and now? whats the problem? it seems (look inside a cr2) these rggb-values from 5dIII are not accurate for all bodies. i will build this things into the next raw2cdng-version, but maybe its necessary to do these rggb-lists for every modell - and maybe these values change with the firmware. if someone's got some time for that, try to build these lists as well..
@g3gg0 , @baldand and @a1ex was this work somewhere done already?
@jbowdach i didnt forgot you, right now i just have the time to solve actual things. if you've got a "smaller" non-functional file, please send it to me.
regards chmee
Hey there,
unless I did not make any mistake, the values are different on my 5D mark III's:
1. Older 5D mark III, bought in 2012, SN 013021001***, FW 1.1.3
2500,1182,1024,1024,31392600,1226,1024,1024,30222700,1274,1024,1024,29212800,1326,1024,1024,28192900,1369,1024,1024,27523000,1413,1024,1024,26893100,1450,1024,1024,25763200,1489,1024,1024,24673300,1526,1024,1024,23993400,1565,1024,1024,23303500,1606,1024,1024,22603600,1638,1024,1024,22083700,1672,1024,1024,21533800,1708,1024,1024,20973900,1739,1024,1024,20564000,1768,1024,1024,20164100,1802,1024,1024,19784200,1833,1024,1024,19384300,1859,1024,1024,19074400,1886,1024,1024,18724500,1913,1024,1024,18434600,1942,1024,1024,18114700,1967,1024,1024,17804800,1990,1024,1024,17534900,2009,1024,1024,17275000,2028,1024,1024,16995100,2048,1024,1024,16755200,2068,1024,1024,16495300,2089,1024,1024,16315400,2110,1024,1024,16135500,2131,1024,1024,15965600,2153,1024,1024,15775700,2171,1024,1024,15635800,2189,1024,1024,15475900,2208,1024,1024,15336000,2226,1024,1024,15176100,2241,1024,1024,15076200,2255,1024,1024,14946300,2270,1024,1024,14836400,2284,1024,1024,14716500,2300,1024,1024,14586600,2315,1024,1024,14486700,2330,1024,1024,14366800,2346,1024,1024,14276900,2362,1024,1024,14157000,2378,1024,1024,14047100,2394,1024,1024,13967200,2405,1024,1024,13897300,2416,1024,1024,13807400,2427,1024,1024,13727500,2439,1024,1024,13647600,2450,1024,1024,13577700,2461,1024,1024,13487800,2473,1024,1024,13417900,2485,1024,1024,13328000,2497,1024,1024,13268100,2509,1024,1024,13178200,2521,1024,1024,13118300,2533,1024,1024,13038400,2545,1024,1024,12988500,2551,1024,1024,12938600,2558,1024,1024,12878700,2570,1024,1024,12828800,2576,1024,1024,12768900,2583,1024,1024,12719000,2595,1024,1024,12669100,2602,1024,1024,12609200,2608,1024,1024,12569300,2615,1024,1024,12509400,2628,1024,1024,12459500,2635,1024,1024,12419600,2641,1024,1024,12359700,2655,1024,1024,12319800,2661,1024,1024,12259900,2668,1024,1024,122110000,2675,1024,1024,1215
Daylight,5200,2068,1024,1024,1649
Shade,7000,2378,1024,1024,1404
Cloudy,6000,2226,1024,1024,1517
Tungsten,3200,1489,1024,1024,2467
Fluorescent,3714,1817,1024,1024,2372
Flash,6320,2305,1024,1024,1492
1. Newer 5D mark III, bought in 2014, SN 268021002***, FW 1.1.3
2500,1159,1024,1024,30132600,1201,1024,1024,29052700,1247,1024,1024,28042800,1295,1024,1024,27092900,1334,1024,1024,26483000,1376,1024,1024,25833100,1413,1024,1024,24793200,1450,1024,1024,23783300,1485,1024,1024,23153400,1522,1024,1024,22503500,1558,1024,1024,21853600,1589,1024,1024,21363700,1621,1024,1024,20853800,1654,1024,1024,20323900,1683,1024,1024,19934000,1713,1024,1024,19564100,1745,1024,1024,19204200,1774,1024,1024,18834300,1799,1024,1024,18534400,1824,1024,1024,18244500,1846,1024,1024,17924600,1872,1024,1024,17654700,1896,1024,1024,17364800,1917,1024,1024,17114900,1935,1024,1024,16865000,1956,1024,1024,16595100,1975,1024,1024,16365200,1993,1024,1024,16115300,2013,1024,1024,15945400,2032,1024,1024,15775500,2052,1024,1024,15605600,2072,1024,1024,15425700,2093,1024,1024,15295800,2110,1024,1024,15135900,2127,1024,1024,15006000,2144,1024,1024,14856100,2158,1024,1024,14756200,2171,1024,1024,14646300,2185,1024,1024,14526400,2198,1024,1024,14426500,2212,1024,1024,14316600,2226,1024,1024,14216700,2241,1024,1024,14096800,2255,1024,1024,14006900,2270,1024,1024,13897000,2284,1024,1024,13787100,2300,1024,1024,13717200,2310,1024,1024,13647300,2320,1024,1024,13557400,2330,1024,1024,13487500,2341,1024,1024,13417600,2351,1024,1024,13327700,2367,1024,1024,13267800,2378,1024,1024,13177900,2389,1024,1024,13118000,2399,1024,1024,13048100,2411,1024,1024,12968200,2422,1024,1024,12908300,2433,1024,1024,12828400,2444,1024,1024,12778500,2450,1024,1024,12738600,2461,1024,1024,12668700,2467,1024,1024,12628800,2473,1024,1024,12578900,2485,1024,1024,12519000,2491,1024,1024,12479100,2497,1024,1024,12429200,2509,1024,1024,12379300,2515,1024,1024,12329400,2521,1024,1024,12289500,2533,1024,1024,12229600,2539,1024,1024,12189700,2545,1024,1024,12149800,2558,1024,1024,12089900,2564,1024,1024,120410000,2570,1024,1024,1198
Daylight,5200,1993,1024,1024,1611
Shade,7000,2284,1024,1024,1378
Cloudy,6000,2144,1024,1024,1485
Tungsten,3200,1450,1024,1024,2378
Fluorescent,3714,1751,1024,1024,2280
Flash,6320,2217,1024,1024,1460
For instance also the shutter sounds a bit different on these, so maybe there's some little design difference?
I also doublechecked that no WB shift was set on both and that both had same settings, same standard profile, disabled all "enchance" functions, etc.
@kaco
thanks a lot. i assumed, that every sensor has got its own indivdual adjustment - maybe once on build or maybe once on switch on or firmware-update.. means, every 5DIII has a slightly other characteristic curve. but for the job as a converter-app with static rggb-values it should be enough - OR - i/we apply this data as external files - i dont think its worth it.
now could be interesting to see, how the curves differ on other models.
thanks and regards chmee
For me it's totally OK to use some "average" values. This values will still be used only when creating proxies and then finetuned in the final color grade.
Im having sync issues when I autosync in Resolve. Theres a consistent gap at the beginning of the audio. Anyone else having this issue or other audio sync issues?
@jbowdach
i just did a small test with ~25 files, recorded (in terms of timecode) in nearly same time. after autosync it went good, no gaps, no real problems. i assume, my timecode-calculations for the audiotrack going wrong if they're broken framerates. did you checked the timecode inside the files?
(http://dl.phreekz.de/raw2cdng/resolve_timecode_sync_audio.jpg)
just found that: maybe thats (http://www.magiclantern.fm/forum/index.php?topic=11017.msg124784#msg124784) kind of similar?
regards chmee
btw: i implemented the WBAL-usage and resolve, premiere and photoshop behaves quite well. did some more exception-trapping and debug-writing. beta7 is near.
Quote from: chmee on August 06, 2014, 11:01:33 AM
btw: i implemented the WBAL-usage and resolve, premiere and photoshop behaves quite well. did some more exception-trapping and debug-writing. beta7 is near.
very nice!
raw2cdng 1.5.0 beta7 online
downloadlink (http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/)
* 12bit shouldnt crash anymore with big/strange resolutions
* WBAL from MLV or CR2 (presets and manual as well) test!
* debuglog reorganized - readability
* if ffmpeg.exe in same folder - proxy output will be as mpeg2 as well (half resolution) (you see mpeg2 in button if ffmpeg is recognised)
regards chmee
@chmee I think I figured my issue. I was bringing in all the files in one folder. Didnt realize I need to bring in EACH folder individually, and use the autosync option on the folder for each clip. Im going to try a mass sync tonight and see how it work along w beta7, as Ive had a few crashes w beta6 (mainly related to scrolling when it is reviewing the clips after dragging them).
Many thanks for the update and testing! :)
you can drag/drop them all first into the mediapool. after that you can select them (all) and use autosync. should work.
@jbowdach crash issues mentioned on bitbucket (https://bitbucket.org/chmee/raw2cdng/issue/20/crash-when-attempting-to-drag-in-mlv-files)
first debug: [beta7] failure on DayFive/M01-0925.MLV (Exception listed at the end, couldnt read MLV because of negative fileofffset, zero frames?!)
second debug: [beta7] failure on DaySeven/M03-1132.MLV (fpsnom is zero and found no VIDF-Block)
third debuglog: [beta6] hmm. Day Five\M01-1214.MLV. somewhere while identifying VIDF-Blocks. please try this one with beta7.
mainly, it seems, something disturbes the reading of the vidf-blocks and/or the blocks at all. If you ha a shorter file from your recording you could send me, it would be great. Did you tried to read the files with mlvdiag (https://secure.magiclantern.fm/forum/index.php?topic=5618.msg119737#msg119737)? shorten a file with it and send it to me. or use HxD and copy the first ~50MB of M03-1132 (and/or M01-0925.MLV) into a new file. send this one.
bytheway: Thanks a lot for the help (and patience) to you all!!
edit:
This is raw2cdng 1.5.0 beta7 under win7/win8 with aero and enabled ffmpeg.exe
(http://dl.phreekz.de/raw2cdng/raw2cdng_150b7.jpg)
and an hour ago i put mlvdiag into the raw2cdng-zip (download it again if needed). now in version 1.0.1 with a light helper. just drag/drop a file on the rectangle on bottom, and it will be copied into a 55mb big file with the ending ".55mb.mlv". no checks, just to copy the first 55mb of the main file to give me a chance to deal with some of the mysterious files.
(http://dl.phreekz.de/raw2cdng/mlvdiag_101.jpg)
Will do with the problem files so you can take a look at them. No clue why these are recording with errors, as they were all recorded at the same time with the same build on the same cards. What's strange as I had 4 previous days FULL of MLVs that decoded without any crashes......(confused). Not sure why it started having issues on Day 5 and 7 (didnt shoot on day 6)
No worries, thank you for your help in diagnosing the issues and debug logs.
@chmee
Please find the links below to the files you requested.
https://dl.dropboxusercontent.com/u/787062/raw2cdng/M01-0925.MLV.55mb.mlv (https://dl.dropboxusercontent.com/u/787062/raw2cdng/M01-0925.MLV.55mb.mlv)
https://dl.dropboxusercontent.com/u/787062/raw2cdng/M01-1214.MLV.55mb.mlv (https://dl.dropboxusercontent.com/u/787062/raw2cdng/M01-1214.MLV.55mb.mlv)
please give me DaySeven/M03-1132.MLV as well. on a short test i could use both shortened mlv's without problems.
Are your problematic files seeable in mlvdiag (dragdrop into list)?
The file appears to be damaged, this is the file as copied off the card. All files came up fine in mkvdiag.
https://dl.dropboxusercontent.com/u/787062/raw2cdng/M03-1132.MLV (https://dl.dropboxusercontent.com/u/787062/raw2cdng/M03-1132.MLV)
On another note, I all of a sudden cant launch raw2cdng. Something happened w my system. ever seen anything like the following?
Problem signature:
Problem Event Name: CLR20r3
Problem Signature 01: raw2cdng.1.5.0.beta7.exe
Problem Signature 02: 1.5.0.0
Problem Signature 03: 53e3fb45
Problem Signature 04: System.Web.Extensions
Problem Signature 05: 4.0.30319.18408
Problem Signature 06: 523111a8
Problem Signature 07: 1a2
Problem Signature 08: ed
Problem Signature 09: System.Windows.Markup.XamlParse
OS Version: 6.1.7601.2.1.0.256.4
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789
EDIT: Not sure what caused it, uninstalled\reinstalled .net and still didnt work. Ended up restoring an image backup to a few days ago (lucky me!)
Hi, first of all thanks for that neat piece of software!
I use it to get mp4 proxies but I have an issue about the color though, the jpegs are off, lifted blacks, wrong gamma.
I copied ffmpeg in the folder and used the proxy option.
Is there a way to get jpegs with the same look as the dng please?
I searched the thread but couldn't find any workaround for this, I apologize if this has been answered before.
Thanks!
(answering from a smartphone) the proxy-option was/is only for cutting purposes. but i will optimize it in the next Version..
Quote from: chmee on August 12, 2014, 12:11:02 AM
(answering from a smartphone) the proxy-option was/is only for cutting purposes. but i will optimize it in the next Version..
thanks for the quick answer and thanks for the work, it's the only program that has worked with MLVs for me so far
@chmee: in beta7 there is still present the bug that the app will crash when scrolling all loaded MLV files during convert
+ it would be perfect if folder drop will be working. Now I use 3 cameras -- I have to copy the mlvs into folders "card1, card2,.." because 2 cameras can create same named files. Then I convert MLVs to cdngs with namespace "[D2]_[F]_[P]_". But I have to manually drop every folder, which is time consuming if I have 40 cards and so.
And very nice future feature would be progress bar in the taskbar button. You know like when you copy files with explorer, the icon in the taskbar will start to fill its background with green. So you know the app is working and you'll see once it's done :)
I have a problem converting one file. It's the last file on the card, so maybe it's corrupted, but extracts with MLV Converter 1.9.2 fine.
Here's debug:
------------- 1.5.0.BETA7 started at 14.08.12 23:49:48 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M00
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M01
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M02
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M03
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M04
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M05
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M06
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M07
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M08
[drop] O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.MLV
[drop] ------
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M00 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M01 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M02 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M03 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M04 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M05 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M06 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M07 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.M08 will be analyzed now.
[drop] -- file O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (10)
[createMLVBlocklist] indexing Blocks in O:\2014-07-08 Freedomky dotacky MLV\k3\M08-1137.MLV
[createMLVBlocklist] --- exception(!) ---
[createMLVBlocklist] blockTag|offset|fileNo : &???|-199613494|0
Message = Non-negative number required.
Parameter name: value
ActualValue =
ParamName = value
Data = System.Collections.ListDictionaryInternal
InnerException =
TargetSite = Void set_Position(Int64)
StackTrace = at System.IO.FileStream.set_Position(Int64 value)
at raw2cdng_v2.io.createMLVBlockList(String filename, raw raw)
HelpLink =
Source = mscorlib
HResult = -2146233086
Also won't open with mlv diag, probably the same problem. The progress is stuck on loading.
Is there any way to integrate pink dot removal or chromasoothing for us t4i users?
@kaco
thanks for ideas.
folderimport is coded (useable with P-Tag, as you've shown)
progress on taskicon, i will see, minor importance.
your file, thats crashes the app, seems to deliver wrong offset-data straight on first block. thats strange and could be a hint on a undeleted value somewhere. will do some longer recording to test it. could be equal to @jbowdachs problems.
@edge11
sorry, i didnt found the time to implement it.. its on the list and if this is done i'd change the versionnumber to some more :) if someone could help me, would be great.
by now i'm on holidays, for the next 10 days. i try to bugfix and implement as much as possible.
regards chmee
chmee is there anyway to mass import the jpg sequences into premiere? it seems i have to do them all individually.
I don't know if debugging and impletementing your program is a holiday for you or not but, thanks for being a team player! I know ML users will be happy!
Quote from: zacshome on August 14, 2014, 11:51:29 AM
chmee is there anyway to mass import the jpg sequences into premiere? it seems i have to do them all individually.
in premiere open the import windows, select the first file of the sequence and in the window there should be checkbox "numbered stills". Then you import jpeg sequence as a "video" and you can change properties and set correct FPS.
Quote from: ChadMuffin on August 14, 2014, 03:10:30 PM
I don't know if debugging and impletementing your program is a holiday for you or not but, thanks for being a team player! I know ML users will be happy!
Another massive thanks for a happy ML \ Cdng user. You've created the best solution for MLV to cdng for windows so far, otherwise we'd still be stuck in DNG sequence land
very similar problem as the last one:
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M00 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M01 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M02 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M03 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M04 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M05 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M06 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M07 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.M08 will be analyzed now.
[drop] -- file O:\2014-06-23 Freedomky MLV\M23-1239.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (10)
[createMLVBlocklist] indexing Blocks in O:\2014-06-23 Freedomky MLV\M23-1239.MLV
[createMLVBlocklist] --- exception(!) ---
[createMLVBlocklist] blockTag|offset|fileNo :
x6?|-1118282230|0
Message = Non-negative number required.
Parameter name: value
ActualValue =
ParamName = value
Data = System.Collections.ListDictionaryInternal
InnerException =
TargetSite = Void set_Position(Int64)
StackTrace = at System.IO.FileStream.set_Position(Int64 value)
at raw2cdng_v2.io.createMLVBlockList(String filename, raw raw)
HelpLink =
Source = mscorlib
HResult = -2146233086
sorry guys. debugging on bigger files does not work here where i am, instead of this i coded a neat extra and optimized the proxy output.
raw2cdng 1.5.0 beta8 (http://dl.phreekz.de/raw2cdng/raw2cdng.1.5.0.beta8.zip) is online.
* mlvplayer embedded - glue the file extension MLV to this app and lets play it :) RAW not tested yet. (no audio)
* whitebalance optimized (hopefully) on jpg/mpeg2 output
* no rec709-log/lut but gamma 2 on proxyoutput.
* fixed glitch on mpeg2-output (length was taken by audio, now it should be ok)
* folder dragdropable. Use P-tag.
for sure the playback is not accurate, but massively helps to overview recordings. and it looks better than ffplay (sorry, this was my first idea)
regards chmee
Quote from: chmee on August 17, 2014, 04:10:54 PM
sorry guys. debugging on bigger files does not work here where i am, instead of this i coded a neat extra and optimized the proxy output.
raw2cdng 1.5.0 beta8 (http://dl.phreekz.de/raw2cdng/raw2cdng.1.5.0.beta8.zip) is online.
* mlvplayer embedded - glue the file extension MLV to this app and lets play it :) RAW not tested yet. (no audio)
* whitebalance optimized (hopefully) on jpg/mpeg2 output
* no rec709-log/lut but gamma 2 on proxyoutput.
* fixed glitch on mpeg2-output (length was taken by audio, now it should be ok)
* folder dragdropable. Use P-tag.
for sure the playback is not accurate, but massively helps to overview recordings. and it looks better than ffplay (sorry, this was my first idea)
regards chmee
Many thanks for the folder option, what do you mean by using the p-tag? For what reason? Seems to work just by dragging a folder
not a Must, but a hint for the filename-generator.
Hey Chmee, trying to get the mpeg proxies to work, I got the exe for ffmpeg in where raw2cndg is and it has the mpeg option next to the jpg proxi text, i click this hoping to make the mpeg proxies but it still just creates jpg's.
Any help with this? Essentially do they also have the same file name, so if i edl//xml the proxies out of premiere into resolve, they will link strait up with the dng files?
Look into the debug. The ffmpeg error should stay there. The mpgs should be indeed in the same folder as the dngs and jpgs.
Chmee,
All my timecode are wrong. I use Resolve and when I import the CDNG sequence it starts at like 17th hour time code. I just want the timecode to be zero. Because when I export into my editor and use After Effects, it starts on 170000+ starting frames!
Let me know what I can do?
Thanks,
Mr D
@mrd77 are the tc-values really wrong or do you just dont need them? Cant you disable or manually reset tc in any app?
timecodes are correct. However, in After Effects, it makes all clips start on frame 170000 or more. It's really high up there and hard to deal with when matching things up between VFX programs, I find. I just found out how to make TC back to zero in Resolve, so I guess I'm good now! 8)
Good To hear.
I have problems to run the 1.50 beta8 and older versions under a virtual machine (Windows7) tested on Parallels and VMvare ESXI two different machines and same problem. I can add a file and press convert but I don't se any progress bar or anything else.
Doesn't the software work under VM?
cant tell you, never tried that. you need .net4.0 to use it.
I got .net 4.5. Windows 7 all uppdates on Winodws.
Machine #1
Mac pro mid 2010, 16RAM, nvidia quadra 4000. 10.9.4
Parallells Desktop 9 (updated)
Windows 7 (updated)
Machine #2
Asus P9D-I
Intel Xeon E3-1265Lv3 2,5GHz
16GB ECC RAM
ESXi 5.5 + Windows 7
The software starts correctly on boths machines but when I press convert nothings more happens. Waited >1h for a 250MB MLV-file.
Posted the debuginfo. Did you add some Anti-Virtual Machine and Anti-Sandbox in the software?
Do you need any special system requirements, cpu speed, cores, ram, hdd, GPU?
------------- 1.5.0 BETA8 started at 14.08.25 21:12:50 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108.MLV
[drop] ------
[drop] -- file C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108.MLV.
[drop] -- file C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
***** [getMLVAttributes] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
*****
***** [getMLVAttributes] metaData Object:
xResolution=1920
yResolution=804
frames=246
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrix=System.Byte[]
lostFrames=0
fpsNom=25000
fpsDen=1000
dropFrame=False
fpsString=25,00
stripByteCount=2701440
stripByteCountReal=2701440
modell=Canon EOS 5D Mark III
camId=MAGIC737d375
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
jpgConvertR=0,540084388185654
jpgConvertG=1
jpgConvertB=0,547008547008547
whiteBalance=4500
whiteBalanceMode=0
whiteBalanceGM=4400
whiteBalanceBA=4400
blackLevelOld=2046
blackLevelNew=2046
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5,05913231434306
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.5.0 BETA8
versionString=System.Byte[]
propertiesString=
*****
***** [getMLVAttributes] fileData Object:
convertIt=False
fileName=C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108.MLV
fileNameOnly=M20-2108
fileNameShort=M202108F
fileNameNum=202108
tempPath=C:\Users\jonte\AppData\Local\Temp\
sourcePath=C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=2014-08-25 21:12:40
modificationTime=2014-08-20 21:07:56
VIDFBlock=
RAWBlock=
*****
***** [getMLVAttributes] lensData Object:
lens=EF50mm f/1.4 USM:
focalLength=50
aperture=160
isoValue=1600
shutter=1/50s
*****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 648
[drop] hasAudio. AUDF-List created. Blocks: 50
[drop] VIDF-List created. Blocks: 244
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[readVIDFBlockData] done: [100][101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124]
[readVIDFBlockData] done: [125][126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143][144][145][146][147][148][149]
[readVIDFBlockData] done: [150][151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174]
[readVIDFBlockData] done: [175][176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199]
[readVIDFBlockData] done: [200][201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224]
[drop] VIDF-Blockdata read and created.
[drop] adding M20-2108 to batchList
[drop] ** Item M20-2108 imported | MLV
[drop] * res 1920x804px | 25,00fps | 244 frames | BL2046 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 244 | has audio
[doWork] started
[doWork] all in all there are 244 frames to convert
[doWork] 1 Cores used
[doWork] -> converting item M20-2108
[doWork] settings format 0 with BL2046 WL15000 with+out maximizingvalue 5,05905511811024
[doWork] prefix - filename generator [F]
[doWork] destinationPath -> M20-2108
[doWork] outputFilename -> M20-2108
[doWork] Directory C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork][for] read/convert MLV VIDF: [0]
[doWork][for] read/convert MLV VIDF: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]
[doWork][for] read/convert MLV VIDF: [26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50]
[doWork][for] read/convert MLV VIDF: [51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75]
[doWork][for] read/convert MLV VIDF: [76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99][100]
[doWork][for] read/convert MLV VIDF: [101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124][125]
[doWork][for] read/convert MLV VIDF: [126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143][144][145][146][147][148][149][150]
[doWork][for] read/convert MLV VIDF: [151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174][175]
[doWork][for] read/convert MLV VIDF: [176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199][200]
[doWork][for] read/convert MLV VIDF: [201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224][225]
it starts and converts. so the .net-framework is working properly. the debug shows an output into folder
C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108but it does not output audio. there are some strange things i have to check in my source. please try with a sample file from here (http://www.magiclantern.fm/forum/index.php?topic=11899.0).
(if the progressbars dont move, seems like a problem with (wpf/silverlight inside) the vm's)
converting 250mb is a no-brainer, should be done in 4-5 seconds.
Quote..Did you add some Anti-Virtual Machine and Anti-Sandbox in the software?..
why should i? its a free software without special ambitions (indirect adware only working on native win-systems? :D now i like the idea, if intel or microsoft would pay me "a lot" for that)
regards chmee
Hey Chmee, Awesome, got the mpg's to work!! only thing is, it takes for ever to import them all into premiere because i have to go into each folder individually which wont work if im shooting a proper gig with 300 clips etc.
Secondly, once i export the xml out of premiere, i cant get the mpgs to link with the dngs in resolve. Any answers to these?
otherwise, love your buddy.
by now i do not work with proxies, so i cant tell you a proper way to work with them in terms of edl/xml. (edit: please tell, how you fixed the ffmpeg-thing, maybe someone will be thankful for this info)
But: Importing: Open an File-Explorer, do a search on *.mpg and dragdrop the files into premiere-bin. should be the fastest way.
Quote from: chmee on August 26, 2014, 07:47:47 AM
it starts and converts. so the .net-framework is working properly. the debug shows an output into folder
C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\M20-2108
but it does not output audio. there are some strange things i have to check in my source. please try with a sample file from here (http://www.magiclantern.fm/forum/index.php?topic=11899.0).
(if the progressbars dont move, seems like a problem with the vm's)
converting 250mb is a no-brainer, should be done in 4-5 seconds.
why should i? its a free software without special ambitions (indirect adware only working on native win-systems? :D)
regards chmee
Tyyed with all four MLV file in second post and same problem, nothing happens when i press convert. Tryed Two diffrent machines parallells and and Vmware ESXi.
Same problem and no problem with a real Windows 7 machine.
------------- 1.5.0 BETA8 started at 14.08.26 11:17:11 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON.MLV
[drop] ------
[drop] -- file C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON.MLV.
[drop] -- file C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
[getMLVAttrib] [!] NO WAVI Block
***** [getMLVAttributes] metaData Object:
xResolution=1600
yResolution=900
frames=15
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrix=System.Byte[]
lostFrames=0
fpsNom=23976
fpsDen=1000
dropFrame=False
fpsString=23,98
stripByteCount=2520000
stripByteCountReal=2520000
modell=Canon EOS 5D Mark III
camId=MAGIC6e4781b
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
jpgConvertR=0,540084388185654
jpgConvertG=1
jpgConvertB=0,547008547008547
whiteBalance=4500
whiteBalanceMode=0
whiteBalanceGM=3200
whiteBalanceBA=3200
blackLevelOld=2048
blackLevelNew=2048
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5,05991352686844
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.5.0 BETA8
versionString=System.Byte[]
propertiesString=
*****
***** [getMLVAttributes] fileData Object:
convertIt=False
fileName=C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON.MLV
fileNameOnly=5DIII_113_14apr13_blfixON
fileNameShort=5DIII113
fileNameNum=51131413
tempPath=C:\Users\jonte\AppData\Local\Temp\
sourcePath=C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=2014-08-26 11:02:18
modificationTime=2014-08-26 11:03:25
VIDFBlock=
RAWBlock=
*****
***** [getMLVAttributes] lensData Object:
lens=120-300mmI:
focalLength=235
aperture=280
isoValue=100
shutter=1/33s
*****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 47
[drop] VIDF-List created. Blocks: 15
[readVIDFBlockData] started
[drop] VIDF-Blockdata read and created.
[drop] adding 5DIII_113_14apr13_blfixON to batchList
[drop] ** Item 5DIII_113_14apr13_blfixON imported | MLV
[drop] * res 1600x900px | 23,98fps | 15 frames | BL2048 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 15 | has no audio
[doWork] started
[doWork] all in all there are 15 frames to convert
[doWork] 1 Cores used
[doWork] -> converting item 5DIII_113_14apr13_blfixON
[doWork] settings format 0 with BL2048 WL15000 with+out maximizingvalue 5,05983631871526
[doWork] prefix - filename generator [F]
[doWork] destinationPath -> 5DIII_113_14apr13_blfixON
[doWork] outputFilename -> 5DIII_113_14apr13_blfixON
[doWork] Directory C:\Users\jonte\Downloads\raw2cdng.1.5.0.beta8\5DIII_113_14apr13_blfixON created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork][for] read/convert MLV VIDF: [0]
@jonas18z
sorry, cant help you and i dont have the time to test under vm's as well. if someone got some hints, fine, will try to solve it.
Understand! Anyone else got the same problem under VM?
Compiled the software and it seems that it stucks on:
Line 572 in MainWindows.xaml.cs
cde.Wait();
Removed that line.
// cde.Wait();
And now the software produce all frames .DNG and .WAV files even if I add more then one file. But the progress bar stucks on 95% or something like that but no problems to add more file after that :)
are you able to give your vm more than one cpu? then try again with cde.wait(). its waiting, 'til all workerthreads ended..
regards chmee
Gave the VM 2 cpu and now it works prefectly!
so 1 CPU wil fail >1 CPU works....
thankful for exploring this behaviour.
thank you. regards chmee
edit: thanks to @beauchampy, sent me some q7-samples, DNG-Data (from dng_validate)
Validating "q7_2k.dng"...
Uses little-endian byte order
Magic number = 42
IFD 0: Offset = 8, Entries = 42
NewSubFileType: Main Image
ImageWidth: 2064
ImageLength: 1088
BitsPerSample: 12
Compression: Uncompressed
PhotometricInterpretation: CFA
FillOrder: 1
Make: "ODYSSEY 7Q RECORDER"
StripOffsets: Offset = 4096
Orientation: 1 - 0th row is top, 0th column is left
SamplesPerPixel: 1
RowsPerStrip: 1088
StripByteCounts: Count = 3368448
PlanarConfiguration: 1
Software: "2.10.141"
CFARepeatPatternDim: Rows = 2, Cols = 2
CFAPattern:
Red Green
Green Blue
DateTimeOriginal: 0000:00:00 00:00:00
DNGVersion: 1.1.0.0
DNGBackwardVersion: 1.0.0.0
UniqueCameraModel: "FS700"
BlackLevelRepeatDim: Rows = 1, Cols = 1
BlackLevel: 256.00
WhiteLevel: 4095
DefaultScale: H = 1.0000 V = 1.0000
DefaultCropOrigin: H = 8.00 V = 4.00
DefaultCropSize: H = 2048.00 V = 1080.00
ColorMatrix1:
0.6290 -0.1537 -0.0334
-0.3453 1.1168 0.2371
-0.1034 0.1852 0.5259
CameraCalibration1:
1.0000 0.0000 0.0000
0.0000 1.0000 0.0000
0.0000 0.0000 1.0000
AsShotNeutral: 0.4186 1.0000 0.5604
BaselineExposure: +0.00
BayerGreenSplit: 0
CameraSerialNumber: "SERIAL"
CalibrationIlluminant1: 5400K
BestQualityScale: 1.0000
ActiveArea: T = 0 L = 0 B = 1088 R = 2064
ProfileEmbedPolicy: Allow copying
ProfileCopyright: "Sony FS700"
*Tag51043: Byte, size = 8
22 05 20 06 00 00 00 00 ". .....
*Tag51044: SRational = 50/1
*Tag51081: "Reel:R001 Sc:SCENE1 "
*Tag51105: "Cam:A Day:0 1"
NextIFD = 0
Validation complete
it seems, its 12bit linear without a linearizationtable
The way I fixed ffmpg was purely by removing mlviewer.. Not sure if that is a legit fix....but I thought it may be messing with the code somewhere... so i gave it a go and it worked.
I dont know if this is a known bug but the app (1.5.0 beta 6, tried beta 7 and it seems that it has the same problem) crashes very often when i am trying to import more than 10-15 files. Just saying something like "program stop working". Because of that i need to put files in small portions, its annoying because leaving app working overnight with dozens of file is impossible. I have also noticed than it crashes every time im trying to import 0,5 fps clips (made with fps override). I am on windows 7 64bit. Is there any log file or something i could post so you could see what is going on?
Thanks for info. There is a debug-button on top. Click, close and open once again.
OK, here is log, tried to put more than 20-25 files and crashed, after that i tried to put mixed fps files 23,976 and 0,5 fps - crashed when importing 0,5fps files and in the third attempt i put only 0,5fps files and crashed immidiately.
https://www.dropbox.com/s/h4xdl1nmprlrrcc/problems%20with%20raw2cdng.2.debug.log?dl=0
Please send me a fps-overridden file, that was shortened with mlvdiag (inside the beta6 or 7) Ah.. Open the taskmanager and look on the memory usage while you import this amount oft files. Does it reach 1.5gb? Could be a hint for a memory leak..
Here is fps overriden mlv file (0,5 fps). I tried and it crashed the app so it should be good for checking.
https://www.dropbox.com/s/kf06lba745aq80d/M07-0857.MLV?dl=0
short before v1.6.0.. i need some videos having verticalbanding. the best would be to provide it to the whole coder-comunity, we could test our algorithms with it. put it into the samplefiles-thread (http://www.magiclantern.fm/forum/index.php?topic=11899.0). thanks a lot.
@kgv5
yeah found the bug, i truncated framerates for the timecode. therefore it was a "division by zero". fixed. framerates below 1 will be now 1. shouldnt be a problem for any workflow. (v1.6.0)
@zacshome
strange, but if it helped. ok.
raw2cdng 1.6.0 online
* bugfixes
* GUI slightly changed
* with thumbnail and histogram
* mlvplayer
* rec709 on proxies (jpg, mpeg2 and h264 with ffmpeg)
(http://dl.phreekz.de/raw2cdng/raw2cdng.1.6.0.jpg)
download
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
(i need files with verticalbanding!!!) regards chmee
I have a 5D3 that has vertical banding. Any particalur things you want me to shoot for you? Lens cap on, blown out, underexposed, time limit? Just let me know and I could send you some.
@chmee
wow, i have just imported 46 mlv files at once and seems that my previous problem with importing many files is gone :) I need to make some further testing though but looks good now. Love the new UI, those previews are great and work so nicely:) Do you think it is possible to implement skipping through preview (with mouse clicking) without a need to watch all thing from the beginning to the end? Thank you for your work :)
@ChadMuffin
nothing especial, my 5DIII does a good job and the diferences between the channels are quite small. to check if my code is working i need some files i can compare with a1ex's code results. send it to
[email protected]. thanks a lot. please take footage that i can put into the samplefiles-thread, so other coders can participate without injuring your privacy.
@kgv5
cheers. idea is good, realisable
@chmee - Sent you a Google Drive link.
Why do I get error with audio when I try to replace a proxy .mov file with the cDNG files in Premiere pro cc 2014.0.1 osx.
(http://112.imagebam.com/download/6gtr4QnmQj7_blm3MrTo4A/35209/352080622/audioproblem.png) (http://www.imagebam.com/image/15a066352080622)
@chadmuffin
already downloaded, thanks. will give an info next days.
@tob
seems, like proxy handling doesnt like mediafile (video with embedded audio) versus splitted video and audio-files. will do a try with an audioless proxyfile. this ones should take automatically the wav-file.
thanks @chadmuffin. vertical Banding will be in 1.6.1 (next days) i assume its not that perfect as a1ex algorithm, but it works :)
(http://dl.phreekz.de/raw2cdng/raw2cdng_verticalbanding.jpg)
regards chmee
Glad to help and appreciate all your hard work! Looking forward to the next release!
@chmee,
Here's a sample with vertical banding, that couldn't be fixed with my algorithm: http://www.magiclantern.fm/forum/index.php?topic=13291
Maybe your algorithm has better luck here?
I love RAW2CDNG! Always using it with my MLV and RAW files. The audio syncing in Davinci Resolve from the MLV files with your app couldn't be better, thanks for that!
/ Karl Allemyr
http://www.allemyr.com/
raw2cdng 1.6.1 is online
* vertical banding (really simple algorithm, but seems to work)
* WPF re-coding by rd4eva - was necessary - thus happend some changes to the ui, will be re-optimized next version
Download raw2cdng 1.6.1 (http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/)
regards chmee
Hi!
Thank you for a really great app. If its possible to unsqueeze 60 FPS 1920x672 to 1920x1080 inside RAW2CDNG? Would be helpfull. Having a a kind of problem inside Davinci Resolve right now, mixing 23,976fps and 60fps.
Thanks in advance!
@allemyr
this problem is quite special and it was my intention to leave the raw-data as original as possible. aproportional rescaling means i'd recalculate the raw-data, therefore the quality would change depending on algorithm. no, too hot, sorry. dng-specification has no "pixel aspect ratio" tag - this would be a solution for me.
but: if read correctly, there's a "edit PAR value" property in Resolve.
resolve 11 - manual (http://software.blackmagicdesign.com/DaVinciResolve/docs/DaVinci_Resolve_11_Manual_June_2014.pdf) page 644
Quote..Input Sizing Adjustments.. additional MODIFY PAR button.. change the pixel aspect ratio of a clip
regards chmee
Looks like we need a Mac version since the-app-we-shall-not-speak-of is no longer supported on this forum.
:D
The vertical banding fix seems to be working great for me. However, I tried 12 bit max, 16 bit and 16 bit max with no luck of importing them into Premiere Pro CC. It would state the files are damaged or corrupt. Possible user error? When I gave it files in one folder it worked fine but, I had files in 5 different folders and it placed them in the batchlist, it loaded them fine but when I hit convert, it crashed. I love the new GUI and the simplicity. You're doing what a company hires a team for! Great work!
Update: There were about 50 MLVs between the 5 folders. And, I had vertical banding fix checked with the coversion before attempting to use them in Premiere Pro CC.
thanks for the info, will investigate next days. (i did tests with 3 folders and about 25 mlv's and raw's, and it worked, so, send me or copy/paste (http://pastebin.com/) a debuglog)
regards chmee
Hi, Chmee. Apologies if this been answered somewhere else, I couldn't seem to find the answer to these:
Does raw2cdng extract wav out of MLV with sound? I'm not getting wav on the output for some reason.
and
Is it possible to put all the .dng output in one folder instead of separate folder for each clip?
Thanks! Great work on that vertical banding! It fixed a lot of the banding I can't seem to get rid of with the other stuff.
(a) yes, wav should be extracted as well. does raw2cdng show ( with audio) behind the filename? look here (http://dl.phreekz.de/raw2cdng/raw2cdng.1.6.0.jpg)
(b) hmm. dont see any benefit beside the fact, it fastens the importing into premiere (but leaves a chaotic harddisk) :) you could manage that with a search *.dng in windows and copy them into one folder.
Quote from: chmee on October 06, 2014, 07:51:26 AM
(a) yes, wav should be extracted as well. does raw2cdng show ( with audio) behind the filename? look here (http://dl.phreekz.de/raw2cdng/raw2cdng.1.6.0.jpg)
Ah! It doesn't say! No wonder. Thanks!
Quote from: chmee on October 06, 2014, 07:51:26 AM
(b) hmm. dont see any benefit beside the fact, it fastens the importing into premiere (but leaves a chaotic harddisk) :) you could manage that with a search *.dng in windows and copy them into one folder.
Yeah, that's what I've been doing. Searching *.dng and moving them into one folder to speed up importing to premiere. Sure, it makes it one big lump but premiere sorts them out just fine. Would've been great if that can be done at the output level though. :D It takes a while for my machine to find all the *.dng and move them out.
Thanks again! Great app!
If you dont copy but move, it will work much faster.
Yeah, I've been moving them instead of copying, but still. Finding then moving tens of thousands of small files takes quite a lot of time.
By outputting to a single folder for all the clips instead of each folders for all the clips will save a lot of time. Especially since premiere will automatically arrange them all into each clips with proper framerates. (I use the media browser to import clips)
Hi Chmee !
I love your tool and each version has a lot of improvement. I just download the last version but i can't resolve one thing : i'm trying to make video proxies with sound from MLVs.
Using raw2cdng, i had a folder with DNG and WAV. Worked great.
Reading the forum, i see that i can also do video proxies if ffmpeg is present in the same folder. I just did it and mpg2 and mpeg04 did appear in the menu.
But it seems not to work. When i'm extracting the files, i still get the DNG+WAV but no video proxy at all. If i select jpg proxies, it works well.
Do i miss something ? And is it possible to make video proxies this way ?
Thanks !
the video-proxies should work :) please turn on the debug (upper right button) and do a test. inside the debug.log should stay the output of the ffmpeg.exe process. right in the end.
regards chmee
btw : 16bit tif (demosaiced), therefore prores and other codecs on its way, still coding and deciding what demosaicing-algorithm i should take. vng4? dcb? ahd? (atmoment on vng4). @a1ex maybe you a statement?
chmee, dose the mlvdiag.1.0.1 take a while to analyze files ? Its kept running with not results.
I stop it after 5min.
I have a problem with mlv+audio file+7 spanning , in total 30GB from 5D2
I tried the new raw2cdng(v1.6.1) could not read it & MlRawViewer 1.2.3 would preview the first 24% with metadata
but could not export anything , MLVBrowseSharp did export 14 out of 30 GB with audio to Raw legacy with preview turn off
before the program crashed. I can preview that Raw legacy with audio file in raw2cdng (v1.6.1) & MlRawViewer 1.2.3.
So this is what happen to cause this problem. :-[
On a shot, the door of the cam on the CF card was open by mistake after the
cam stop recording but before the cam was turn off the card was not removed thou.
So I turned the cam of remove the card , dump card to laptop and kept on filming with out any more issue .
I also tried mlv_dump by command line no go , so all this leads me to believe that after the 2nd spanning file it
loose its path.
So I was hoping that mlvdiag could point to a problem I could repair. So far I saved 4700 frames out of 10,000 frames
about 3min out of 8min . Any advice is appreciated thanks :D
it seems, the problem is more on the following split-files. maybe a debugfile from raw2cdng can help on solving.. (pastebin please)
OK, thanks
I think see the problem
"NO RAWI Block"
http://pastebin.com/embed_iframe.php?i=6B1zPAGs
No sure how to embed in post
Quote from: reddeercity on October 13, 2014, 01:02:02 AM
No sure how to embed in post
For text you can use the code tags.
@reddeercity:
use mlvdiag 1.01 and drag/drop your file straight on the 55mb rectangle on bottom. make it available for us as a download. cause its really strange it does not find a RAWI-Block - tell us the firmware-version, build-date and changeset, if possible.
@ chmee
Here you go , dropbox link
http://www.dropbox.com/s/cayxrw4m61ncfus/M30-1952.MLV.55mb.mlv?dl=0
I copied the info from the "
autoexec.bin" file
Magic Lantern v2.3.NEXT.2014Feb16.5D2212
Mercurial changeset: 536fbadfcc31 (unified) tip
Built on 2014-02-15 23:11:03 by
[email protected]Camera : 5D2
Firmware: 212-e -e
Is this the info you want ?
Can I repair the spanning files , or can I extract the spanning files individually some how ?
No suggestions then on how to fix the missing "RAWI-Block" or how to export the spanning file to cdng's ?
sorry, hadnt had the time, will look into it today.
No help ??? I Guess on my own to try to fix this.
Very sad state ML is in when you can't get a little info!
But I guess every one is more concern with
The OTHER camera >:(
Look like ML is in for a slow death . As no one cares anymore
Here!
@reddeercity
sorry, it was weekend, there's more than internet. i could import your file into mlvdiag and it shows a rawi-tag. i was able to use it via raw2cdng as well. so. this shortened version works. you said, its a splitted file, but in your debug.log theres counted only one file. so, yes without doubt i believe you theres a problem, but i cant get it. you said, another converters are having problems also?!
Sorry from before just frustrated.
Quote from: chmee on October 19, 2014, 10:48:45 PM
@reddeercity
you said, another converters are having problems also?!
Yes , I can extract about 4700 frame to raw legacy with MLVBrowseSharp out of 10,000 frames then crashes (just stops responding & with no preview enabled )
MlRawViewer 1.2.3. will load the spanning 30 GB file but with a still image and stops responding at 25% on the progress bar.
Is there a way for checking to see which spanning file is missing the "RAWI Block"
And can a RAWI Block be copy from a different spanning file with change or modified data to work?
I think its at the 2-3 spit file but not sure.
Spanned files don't have to have a rawi block. There just needs to be at least one somewhere.
Strange :o
I Can extract with old version of mlv2dng from August, 28th 2013, with header error audio & missing 804 or 10,320 frames
as After Effect reports , doing a quick render to see what happens.
but at least I have 90% of it in dng's
Quote from: dmilligan on October 20, 2014, 02:46:45 AM
Spanned files don't have to have a rawi block. There just needs to be at least one somewhere.
So would that be a damage header , also is that where the spanning info is ?
The MLV file format is a "block" based format. There isn't really just a header, there are lots of blocks each with their own headers. There isn't really spanning information anywhere, spanned files have to be auto-detected by the converter. You should try mlv_dump with the -v option (this will list all the blocks).
Ok I'll give that a try
I am having issues importing the cDNGs from my 5D3 into Premiere Pro CC.
(http://i.imgur.com/ORZ22ma.jpg)
Here is some info:
raw2cdng 1.6.1 - 12bit max, highlight fix, vertical banding
5D3 1.1.3 ML Oct 7, 14
50mm 1.4
KB 64GB 1000x (exfat)
MLV module
1920x818
Files > 4GB: ON
Buffer fill method: 4
CF-only buffers: 1
Reserve card space: ON
(if not listed, off or default)
Premiere Pro CC 2014.1
Windows 8.1 64bit
i7 990x
16GB RAM
GTX 780
.NET Framework 4.5.1
Debug:
------------- 1.6.1 started at 14.10.20 15:04:02 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file C:\Users\Chad\Desktop\Test cDNG\M20-1456.MLV.
[drop] ------
[drop] -- file C:\Users\Chad\Desktop\Test cDNG\M20-1456.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in C:\Users\Chad\Desktop\Test cDNG\M20-1456.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
***** [getMLVAttributes] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
*****
***** [getMLVAttributes] metaData Object:
xResolution=1920
yResolution=818
frames=117
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrix=System.Byte[]
lostFrames=0
fpsNom=23976
fpsDen=1000
duration=00:04:01
dropFrame=False
fpsString=23.98
stripByteCount=2748480
stripByteCountReal=2748480
modell=Canon EOS 5D Mark III
camId=MAGIC9bc9385
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0.629766297662977
wb_G=1
wb_B=0.457960644007156
whiteBalance=3600
whiteBalanceMode=9
whiteBalanceGM=3600
whiteBalanceBA=3600
blackLevelOld=2047
blackLevelNew=2047
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5.05952289045009
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.1
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
*****
***** [getMLVAttributes] fileData Object:
fileName=C:\Users\Chad\Desktop\Test cDNG\M20-1456.MLV
fileNameOnly=M20-1456
fileNameShort=M201456K
fileNameNum=201456
tempPath=C:\Users\Chad\AppData\Local\Temp\
sourcePath=C:\Users\Chad\Desktop\Test cDNG
parentSourcePath=Test cDNG
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=10/20/2014 3:03:58 PM
modificationTime=10/20/2014 2:56:46 PM
VIDFBlock=
RAWBlock=
*****
***** [getMLVAttributes] lensData Object:
lens=EF50mm f/1.4 USM:
focalLength=50
aperture=220
isoValue=200
shutter=1/64s
*****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 394
[drop] hasAudio. AUDF-List created. Blocks: 26
[drop] VIDF-List created. Blocks: 115
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[drop] VIDF-Blockdata read and created.
[drop] adding M20-1456 to batchList
[showPicture] started
[drop] verticalBanding enabled. decision: do it
[drop] verticalBanding CoEffs: 1 0.996742462104726 1.00024283298405 1.00935884271917 0.998710116109811 0.997916433956018 0.99985583655609 0.985100399264895 1
[drop] ** Item M20-1456 imported | MLV
[drop] * res 1920x818px | 23.98fps | 115 frames | BL2047 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 115 | has audio
[batchList_Click] showPicture
[showPicture] started
[batchList_Click] read Data from M20-1456 frame 0
[batchList_Click] * M20-1456
[previewTimer_Tick] show previewframe 1 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 2 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 3 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 4 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 5 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 6 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 7 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 8 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 9 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 10 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 11 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 12 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 13 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 14 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 15 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 16 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 17 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 18 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 19 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 20 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 21 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 22 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 23 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 24 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 25 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 26 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 27 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 28 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 29 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 30 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 31 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 32 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 33 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 34 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 35 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 36 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 37 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 38 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 39 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 40 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 41 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 42 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 43 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 44 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 45 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 46 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 47 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 48 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 49 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 50 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 51 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 52 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 53 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 54 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 55 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 56 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 57 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 58 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 59 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 60 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 61 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 62 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 63 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 64 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 65 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 66 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 67 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 68 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 69 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 70 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 71 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 72 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 73 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 74 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 75 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 76 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 77 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 78 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 79 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 80 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 81 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 82 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 83 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 84 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 85 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 86 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 87 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 88 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 89 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 90 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 91 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 92 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 93 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 94 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 95 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 96 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 97 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 98 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 99 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 100 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 101 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 102 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 103 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 104 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 105 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 106 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 107 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 108 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 109 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 110 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 111 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 112 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 113 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 114 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 0 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 0 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 1 from M20-1456
[showPicture] started
[previewTimer_Tick] show previewframe 2 from M20-1456
[showPicture] started
[batchList_Click] showPicture
[showPicture] started
[batchList_Click] read Data from M20-1456 frame 2
[batchList_Click] * M20-1456
[convert_Click] all in all there are 115 frames to convert
[doWork] started
[doWork] 12 Cores used
[doWork] -> converting item M20-1456
[doWork] settings format 4 with BL0 WL4095 with+ maximizingvalue 5
[doWork] prefix - filename generator Test12m_[D]_[C]
[doWork] destinationPath -> Test12m_141020_201456
[doWork] outputFilename -> Test12m_141020_201456
[doWork] Directory C:\Users\Chad\Desktop\Test cDNG\Test12m_141020_201456 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork][for] read/convert MLV VIDF: [0]
[doWork][for] read/convert MLV VIDF: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]
[doWork][for] read/convert MLV VIDF: [26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50]
[doWork][for] read/convert MLV VIDF: [51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75]
[doWork][for] read/convert MLV VIDF: [76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99][100]
[doWork] convert Done
[doWork] hasAudio file -> Test12m_141020_201456.wav
[saveAudio] started
***** [saveAudio] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
*****
[saveAudio] bextName: Canon EOS 5D Mark III
[saveAudio] UDI: DECanMAGIC9BC9385145646302567450
[saveAudio] datetime: 2014:10:2014:56:46
[saveAudio] timeRefMultiplier: 1.001001001001
[saveAudio] time2frame: 54292.2922922923
[saveAudio] timeRef (samplingRate*time2frame): 2606030030
[saveAudio] fpsString: 23976/1000
[saveAudio] DF: NDF
[saveAudio] subChunkSize :998400
[saveAudio] success
MLV Download:
https://drive.google.com/file/d/0Bwd3r5HwGFltOUZHT0RJenpvUDg/view?usp=sharing
@chadmuffin
did a test. Test12m_[D]_[C] leads to very long sequencingnumbers (you: Test12m_141020_20145600001)
try with an underline in parenthesis on end. Test12m_[D]_[C](_)
edit: (ah sorry, i tested first with 16bit maximized/highlight fix). but yes, with 12bit it works as well.
[F]_[D]_001_C0000(_) (for example)
@chee - That did the trick! Working great now.
I think I found the problem, Open all the spanning files with a hex editor and found that .M02
has "0" data from Offset (6C98CD00) to (FF6EA030)
***first part is the beginning of the file***
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
00000000 4D 4C 56 49 34 00 00 00 76 32 2E 30 00 00 00 00 MLVI4...v2.0....
00000010 8C 9B 9F 10 1A 70 2F AC 03 00 01 00 01 00 00 00 Œ›Ÿ..p/¬........
00000020 01 00 01 00 73 05 00 00 8E 04 00 00 A8 5D 00 00 ....s...Ž...¨]..
00000030 E8 03 00 00 56 49 44 46 E0 D9 2E 00 EF 61 62 0A è...VIDFàÙ..ïab.
00000040 00 00 00 00 50 10 00 00 A0 00 B0 00 A0 00 AF 00 ....P... .°. .¯.
00000050 E0 0F 00 00 E0 D9 2E 00 52 54 43 49 2C 00 00 00 à...àÙ..RTCI,...
00000060 2E 33 FC 09 00 00 00 00 39 00 36 00 13 00 1E 00 .3ü.....9.6.....
00000070 08 00 72 00 02 00 00 00 00 00 00 00 00 00 00 00 ..r.............
00000080 00 00 00 00 56 49 44 46 90 D9 2E 00 77 60 F9 09 ....VIDF.Ù..w`ù.
00000090 00 00 00 00 AB 0F 00 00 A0 00 B0 00 A0 00 AF 00 ....«... .°. .¯.
000000A0 90 0F 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000000B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000000C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000000D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
000000E0 57 42 41 4C 2C 00 00 00 A2 6E 94 03 00 00 00 00 WBAL,...¢n".....
000000F0 09 00 00 00 E4 0C 00 00 CD 01 00 00 00 04 00 00 ....ä...Í.......
00000100 4E 02 00 00 00 00 00 00 00 00 00 00 56 49 44 46 N...........VIDF
00000110 08 D9 2E 00 89 C6 8E 03 00 00 00 00 96 05 00 00 .Ù..‰ÆŽ.....–...
00000120 A0 00 B0 00 A0 00 AF 00 08 0F 00 00 10 00 08 80 .°. .¯........€
00000130 FF F7 E7 F5 40 58 62 00 FF FB FF FF 00 00 00 40 ÿ÷çõ@Xb.ÿûÿÿ...@
00000140 7B FD FF FE 00 00 00 10 FF FF FF FF 00 00 00 20 {ýÿþ....ÿÿÿÿ...
00000150 FF FF FF BF 00 40 00 42 DF FF FF 7F 00 02 30 00 ÿÿÿ¿[email protected]ßÿÿ...0.
00000160 7F DF 7F FF 00 00 20 00 FF EF FF FF 01 20 41 81 .ß.ÿ.. .ÿïÿÿ. A.
00000170 FF EF FF FF 06 12 00 08 DF DF FF FF 80 02 00 02 ÿïÿÿ....ßßÿÿ€...
00000180 7F FF FF FE 00 80 00 04 FF F5 FF F7 00 01 04 00 .ÿÿþ.€..ÿõÿ÷....
00000190 B7 F7 FF FF 80 10 00 00 FF FF 7E FF 00 00 00 00 ·÷ÿÿ€...ÿÿ~ÿ....
000001A0 BF 5A FF FE 00 20 00 00 FF FF FF EF 00 01 00 00 ¿Zÿþ. ..ÿÿÿï....
000001B0 ED FF FD FF C2 08 50 12 FE FF FF FE 02 48 02 10 íÿýÿÂ.P.þÿÿþ.H..
000001C0 FF FF FE FF 01 00 11 80 FF FF BF E7 00 84 40 80 ÿÿþÿ...€ÿÿ¿ç.,,@€
000001D0 F7 E5 AF F5 10 00 00 0A FF FF 7F FF 00 10 00 12 ÷å¯õ....ÿÿ.ÿ....
***this part of the file no data***
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
6C98CD00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CD10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CD20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CD30 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CD40 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CD50 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CD60 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CD70 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CD80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CD90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CDA0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CDB0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CDC0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CDD0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CDE0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CDF0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
6C98CE00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
***end of the file***
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
FF6E9F90 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
FF6E9FA0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
FF6E9FB0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
FF6E9FC0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
FF6E9FD0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
FF6E9FE0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
FF6E9FF0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
FF6EA000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
FF6EA010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
FF6EA020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
FF6EA030 00 00 00 00 ....
Look to me as to be no info there . So the only way I can extract at this point is with mlv2dng with 800 frames missing of 10,320.
In which I have done but can't get the audio thou.
Better then nothing ::)
@reddeercity
because mlv is a block-based format it should work, you delete empty stuff and partly destroyed blocks. then merge part a and b. do not forget to make a backup..
(http://s14.directupload.net/images/141021/4ye53udn.jpg) (http://www.directupload.net)
and yes, thats a good hint for mlvdiag, to cope with those errors.
short info: doin and testing chroma smooth - still not 100% satisfied - but on the way.. (1.6.2)
(http://dl.phreekz.de/raw2cdng/raw2cdng1.6.2_chromasmooth.jpg)
wow!thats gonna b great! i hv stopped shooting raw after buying 650D as it has so many problems like pink dots n audio which i cant turn off agc. after releasing raw2cdn version 1.6.2, i sould definitely start shooting raw again. raw2cdng is the best converter out there. many people dont know the right converter so they r having problems recording raw but for me recording n editing raw is as easy as recording n editing h264
Thanks Chmee for implementing the chromasoothing!
What's peoples' opinions on the best workflow using CDNG? Do you run it through Davinci first or drop it straight into PPro CC?
With the latter, my prem pro is struggling to play the files smoothly, it plays about 1-2 seconds before going jumpy/stopping. It's the same with the former method when compressed to MPEG Quicktime and dropped into PPro.
What settings do people use for sequence Preview File Format, if that makes a difference? thanks
16gb ram, i7-4770 @ 3.5ghz, gtx 660, gpu acceleration
what kind of harddisk(s) do you've got? my opinion: i tried to get a straight workflow inside adobe world. so, yes, its possible without DaVinci. i've got for that 2 ssd's (crucial mx100 512gb), they do at least 120mb/s on very small files and go up til 500mb/s (https://twitter.com/phreekz/status/525596432372625408).
Premiere's running off a SSD but the footage is stored on a sata 7200rpm drive, do you think that could be it then?
Although when I play the original MLV file using an mlv player, it plays around 18-19fps, much better than the jumpy premiere pro
thanks
test it yourself and copy some dng's-folders to the ssd. does it change? dng-sequences are not small, as you know. yes, i assume, the sata-hdd limits..
Didn't make a difference, however I tried converting to 12bit instead of 16bit and now it runs smoothly. Am I losing any data by doing this? Sorry if this has been answered before, did have a quick search.
thanks for a great tool by the way!
mathematically yes, practically no. the advantages over 8bit are still huge. (and 12bit means as well, lowering the datarate by ~25% compared to 16bit)
short sign of life.
(*) i embedded the chroma smoothing, as i said. with the first sample i had some problems, because i couldnt eliminate all false-pixels. then i understood, the chroma smoothing is only able to fix red and blue pixels. the green ones blur a little and rise. i'd recommend to use the badpixel-elimination inside the canon body (http://www.bluehoursite.com/articles/how-get-rid-stuckdead-pixels-canon-dslr-camera) before working with ml. the version with cs will be here next days.
(http://dl.phreekz.de/raw2cdng/M07-2201_CS_analyse.jpg)
M07-2201 from samplefiles (ML-RAW June 2013--EOS 550D @glubber) (http://www.magiclantern.fm/forum/index.php?topic=11899.msg130484#msg130484) - a good point to start with a green hot/stucked pixel algorithm.
(*) in general i'll code a demosaicing-algorithm to be able transcoding into tif, then prores/dnxhd or other codecs (maybe i can realize the arriraw-output)
BUT
(*) my thoughts are, its time to end this tool. it works well and all effort should be put now into the mlvfs-apps. this is a clever timesaving-idea and if something's problematic with cdng i think i can help there to even make it better :)
regards chmee
Hello Chmee, thx for all your effort!
I use mlvfs (as you probably know from the mlvfs thread) i have also some pixel problems with chroma smoothing. I decided to disable it, but perhaps it could be improved to handle the dark areas right. Mlvfs is indeed the easiest and fastest way to get generated dngs.
Greets swinxx
thanks to @dmilligan
* cdngs and audiofile behave under resolve as wanted :) autosync as a mediafile now, not only autosync-functional.
* found a 16->12bit bug. fixed.
so, give me some more days and a new version will arrive..
@swinxx
jepp, but i will have marked this tool as "complete" - and im still learning with every bug i find.
raw2cdng 1.6.3 beta is online
* chroma smoothing
* corrected length of wav-audio, thus it works now in resolve. (autosynced media-bins with according timecode)
* preview now scrubbing instead of playing.
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
regards chmee
thank q so much for implementing chroma smoothing. but i can test it only next month as im out of my town
OMG Thank you for chroma smoothing!
Btw, does anyone getting different K of white balance when using v1.6.3 vs v.1.6.1?
@feureau
could you show me a screenshot or write some more infos to it? i have rewritten much of the code, its possible i forgot something
Screenshot: http://i.imgur.com/PMljcpD.png
Basically, they're the same MLVs loaded on two versions of raw2cdng. The resulting DNGs seem to come out correctly though. And it seems the 1.6.1 didn't preview WB correctly.
Also: I've getting this error:
Problem signature:
Problem Event Name: CLR20r3
Problem Signature 01: raw2cdng.1.6.3.beta.exe
Problem Signature 02: 1.5.0.8
Problem Signature 03: 546f5484
Problem Signature 04: mscorlib
Problem Signature 05: 4.0.30319.18444
Problem Signature 06: 52717edc
Problem Signature 07: 442a
Problem Signature 08: 0
Problem Signature 09: System.IO.IOException
OS Version: 6.1.7601.2.1.0.256.1
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789
Also: That chroma smoothing really slow conversions down... But I'm not complaining. :3
(*) thanks. assume, i know, where's the bug.
(*) there s an io-exception. are they used/opened in an app in background? it seems, it couldnt erase/rewrite them. fi with premiere?
(*) will look, if i can optimize, but yes, chromasmoothing slows down the conversion massivelly. before that, there only have been bit-shiftings and multiplications of values..
regards chmee
Can you tell me the difference between 16 bit and 16 bit Maximized?
ps. this is my new Go To tool for conversion.
Thank you so much Chmee!
as you maybe know, the canon-rawdata has values between blacklevel (fi 2047) and whitelevel (15000). it was necessary to maximize the valuerange for premiere. subtracting the blacklevel and scaling the values to the maximum value (16bit = 65535). i'd say, in general there's no drawback, but for workflow with acr (photoshop/ae) and lightroom i assume, its good to have these sparedata, you can reconstruct more highlights and optimize the noise-recognition.
btw: i wrote this pinkhighlight-fix for premiere, but it seems, its not needed anymore, adobe-coders did something against it. prove me wrong :)
regards chmee
Quote from: chmee on November 24, 2014, 11:11:09 AM
(*) thanks. assume, i know, where's the bug.
(*) will look, if i can optimize, but yes, chromasmoothing slows down the conversion massivelly. before that, there only have been bit-shiftings and multiplications of values..
regards chmee
You're welcome, and thank you!! :D
Quote from: chmee on November 24, 2014, 11:11:09 AM
(*) there s an io-exception. are they used/opened in an app in background? it seems, it couldnt erase/rewrite them. fi with premiere?
I don't have anything else running at the time except regular background stuff like chrome background etc.
Quote from: chmee on November 24, 2014, 01:13:42 PM
as you maybe know, the canon-rawdata has values between blacklevel (fi 2047) and whitelevel (15000). it was necessary to maximize the valuerange for premiere. subtracting the blacklevel and scaling the values to the maximum value (16bit = 65535). i'd say, in general there's no drawback, but for workflow with acr (photoshop/ae) and lightroom i assume, its good to have these sparedata, you can reconstruct more highlights and optimize the noise-recognition.
btw: i wrote this pinkhighlight-fix for premiere, but it seems, its not needed anymore, adobe-coders did something against it. prove me wrong :)
regards chmee
So, to verify: For ACR/lightroom/Resolve - best to convert without Maximized?
QuoteSo, to verify: For ACR/lightroom/Resolve - best to convert without Maximized?
has to be proven as well.
@feureau
sample-pictures regarding wb above. you recorded outside, i'd say ~6000°K, and set the wb manually to 3100°K. i think, the newer picture is colder because its the wrong wb. do a test - dragdrop the file on raw2cdng-icon (not into the opened app), it should open the player. with arrows left/right, P(resets) and O(riginal) you can look on wb-changes. is it maybe fine, if you change it to something like 6000°k?
regards chmee
Quote from: chmee on November 24, 2014, 02:27:14 PM
has to be proven as well.
@feureau
sample-pictures regarding wb above. you recorded outside, i'd say ~6000°K, and set the wb manually to 3100°K. i think, the newer picture is colder because its the wrong wb. do a test - dragdrop the file on raw2cdng-icon (not into the opened app), it should open the player. with arrows left/right, P(resets) and O(riginal) you can look on wb-changes. is it maybe fine, if you change it to something like 6000°k?
Using 1.6.3 - Looks about neutral with AWB preset at 4500K, filedata is at 3100K Manual WB
BTW, I was just converting some DNGs, and found that the latter part of the batch turned up like this:
http://i.imgur.com/gokLxhZ.png
It's not wb issue, there's nothing in the Resolve raw settings I can play with that would fix it. I tried converting the MLV with mlrawviewer and it come out just fine. I tried converting other MLVs with raw2cdng and it turns out this way all the time. I even tried re-downloading the app and it still come out this way. Any ideas?
Furthermore, about half of the DNG that came out right can't be synced with the wav in resolve. Anyone having this issue too?
@feureau
there's a debug button on top, enable, close, open. it will be written into the same folder as the app. send me a non-functional dng (or mlv) file to
[email protected] and show me that debug.
regards chmee
@feureau
thanks. my tool doesnt change black- and whitelevel in this beta. thanks for hint. will be solved next hours. if you know exiftool, you can change blacklevel to 0 and whitelevel to 60.000.
Thank you, too!
Can't wait to use the fixed version :D
Hi, chmee. I now kept getting entirely black DNGs. Tested this from 1.5.0b6 up to 1.6.3b. I tried generating proxy, and resulted in this: http://i.imgur.com/sMEAyHv.png Otherwise, the dng is just a black frame.
Also, raw2cdng 1.6.3b seems to crash when I feed it 1 kb MLVs:
https://mega.co.nz/#!P9EklSra!bKh4OjnL8YuEdv4QA78JgPATRRV6d8gDViOqEHcbmYs
https://mega.co.nz/#!KhFFkSYL!0XwzGSNn-AGAaf8iPYoMacie-q_Wain7GZHaihm4P44
They were created when I hit record then quickly stop the recording. (usually pushing the wrong button)
How's that update coming along btw?
:) those files are for sure invalid. so i could do a simple request, if a file is at least 1mb. sorry for the delay with next version, i have to work a lot. try it next days.
I couldn't get one close to 1mb, but anything above that doesn't crash raw2cdng.
Still getting black DNGs though. Is this something to do with .net framework? Does raw2cdng write settings somewhere? It seems to remember certain options when I switch between version. Maybe I need to reset some settings..?
Thanks for this btw. Looking to try the next version in a few days. :D
Just ran a test with raw2cdng 1.6.3. I was unable to import the cDNGs into AE. I recieved this error.
(http://i.imgur.com/nLGkxQp.jpg)
With 1.6.3 I used 12bit max with vertical banding on and chroma on and off. Normally, I can see the .dng's in Windows Explorer on Windows 8.1 but, they did not load and I was unable to see them in Windows Photo Viewer.
raw2cdng 1.6.1 worked just fine.
Here is the debug from raw2cdng 1.6.3
------------- 1.6.3.beta started at 14.12.10 11:45:38 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file C:\Users\Chad\Desktop\M10-1231.MLV.
[drop] ------
[drop] -- file C:\Users\Chad\Desktop\M10-1231.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in C:\Users\Chad\Desktop\M10-1231.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
***** [getMLVAttributes] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
*****
***** [getMLVAttributes] metaData Object:
xResolution=1920
yResolution=818
frames=116
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrix=System.Byte[]
lostFrames=0
fpsNom=23976
fpsDen=1000
duration=00:04:00
dropFrame=False
fpsString=23.98
stripByteCount=2748480
stripByteCountReal=2748480
modell=Canon EOS 5D Mark III
camId=MAGICf24bd98
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0.69235970250169
wb_G=1
wb_B=0.410092110532639
whiteBalance=3200
whiteBalanceMode=3
whiteBalanceGM=4900
whiteBalanceBA=4900
blackLevelOld=2047
blackLevelNew=2047
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5.05952289045009
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.3.beta
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
deadSensel=System.Collections.Generic.List`1[raw2cdng_v2.point]
*****
***** [getMLVAttributes] fileData Object:
fileName=C:\Users\Chad\Desktop\M10-1231.MLV
fileNameOnly=M10-1231
fileNameShort=M101231W
fileNameNum=101231
tempPath=C:\Users\Chad\AppData\Local\Temp\
sourcePath=C:\Users\Chad\Desktop
parentSourcePath=Desktop
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=12/10/2014 11:39:48 AM
modificationTime=12/10/2014 12:31:31 PM
VIDFBlock=
RAWBlock=
*****
***** [getMLVAttributes] lensData Object:
lens=24-70mmP4I:
focalLength=70
aperture=0
isoValue=3200
shutter=1/64s
*****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 393
[drop] hasAudio. AUDF-List created. Blocks: 25
[drop] VIDF-List created. Blocks: 114
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[drop] VIDF-Blockdata read and created.
[drop] adding M10-1231 to batchList
[showPicture] started
[drop] calculating verticalBandingCoeefs and searching hot/dead-pixels..
[drop] NO dead sensels found
[drop] verticalBanding enabled. decision: do it
[drop] verticalBanding CoEffs: 1 1.0008630905897 1.00138402455486 1.01540475086815 1.00000057378494 1.00183129887873 1.00058909931684 0.987164737274471 1
[drop] ** Item M10-1231 imported | MLV
[drop] * res 1920x818px | 23.98fps | 114 frames | BL2047 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 114 | has audio
[convert_Click] all in all there are 114 frames to convert
[doWork] started
[doWork] 12 Cores used
[doWork] -> converting item M10-1231
[doWork] settings format 4 with BL0 WL4095 with+ maximizingvalue 5
[doWork] prefix - filename generator 1.6.3 Riggs
[doWork] destinationPath -> 1.6.3 Riggs
[doWork] outputFilename -> 1.6.3 Riggs
[doWork] Directory C:\Users\Chad\Desktop\Video Test\1.6.3 Riggs created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork][for] read/convert MLV VIDF: [0]
[doWork][for] read/convert MLV VIDF: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]
[doWork][for] read/convert MLV VIDF: [26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50]
[doWork][for] read/convert MLV VIDF: [51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75]
[doWork][for] read/convert MLV VIDF: [76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99][100]
[doWork] convert Done
[doWork] hasAudio file -> 1.6.3 Riggs.wav
[saveAudio] started
***** [saveAudio] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
*****
[saveAudio] bextName: Canon EOS 5D Mark III
[saveAudio] UDI: DECanMAGICF24BD98123131484946241
[saveAudio] datetime: 2014:12:1012:31:31
[saveAudio] timeRefMultiplier: 1.001001001001
[saveAudio] time2frame: 42030.03003003
[saveAudio] timeRef (samplingRate*time2frame): 2017441441
[saveAudio] fpsString: 23976/1000
[saveAudio] DF: NDF
[saveAudio] subChunkSize :951652
[saveAudio] success
I can send you the MLV if you would like.
i'm pretty sure, its about some wrong black/whitelevel. please be patient and use 1.6.1 by now.
No problem! Thanks for all the effort you have put into the product.
I was working in 1.6.1 and recieved black frames as well. This was after I used 1.6.3. I deleted all of the files for raw2cdng that I downloaded and redownloaded 1.6.1 and had the same issue. I also noticed all of my settings, such as the filename, were the same even after deleting the file. Does this product save files else where for settings? Should they be deleted to correct this issue? If you need me to run some test, or send you some files, I'll be glad to help.
there is the settings-file inside User\[you]\appdata\roaming named raw2cdng.settings.json
regards chmee
So that's where that is...
Might be a good idea to keep these things inside the same folder as the main program files. I mean, it's kinda hard to find...
That did the trick! Thanks!
Hey Chmee.
I was converting files and this is the first time I try Vertical Banding fix. Not something usually pay attention to in my footage, but everyone is complaining about it so I thought I better "fix" the issue with my CDNG's.
Problem is that some of the shots now have extreme vertical banding and some dont.. Even though all were exported in one run.
Here is an example: https://www.dropbox.com/s/3ilasxv87den79i/Austria%201_M12-0712_140819_12071200000.dng?dl=0 You can see it especially in shadows. This cdng has VisionLog on it.
Could there be some setting that I am doing wrong or something just went wrong with the conversion?
Ver: 1.6.3 beta
16 bit (not maximized)
Vertical Banding on (Green)
no chroma smoothing
No proxy
No highlight fix
sometimes the estimation of vertical differences fails, so yes, depending on footage it can deliver worse data. i wouldnt enable vertical banding by default.
[please be still patient solving bugs, i've got birthday today, and the next days i will party as well :)]
regards chmee
Happy birthday!! Hope you have a great one :)
and thanks for the quick reply!
Yeah I will stick to my usual conversion without vertical fix and if I see vertical banding, I will try vertical fix.
right on @chmee and Happy Birthday, mister!
Be safe.
happy birthday chmee
Happy Birthday, chmee! :D
raw2cdng newer versions like 1.6.1 and 1.6.3 has problem while importing xml from premiere pro cc 2014 to resolve lite. older versions are ok. i also tried shootin@50fps and use version 1.6.3 but there was no problem but those which were shot @23.97 fps has problem. resolve keep on crashing while importing xml. i m unable to figure out why this is happenin. older version like 1.4.9.3 works fine
Happy Birthday Chmee. Hope your party rocks/ed
thanks. (today is the main party, so it didnt ended yet :)) so, i ll address bugs asap. one by one. important is the first one with the wrong black/whitelevel, then i will look for the problems with resolve/premiere roundtripping..
@andy_kh
i think its a problem with dropped-framerates. have to re-check/test my code for none-integer fps. did you tested with/without sound?
cheers chmee
yeah i hv done test with sound and without sound @chmee
BTW enjoy your party
hi chmee, happy birthday :) :)... I am using 1.6.1 meanwhile.. but,
1) I get black frames and the tool closes showing "raw2cdng stopped working" even after deleting the .json file in c/user/appdata folder
2) I drop six to seven files (around 1 to 3GB), the tool closes showing the above error. This also occurs when I drop seven files and uncheck five of them to extract only two.
3) The tool shows "converting" even after the files have been extracted to cdng... If I load despite that, the tool closes showing error.
The above all occurs in 1.6.1.... thanks.. :)
Did you check if you are actually missing frames?
Cant help you with the other questions, sorry.
Why not try 1.6.3 ?
hi kharak, there are no missing frames.
I used 1.6.3 beta after facing above problems in 1.6.1, but 1.6.3 beta also crashes upon loading even two files at a time, each with frames greater than 1000.
since then, I am uploading and extracting single file at a time (total includes 100s and 100s of files for serious video purpose), very laborious :(... but seriously, glad to have this tool :)
CClean and Re-install, probably some registry still interferring from older version. Just a guess, I know this worked for me on Mlrawviewer.
Good luck!
@anandkamal if you show a debug, i/we can assume, on what point it crashes - btw. the crash on closing is known but imho not essential for anything (!)and: my tool doesnt write any data into the registry. someone had installed mlvrawviewer and my tool crashes, ater deleting mlvrawviewer mine worked, so sometimes its only magic :)
Hi Chmee! Happy birthday some days late :)
I see that you can't download the 1.6.3 version anymore. I found a bug in the audio from that release, only the first 10% of the audio was there in Davinci Resolve. i use the 1.6.1 for now, and the problem isn't there as it wasn't with previous versions as well.
Thanks again for a great tool. Really love it togheter with Davinci Resolve.
Kind Regards
Karl Allemyr
Hi chmee and friends, happy christmas ;)
the footage that is shot when a CF card is almost full, popping up a "card full" message, at times crashes the tool. WIll send a debug next time... btw, curious to knw how the update is coming up :D :D ;D
merry christmas. my december is kind of "training day" in party'ing. i did some tests with my footage, it works flawless. i already coded the new colormatrices. because i didnt implemented much exception-trapping/handling, most of these problems while recording will lead to a crash. by now i m still interested in the black dng's.. audiowise, i will put some effort into proper audiolength-handling.
finally i hope, i'll give you a new version this year.
wow this year!
Looking forward to it!
eagerly waiting....
vow.. u r awesome :)
put a 1.6.5 on my server (sourcecode in bitbucket refreshed as well)
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
(+) timeRef adjusted for proper sync-handling in resolve (seems to work now)
(+) new color-matrices as stated by andy600 (dmilligan uses them in mlvfs already (https://bitbucket.org/dmilligan/mlvfs/src/ae3c4696191da1a1e4b142780d6b0bee6e64a7df/mlvfs/dng.c?at=master))
but:
(-) i couldnt find any showstopper-problems with my mlv-files. @feureau @anandkamal @andy_kh @chadmuffin please give me some more informations (body, mlv-file) again, maybe write them into bitbucket as issue (https://bitbucket.org/chmee/raw2cdng/issues)
@feureau regarding wb-levels-problem, use mlvdiag1.01 from my site and shorten the file to ~55mb. send it to me. under what circumstances does this happen? (output-properties? bitdepth? proxy? debugfile?)
@chadmuffin not recognised by adobe-apps. yeah, had that as well, seems to be gone now. please test and tell.
did the blackframes went away after deleting settings-file?
@andy_kh i will do more tests now with xml-roundtripping, maybe its the same bug i fixed now (timeRef in audiofile) with two files just tested it worked. no crashes.
regards chmee
Thank you for 1.6.5 \o/
I'm gonna have to try shoot some stuff for that wb thing and see if it's still / how it's happening. Doesn't seem to be a problem so far though.
btw. i did this chart for beginners to clarify some questions.
(http://dl.phreekz.de/raw2cdng/cdng_mlv_raw_ml_bitdepth.gif)
regards chmee
thank q for the new version. i wil try once i get back home
Thanks for all your hard work! I'll run tests after the holiday season.
i am not able to drag and drop mlv file in the new version. i also tried the same mlv file in version 1.6.1 but no problem
@andy_kh
i know, windows sometimes disables dragdrop an apps, but works after time. if its not a systemwide problem, try to delete and/or reuse from another place.. or a simple shutdown/restart..
links:
http://www.sevenforums.com/general-discussion/299845-my-drag-drop-function-not-working.html
http://forum.thewindowsclub.com/windows-7-management-support/32562-drag-drop-not-working-windows-7-explorer.html
regards chmee
after shutdown/restart i cant stil drag n drop mlv files in the new version 1.6.5 whereas i dont have any problem with version 1.6.3 and 1.6.1
You may need to delete some old settings file. On windows: User\[you]\appdata\roaming named raw2cdng.settings.json. It helped me when using 1.6.3 with black frames, might help with you. Also, removed your old raw2cdngs and download the lastest stable one for you. You may need to run it as an administrator in Windows. If you are using a different OS, don't know...
Btw, chmee, is that black/white level dng supposed to be fixed in 1.6.5? I'm still getting that green cast. (using CDNG 16bit and vertical banding and chroma smoothing on)
http://i.imgur.com/EwfICfY.jpg
I deleted the .json too btw.
EDIT: For those looking to fix DNGs with similar issues using exiftool and kept failing to get black/whitelevel to change:
BlackLevel is written to SubIFD by default. Nothing will happen if the DNG doesn't contain a SubIFD.
Use this command to figure out where BlackLevel is stored:
exiftool -G1 -blacklevel image.dng
then write it to the same place:
exiftool -GROUP:BlackLevel=0 image.dng
In my case the GROUP is IFD0The complete command is thusly:
exiftool -IFD0:BlackLevel=0 *.dng
Same with whitelevel
@feureau
please send me a short mlv with that problem. (pleaseplease)
Quote..I'm still getting that green cast. (using CDNG 16bit and vertical banding and chroma smoothing on)..
maybe thats a hint i have to re-check. seems, its not re-setting bl/wl if helpers are enabled. thx.
You're welcome :D Looking forward for the fix.
Btw, any plans for hot pixel detection/removal thingy? :P
I ran some tests and here are my results:
5D3 with raw2cdng 1.6.5 in Adobe products on Windows 8.1.
I used vertical banding and chroma smoothing 2x2 in raw2cdng (after I deleted settings.json from previous builds). raw2cdng worked great! White balance was from the camera correctly, it correctly knew my 23.976 fps, 5 fps and 48 fps. There were no black frames or green tint. It did not know my aperture from my lens though. Came back as f/0, which could be because I use Tamron lenses? Not hindering the workflow though so not really an issue. I was able to successfully view them in Windows Explorer and the native photo viewer.
After Effects read the files just fine. I had to manually set the fps from 30 fps which seems to be typical. The .wav file started at the correct point and it went a few frames after but, that is easy to just cut that part out with exporting the clip. I used a clapper board over a span of 20 sec and did not see it the waveform off beat or visual see/hear it off beat.
I pulled it into Premiere Pro and brought in the .cdngs in through the media browser. The audio was on time, did not go past the video. However, it did not know my custom fps. It is an easy fix by adjusting the speed in Premiere.
I did not use Resolve.
Everything seems to be working great on my end! If you need more info or tests, just let me know.
Chmee, I think I narrowed it down a little bit more. It seems if vertical banding is triggered, the white/black level would be exif-clipped. If I select vertical banding and chroma smoothing first, then go change the CDNG bit level, it would not be exif-clipped.
Also: Enabling vertical banding on a 7D clip causes vertical stripes.
@ChadMuffin
thx for response. good to hear. aperture is just a string. i think its not really worth investigating on it. but if you#ve got a shortened mlv for me i'll try to look into this string.
@feureau
thanks a lot. i'll test this behaviour. as i said before, vertical banding can deliver worse data than before. if you want, send it as well, i'll try to narrow down, if its a coding or a logical problem.
regards chmee
I did a test with all of my lenses to see if it was specific lenses or all of them. I only have a small handful of lenses to test to include Canon 15mm Fisheye f/2.8, Tamron 24-70mm f/2.8 VC, Canon 50mm f/1.4, Tamron 70-200mm f/2.8 VC. I also have the Tamron SP AF Tele-Converter. The only 2 lenses that had "f/0" were the Canon 15mm and the Tamron 24-70mm. Also, these lenses had typos in their names. "EF15mm f/2.8 Fishee" and "24-70mmP4I" The other lenses were named properly. I also tried the tele-converter on my lenses just to see and the same result happened on the same lenses. The tele-converter did not change the results. I was able to recreate this test 3 times with the same results. Your software seems to be consistent, it appears to be with specific lenses. I have included an MLV of about 3 seconds from each of my lenses and one with the tele-converter for you to check out.
https://drive.google.com/file/d/0Bwd3r5HwGFltQ1dzMUluckU3TXM/view?usp=sharing
I'm still testing some footage, especially with the new color matrix. This is the same Adobe Standard color profile as in the Adobe Raw converter thingy, right? Do you have the Canon Standard profile too? Would be awesome to have that and the Portrait profile for awesome skin tones.
A slice of MLV: https://www.dropbox.com/s/bwbm0jbo0qqi992/M03-1438.MLV.55mb.mlv?dl=0
Hi all, i though its about time i try raw2cdng and Davinci Resolve. Now this here is off of 1.6.5 and there's something totally wrong in what i'm doing, could you please shoot me :)
https://www.youtube.com/watch?v=BPEn2kjDWvA
Five different exports:
1st (far left) - 12bit maximized
2nd - 16bit with chroma smoothing ON
3rd - 16bit with vertical banding ON
4th - 16bit with highlights fix ON
5th(far right) - 16bit (all fixes are OFF)
Those 5 were imported in Resolve 11.1 with just Color space changed to BMD Film and Highlight recovery On (which didn't do much)... pretty much nothing else!
not so much wrong :) it seems, if the buttons are clicked in the wrong order, black- and whitelevel are wrong set. i'm off to detroit next 10 days, i'll try to find the main problem there. pardon for this bug and the patience u need now.
(simple fix: delete the raw2cdng.settings-file in /users/[user]/appdata/roaming and then try only with 16bit without nothing else) - ah and look if you've got blacklevel-fix in body en- or disabled.
regards chmee
hmmm, will try that and report back :) and for the black level in cam, i'm pretty sure it is ON since whenever g3gg0 implemented it, will double check though
P.S. the simple fix did it!!! a thing of beauty :) thanks a ton, and btw i'll be using your tool a lot, so if you need material off a 5dm3, i'll be around
Quote from: chmee on January 02, 2015, 02:55:02 PM
ah and look if you've got blacklevel-fix in body en- or disabled.
Btw, I have this turned on in my 7D. Without it the darks will have a weird green cast, especially in high ISO.
Yeah, this one is a must! I thought it's a setting just for the 5DM3 & M2 ( as well as 50D ), but it's good to know it's also valid for the 7D
Hi.. before downloading 1.6.5, I had to do a video with 5d3 mlv extracted through earlier version (don't remember the version of raw2cdng). The procedure is premiere-resolve roundtrip. Roundtrip was successful. Every effects and cuts were maintained. Client was happy. Resolve (full), premiere cc 2014.
I now extracted new mlv files from 5d3 through 1.6.5 raw2cdng. I imported files to resolve after setting "master project settings" "assist reel name.." etc.,. I exported dnxhd 36 for proxy (as usual) and in premiere I did offline editing and exported finalcutpro XML for resolve roundtrip. This time I did not get the "FCP translation result.." notepad along with the xml (unlike in previous project). When i imported the XML to resolve 11. Resolve could not link the timeline to media pool, also nothing is seen under "reel name" section in resolve. Roundtrip failed and i had to manually link files.
Am i doing something wrong? any new tutorial on this roundtrip?
I think that was a resolve-premiere roundtripping issue. Did you uncheck the import material thingy when going back to resolve?
yes feureau.. i did uncheck the "automatically import..." The reel name is blank when i import the xml from premiere.... Is premeire-resolve roundtrip with mlv's extracted via 1.6.5 working for someone pls?? It worked a few days ago with old mlv files. Can u guide pls?
i found a strange problem with cdng files. im unable to import xml in resolve exported from premiere pro. i thought the problem may be with raw2cdng converter so i tried converting in mlvRawViewer and found same problem. resolve keep crashing. i also test shoting at different framrates. only 23.97 has the problem. shot at other framerate like 25 and 50 works fine. unable to figure out where is the problem. i test some old mlv shot wit 550D and they all work fine again
@andy_kh
23,97fps is defined as 24000/1001, whereas ml writes 23976/1000. in earlier versions i readjusted manually broken frames from fi 59940/1000 to 60000/1001. maybe thats a solution.
regards chmee
yeah i hope so. thanks for ur reply
regards andy kh
"23,97fps is defined as 24000/1001, whereas ml writes 23976/1000. in earlier versions i readjusted manually broken frames from fi 59940/1000 to 60000/1001. maybe thats a solution."
chmee.. sorry i don't understand. By earlier versions, do u mean raw2cdng tool?
How do i do that (manually adjusting broken frames)? thanks nd sorry for not knowing :)
fps are defined as fractions in cdng.
25/1 is 25fps, 25000/1000 also..
dropped frames look like
23.976fps = 23976/1000 or 24000/1001 (the first is the value ml saves, the second is the official definition)
i assume you can change them with exiftools (tag 51044 or C764.H), just for a test. its quite the same like changing black- or whitelevels. if this works, leave me an info, i will do back the handling for known broken framerates.
regards chmee
hi chmee...
may i know the preferred settings in raw2cdng 1.6.5 for direct cdng editing in premiere pro cc (2014), for DCP/theatrical projection output?
the info is all spread out in posts and a compiled answer could save many beginners like me :) :)
thanks
Problem raw2cdng 1.6.5 won't allow me to load any mlv files. I am on windows7 64bit.
Quote from: Jotys on January 11, 2015, 05:55:30 PM
Problem raw2cdng 1.6.5 won't allow me to load any mlv files. I am on windows7 64bit.
same problem with me
does it show anything in the debug-file?
@anandkamal
my last tests were with 12bit, 16bit and 16bit maximized WITHOUT highlightfix. Verticalbinding IF needed, Chromasmoothing IF needed. all these cdng's are usable in premiere - but i have to say, i didnt had done short- or musicvideos a long time - so i cant say, what are the optimum settings.
Opened RAW2CDNG and dropped a MLV file.
------------- 1.6.5 started at 15.01.11 20:54:16 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file E:\RAW2CDNG\M11-1112.MLV.
[drop] ------
[drop] -- file E:\RAW2CDNG\M11-1112.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] filesize counted (1348MB)
[createMLVBlocklist] indexing Blocks in E:\RAW2CDNG\M11-1112.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
[getMLVAttrib] [!] NO WAVI Block
***** [getMLVAttributes] metaData Object:
xResolution=1344
yResolution=756
frames=792
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=
lostFrames=0
fpsNom=23973
fpsDen=1000
duration=00:33:09
dropFrame=False
fpsString=23,97
stripByteCount=1778112
stripByteCountReal=1778112
modell=ERR:1 md:0x 0 ml:0r
camId=MAGIC21edb63
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0,730385164051355
wb_G=1
wb_B=0,376886271623114
whiteBalance=3000
whiteBalanceMode=9
whiteBalanceGM=3000
whiteBalanceBA=3000
blackLevelOld=2047
blackLevelNew=2047
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5,05952289045009
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.5
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
deadSensel=System.Collections.Generic.List`1[raw2cdng_v2.point]
*****
***** [getMLVAttributes] fileData Object:
fileName=E:\RAW2CDNG\M11-1112.MLV
fileNameOnly=M11-1112
fileNameShort=M111112V
fileNameNum=111112
tempPath=C:\Users\<Username>\AppData\Local\Temp\
sourcePath=E:\RAW2CDNG
parentSourcePath=RAW2CDNG
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=11.01.2015 18:34:27
modificationTime=11.01.2015 11:13:10
fileSize=1348
VIDFBlock=
RAWBlock=
*****
***** [getMLVAttributes] lensData Object:
lens=18-125mms9:
focalLength=18
aperture=0
isoValue=200
shutter=1/38s
*****
Left box looked like this:
(http://picload.org/image/cccaopw/raw2cdng_err.jpg)
Grey rectangle is there after drop. Red oval was added by me.
thanks. so:
Quotemodell=ERR:1 md:0x 0 ml:0r
is the problem. i assumed, the modellname will be written with no hickups (but im wrong). if the modell-name is not accordingly, it cant set colormatrix and forwardmatrix - this leads to another things. so, i will simulate that and fix it in the coming version.
(the same file opens fine in versions before 1.6.5?!)
Quote from: chmee on January 12, 2015, 12:25:19 AM(the same file opens fine in versions before 1.6.5?!)
Tested with 1.6.1: Working fine.
------------- 1.6.1 started at 15.01.12 00:37:03 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file E:\RAW2CDNG\M11-1112.MLV.
[drop] ------
[drop] -- file E:\RAW2CDNG\M11-1112.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in E:\RAW2CDNG\M11-1112.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
[getMLVAttrib] [!] NO WAVI Block
***** [getMLVAttributes] metaData Object:
xResolution=1344
yResolution=756
frames=792
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrix=System.Byte[]
lostFrames=0
fpsNom=23973
fpsDen=1000
duration=00:33:09
dropFrame=False
fpsString=23,97
stripByteCount=1778112
stripByteCountReal=1778112
modell=ERR:1 md:0x 0 ml:0r
camId=MAGIC5e57b61
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0,730385164051355
wb_G=1
wb_B=0,376886271623114
whiteBalance=3000
whiteBalanceMode=9
whiteBalanceGM=3000
whiteBalanceBA=3000
blackLevelOld=2047
blackLevelNew=2047
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5,05952289045009
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.1
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
*****
***** [getMLVAttributes] fileData Object:
fileName=E:\RAW2CDNG\M11-1112.MLV
fileNameOnly=M11-1112
fileNameShort=M111112V
fileNameNum=111112
tempPath=C:\Users\<username>\AppData\Local\Temp\
sourcePath=E:\RAW2CDNG
parentSourcePath=RAW2CDNG
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=11.01.2015 18:34:27
modificationTime=11.01.2015 11:13:10
VIDFBlock=
RAWBlock=
*****
***** [getMLVAttributes] lensData Object:
lens=18-125mms9:
focalLength=18
aperture=0
isoValue=200
shutter=1/38s
*****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 1617
[drop] VIDF-List created. Blocks: 792
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[readVIDFBlockData] done: [100][101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124]
[readVIDFBlockData] done: [125][126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143][144][145][146][147][148][149]
[readVIDFBlockData] done: [150][151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174]
[readVIDFBlockData] done: [175][176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199]
[readVIDFBlockData] done: [200][201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224]
[readVIDFBlockData] done: [225][226][227][228][229][230][231][232][233][234][235][236][237][238][239][240][241][242][243][244][245][246][247][248][249]
[readVIDFBlockData] done: [250][251][252][253][254][255][256][257][258][259][260][261][262][263][264][265][266][267][268][269][270][271][272][273][274]
[readVIDFBlockData] done: [275][276][277][278][279][280][281][282][283][284][285][286][287][288][289][290][291][292][293][294][295][296][297][298][299]
[readVIDFBlockData] done: [300][301][302][303][304][305][306][307][308][309][310][311][312][313][314][315][316][317][318][319][320][321][322][323][324]
[readVIDFBlockData] done: [325][326][327][328][329][330][331][332][333][334][335][336][337][338][339][340][341][342][343][344][345][346][347][348][349]
[readVIDFBlockData] done: [350][351][352][353][354][355][356][357][358][359][360][361][362][363][364][365][366][367][368][369][370][371][372][373][374]
[readVIDFBlockData] done: [375][376][377][378][379][380][381][382][383][384][385][386][387][388][389][390][391][392][393][394][395][396][397][398][399]
[readVIDFBlockData] done: [400][401][402][403][404][405][406][407][408][409][410][411][412][413][414][415][416][417][418][419][420][421][422][423][424]
[readVIDFBlockData] done: [425][426][427][428][429][430][431][432][433][434][435][436][437][438][439][440][441][442][443][444][445][446][447][448][449]
[readVIDFBlockData] done: [450][451][452][453][454][455][456][457][458][459][460][461][462][463][464][465][466][467][468][469][470][471][472][473][474]
[readVIDFBlockData] done: [475][476][477][478][479][480][481][482][483][484][485][486][487][488][489][490][491][492][493][494][495][496][497][498][499]
[readVIDFBlockData] done: [500][501][502][503][504][505][506][507][508][509][510][511][512][513][514][515][516][517][518][519][520][521][522][523][524]
[readVIDFBlockData] done: [525][526][527][528][529][530][531][532][533][534][535][536][537][538][539][540][541][542][543][544][545][546][547][548][549]
[readVIDFBlockData] done: [550][551][552][553][554][555][556][557][558][559][560][561][562][563][564][565][566][567][568][569][570][571][572][573][574]
[readVIDFBlockData] done: [575][576][577][578][579][580][581][582][583][584][585][586][587][588][589][590][591][592][593][594][595][596][597][598][599]
[readVIDFBlockData] done: [600][601][602][603][604][605][606][607][608][609][610][611][612][613][614][615][616][617][618][619][620][621][622][623][624]
[readVIDFBlockData] done: [625][626][627][628][629][630][631][632][633][634][635][636][637][638][639][640][641][642][643][644][645][646][647][648][649]
[readVIDFBlockData] done: [650][651][652][653][654][655][656][657][658][659][660][661][662][663][664][665][666][667][668][669][670][671][672][673][674]
[readVIDFBlockData] done: [675][676][677][678][679][680][681][682][683][684][685][686][687][688][689][690][691][692][693][694][695][696][697][698][699]
[readVIDFBlockData] done: [700][701][702][703][704][705][706][707][708][709][710][711][712][713][714][715][716][717][718][719][720][721][722][723][724]
[readVIDFBlockData] done: [725][726][727][728][729][730][731][732][733][734][735][736][737][738][739][740][741][742][743][744][745][746][747][748][749]
[readVIDFBlockData] done: [750][751][752][753][754][755][756][757][758][759][760][761][762][763][764][765][766][767][768][769][770][771][772][773][774]
[drop] VIDF-Blockdata read and created.
[drop] adding M11-1112 to batchList
[showPicture] started
[drop] verticalBanding enabled. decision: not needed
[drop] verticalBanding CoEffs: 1 1,00241055350834 0,998533097079163 0,999124084358047 0,997837972587185 0,999753836807715 0,995905262997038 0,998138322767508 0
[drop] ** Item M11-1112 imported | MLV
[drop] * res 1344x756px | 23,97fps | 792 frames | BL2047 | WL15000
[drop] * modell ERR:1 md:0x 0 ml:0r | vidfblocks 792 | has no audio
ok. so i have to do a check against a list of models, if its not in, i have to set to one. (or recheck against the first value of the colormatrix inside the raw/mlv) thats not really perfect, but gives an output. (notice for me: giving an info to the user for those errors/bugs inside the gui).
what camera/ml-firmware do you've got?
i assume, 1.6.5 works ok, if you use a raw instead of a mlv-file :) (for raw-files i use the model-assumption on color-matrix-values)
7D and 650D available here. Testing different ML versions is not the problem. Some cards here, too.
File logged was created with 650D and ML version 2015Jan03.
EDIT: RAW seems to work fine.
EDITEDIT: I may revise that ...
[drop] * modell Canon EOS 700D | rawblocks 240 | has noaudio
hmm.. maybe its only the newest ml-version - i hope. can be a discussion thing for cameraspecific-thread or as general bug in newest ml-fw.
EDIT: yeah, the raw-module recognises the body out of the colormatrix (with raw the only way to define it). as i assumed, the model-writing inside the body seems faulty. from my sourcecode:
case 6602: // first value of colormatrix
rData.data.metaData.modell = "Canon EOS 700D"; // thats for 650D, 100D, EOSM as well
(bitbucket io.cs line ~770 (https://bitbucket.org/chmee/raw2cdng/src/e62afbaf5fc524f884b813e1f2d77510b38c4b39/csharp2010_code/raw2cdng_v2/io.cs?at=master))
650D, ML build 2014Oct07 and MLV recording: No, same error.
And MLV created with ML version 2014Apr29 won't do, either.
QuoteI imported files to resolve after setting "master project settings" "assist reel name.." etc.,. I exported dnxhd 36 for proxy (as usual) and in premiere I did offline editing and exported finalcutpro XML for resolve roundtrip. This time I did not get the "FCP translation result.." notepad along with the xml (unlike in previous project). When i imported the XML to resolve 11. Resolve could not link the timeline to media pool, also nothing is seen under "reel name" section in resolve. Roundtrip failed and i had to manually link files.
Am i doing something wrong? any new tutorial on this roundtrip?
I test new workflow with raw2cdng 1.6.5 and AP CC 2014 (Davinci->AP->Davinci) and all work correct for me. If this topic is of interest to someone I could write a short lesson about it.
But one question for @anandkamal - in Resolve after AP you edit cdng or you render some kind of Log?
thanks Artyg.. in resolve i just go on with coloring after linking the offline-edited timeline to cdng mediapool. I don't render anything for resolve. But after I grade, I would render dnxhd 220 for final import into AP.
I am currently editing cdngs directly in premiere cc. Even if I get pink highlights or other issues, I don't have to worry as I will import the xml to resolve checking on "automatically import files from media pool." Resolve never shows issues with cdng files (like pink highlights etc.,.). After coloring, I render high quality footage back to premiere for final mastering.
It would be great to have a lesson on the resolve-premiere-resolve-premiere roundtrip, as things are updated now :)
I did a quick test with cdng (not log renders) and you are right - Resolve crash when I try import xml from AP. Only EDL I was able to import to Resolve. But content not linked. Only edit (cut) positions. Some problem with reel names. I will try tomorrow more detail test. BTW I recomend you to use Cinelog-C to render high quality log source and work with it, not cdng.
hi artyg..thanks and how do i do that? rendering cinelog-c..?
This is is a paid lut bank. Visit official site http://www.cinelogdcp.com/ (http://www.cinelogdcp.com/) and forum thread http://www.magiclantern.fm/forum/index.php?topic=10151.0 about additional info. BTW I will try to record my whole roundtrip Resolve-AP and publish it.
hi.. there seems to be a problem.. Few days ago, when I imported my .raw files to raw2cdng 1.6.5, it showed me AWB (autowhite balance?) and it set the temperature to 4500 for all .raw files. I carried on with that along with my .mlv files, which got extracted fine (showed the temperature that i shot with).
Now, my .mlv files too show in the tool that the temperature is 4500 AWB.. This was not the temperature i set while shooting.
------------- 1.6.5 started at 15.01.24 05:47:40 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file L:\NSamanpdu\Taxi Green\sel14\M26-0441.MLV.
[drop] ------
[drop] -- file L:\NSamanpdu\Taxi Green\sel14\M26-0441.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] filesize counted (1545MB)
[createMLVBlocklist] indexing Blocks in L:\NSamanpdu\Taxi Green\sel14\M26-0441.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
***** [getMLVAttributes] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
*****
***** [getMLVAttributes] metaData Object:
xResolution=1920
yResolution=1080
frames=446
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=
lostFrames=0
fpsNom=23976
fpsDen=1000
duration=00:18:08
dropFrame=False
fpsString=23.98
stripByteCount=3628800
stripByteCountReal=3628800
modell=Canon EOS 5D Mark III
camId=MAGICa8ff4d4
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0.540084388185654
wb_G=1
wb_B=0.547008547008547
whiteBalance=4500
whiteBalanceMode=9
whiteBalanceGM=4500
whiteBalanceBA=4500
blackLevelOld=2048
blackLevelNew=2048
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5.05991352686844
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.5
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
deadSensel=System.Collections.Generic.List`1[raw2cdng_v2.point]
*****
***** [getMLVAttributes] fileData Object:
fileName=L:\NSamanpdu\Taxi Green\sel14\M26-0441.MLV
fileNameOnly=M26-0441
fileNameShort=M260441J
fileNameNum=260441
tempPath=C:\Users\Kamali\AppData\Local\Temp\
sourcePath=L:\NSamanpdu\Taxi Green\sel14
parentSourcePath=sel14
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=7/26/2014 3:49:42 PM
modificationTime=7/26/2014 4:41:04 AM
fileSize=1545
VIDFBlock=
RAWBlock=
*****
***** [getMLVAttributes] lensData Object:
lens=EF50mm f/1.8 II:
focalLength=50
aperture=280
isoValue=3200
shutter=1/50s
*****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 1193
[drop] hasAudio. AUDF-List created. Blocks: 94
[drop] VIDF-List created. Blocks: 443
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[readVIDFBlockData] done: [100][101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124]
[readVIDFBlockData] done: [125][126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143][144][145][146][147][148][149]
[readVIDFBlockData] done: [150][151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174]
[readVIDFBlockData] done: [175][176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199]
[readVIDFBlockData] done: [200][201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224]
[readVIDFBlockData] done: [225][226][227][228][229][230][231][232][233][234][235][236][237][238][239][240][241][242][243][244][245][246][247][248][249]
[readVIDFBlockData] done: [250][251][252][253][254][255][256][257][258][259][260][261][262][263][264][265][266][267][268][269][270][271][272][273][274]
[readVIDFBlockData] done: [275][276][277][278][279][280][281][282][283][284][285][286][287][288][289][290][291][292][293][294][295][296][297][298][299]
[readVIDFBlockData] done: [300][301][302][303][304][305][306][307][308][309][310][311][312][313][314][315][316][317][318][319][320][321][322][323][324]
[readVIDFBlockData] done: [325][326][327][328][329][330][331][332][333][334][335][336][337][338][339][340][341][342][343][344][345][346][347][348][349]
[readVIDFBlockData] done: [350][351][352][353][354][355][356][357][358][359][360][361][362][363][364][365][366][367][368][369][370][371][372][373][374]
[readVIDFBlockData] done: [375][376][377][378][379][380][381][382][383][384][385][386][387][388][389][390][391][392][393][394][395][396][397][398][399]
[readVIDFBlockData] done: [400][401][402][403][404][405][406][407][408][409][410][411][412][413][414][415][416][417][418][419][420][421][422][423][424]
[drop] VIDF-Blockdata read and created.
[drop] adding M26-0441 to batchList
[showPicture] started
[drop] calculating verticalBandingCoeefs and searching hot/dead-pixels..
[drop] NO dead sensels found
[drop] verticalBanding enabled. decision: do it
[drop] verticalBanding CoEffs: 1 0.983723055487067 0.996604155729983 0.989969260901431 0.999345336570097 0.982056042319198 0.990180069471225 0.970404062519835 1
[drop] ** Item M26-0441 imported | MLV
[drop] * res 1920x1080px | 23.98fps | 443 frames | BL2048 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 443 | has audio
[convert_Click] all in all there are 443 frames to convert
[doWork] started
[doWork] 8 Cores used
[doWork] -> converting item M26-0441
[doWork] settings format 0 with BL2048 WL15000 with+out maximizingvalue 5.05983631871526
[doWork] prefix - filename generator [F]_[D]_[T](_)
[doWork] destinationPath -> M26-0441_140726_154942
[doWork] outputFilename -> M26-0441_140726_154942_
[doWork] Directory F:\CDNGrepairs\M26-0441_140726_154942 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork][for] read/convert MLV VIDF: [0]
[doWork][for] read/convert MLV VIDF: [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]
[doWork][for] read/convert MLV VIDF: [26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49][50]
[doWork][for] read/convert MLV VIDF: [51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75]
[doWork][for] read/convert MLV VIDF: [76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99][100]
[doWork][for] read/convert MLV VIDF: [101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124][125]
[doWork][for] read/convert MLV VIDF: [126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143][144][145][146][147][148][149][150]
[doWork][for] read/convert MLV VIDF: [151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174][175]
[doWork][for] read/convert MLV VIDF: [176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199][200]
[doWork][for] read/convert MLV VIDF: [201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224][225]
[doWork][for] read/convert MLV VIDF: [226][227][228][229][230][231][232][233][234][235][236][237][238][239][240][241][242][243][244][245][246][247][248][249][250]
[doWork][for] read/convert MLV VIDF: [251][252][253][254][255][256][257][258][259][260][261][262][263][264][265][266][267][268][269][270][271][272][273][274][275]
[doWork][for] read/convert MLV VIDF: [276][277][278][279][280][281][282][283][284][285][286][287][288][289][290][291][292][293][294][295][296][297][298][299][300]
[doWork][for] read/convert MLV VIDF: [301][302][303][304][305][306][307][308][309][310][311][312][313][314][315][316][317][318][319][320][321][322][323][324][325]
[doWork][for] read/convert MLV VIDF: [326][327][328][329][330][331][332][333][334][335][336][337][338][339][340][341][342][343][344][345][346][347][348][349][350]
[doWork][for] read/convert MLV VIDF: [351][352][353][354][355][356][357][358][359][360][361][362][363][364][365][366][367][368][369][370][371][372][373][374][375]
[doWork][for] read/convert MLV VIDF: [376][377][378][379][380][381][382][383][384][385][386][387][388][389][390][391][392][393][394][395][396][397][398][399][400]
[doWork][for] read/convert MLV VIDF: [401][402][403][404][405][406][407][408][409][410][411][412][413][414][415][416][417][418][419][420][421][422][423][424][425]
[doWork] convert Done
[doWork] hasAudio file -> M26-0441_140726_154942.wav
[saveAudio] started
***** [saveAudio] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
*****
[saveAudio] bextName: Canon EOS 5D Mark III
[saveAudio] UDI: DECanMAGICA8FF4D4044104285728854
[saveAudio] datetime: 2014:07:2604:41:04
[saveAudio] timeRefMultiplier: 1.001001001001
[saveAudio] time2frame: 16880.8808808809
[saveAudio] timeRef (samplingRate*time2frame): 810282282
[saveAudio] fpsString: 23976/1000
[saveAudio] DF: NDF
[saveAudio] subChunkSize :3698084
[saveAudio] success
other .mlv files in other folders are each opening with some temperatures like 5800, 5000 etc.,. Each folder/shooting location has its own temperature shown for all the files contained in it. Is this how it will be? or am i doing something wrong?
You are converting to CinemaDNG RAW. I really don't think it matters what white balance you set it to in Camera or how they are converted out, you can always change the WB before you actually start working on footage.
For in camera WB, its nice to set it somewhere close to what you believe is the right WB when shooting, but only for shooting preferences, the beauty of RAW is that you don't have to care about WB, you can set it to 12000K or 2000K, it doesn't affect the final image.
Hi all,
chmee, thank you for your efforts and results. I'd like to put raw2cdng in my workflow and there is just an element I'd like to understand:
I am using 1.6.5 version and trying to understand how to solve some vertical banding issues: in order to reduce vertical banding, should the "vertical banding" button be green or white?
I am asking because when the button is white I get absolutely better results than when it is green. When the button is green, vertical banding is very strong.
I am also comparing the results with MLRawViewer's output and I tink that banding in this MLRV is almost absent (with banding reduction on).
I'd really like to pass to raw2cdng since I find it more stable and I see you are still developing it.
Thank you. Cheers.
(//)
would be fine you send me an example of yours to inspect it. (use mlvdiag from my site to extract first 55mb from this example and send it to
[email protected]) - you re not the first one and it seems its really necessary to change some code ;)
by the way, as every version, please some patience, i'm recoding the file-read-function to get faster transcoding straight from cf-card.. thanks
regards chmee
wetransfer sent ;)
Thank you. Regards.
:) thx. will do my best.
ok. here s a 1.6.9.ALPHA DO NOT USE FOR PRODUCTION PURPOSES!
Download 1.6.9.alpha (http://dl.phreekz.de/raw2cdng/raw2cdng.1.6.9.ALPHA.zip)
(*) by now it only can transcode single MLV - not stripped files. (!) no raw(!)
-- Fixes/Patches --
(*) modelname, if not found/wrong in mlv, will be recreated from colormatrix-values @Walter_Schulz
(*) fps-tag - dropped frames are now well-formed (25000/1001 instead of 24975/1000)
(*) massive speed up on transcoding straight from cf-card/hdd
-- cf-card(usb3.0) to ssd - 2GB file (1.6.5 ~3min, 1.6.9 ~30seconds) 6x faster
-- hdd to hdd - 2GB file (1.6.5 ~34.9sec, 1.6.9 ~9.5sec) more than 3x faster
-- same hdd read write (1.6.5 ~32.4sec, 1.6.9 ~10.8 seconds) 3x faster
-- hdd to ssd - 2GB file (1.6.5 ~23.5sec, 1.6.9 ~11.5sec) 2x faster
-- ssd(ntfs) to ssd(exfat) - same 2GB file (1.6.5 ~10.1sec, 1.6.9 ~10.9sec) 8% slower (look beneath!)
-- ssd(exfat) to ssd(ntfs) - same 2GB file (1.6.5 ~6.6sec, 1.6.9 ~6.2sec) 6% faster (look beneath!)
-- same ssd (exFat) read/write (1.6.5 ~12.5sec, 1.6.9 10.8sec) 15% faster
-- same ssd (ntfs) read/write (1.6.5 ~9.5sec, 1.6.9 ~6.9sec) 37% faster
(exFat vs ntfs - cant explain that.. seems, write-rates on exfat here are slow(er) than expected..)
-- todo for 1.7.0
(*) verification of you, if fixes above do well - rewriting for all file-variations.
(*) verticalbanding refinement
(*) some GUI infos (warnings and errors on files that are making problems while dropping)
regards chmee
wish I had a computer to test your new build.
6x performance.. Wow! how did you pull that off?
Quote"You are converting to CinemaDNG RAW. I really don't think it matters what white balance you set it to in Camera or how they are converted out, you can always change the WB before you actually start working on footage.
For in camera WB, its nice to set it somewhere close to what you believe is the right WB when shooting, but only for shooting preferences, the beauty of RAW is that you don't have to care about WB, you can set it to 12000K or 2000K, it doesn't affect the final image."
yes ..thanks kharak
so eager to test it bt im away from home
I did a test for you. 5D3 with 1.2.3.
Appeared to work fine in raw2cdng 1.6.9 Alpha but when I pulled it into AE, error on the 50th frame, as well as in Premiere Pro. I had vertical banding and chroma smoothing on limited to 12bit.
Error in AE when importing
(http://i.imgur.com/F28I0To.jpg)
The 50th Frame
(http://i.imgur.com/YEjNGnz.jpg)
Importing into Premiere Pro did not give me an error through the Media Browser but, this is the 50th Frame
(http://i.imgur.com/aNZ5tcm.jpg)
Here is the debug from raw2cdng 1.6.9 Alpha
------------- 1.6.9 started at 15.01.29 11:31:40 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file F:\DCIM\100EOS5D\M26-2005.MLV.
[drop] ------
[drop] -- file F:\DCIM\100EOS5D\M26-2005.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] filesize counted (426MB)
[createMLVBlocklist] indexing Blocks in F:\DCIM\100EOS5D\M26-2005.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
***** [getMLVAttributes] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
*****
***** [getMLVAttributes] metaData Object:
xResolution=1920
yResolution=1080
frames=123
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=
lostFrames=0
fpsNom=24000
fpsDen=1001
duration=00:05:08
dropFrame=True
fpsString=23.98
stripByteCount=3628800
stripByteCountReal=3628800
modell=Canon EOS 5D Mark III
camId=MAGIC131b4df
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0.69235970250169
wb_G=1
wb_B=0.410092110532639
whiteBalance=3200
whiteBalanceMode=3
whiteBalanceGM=4800
whiteBalanceBA=4800
blackLevelOld=2048
blackLevelNew=2048
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=5.05991352686844
previewPic=
previewFrame=0
errorString=
isMLV=True
DNGHeader=
version=1.6.9
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
deadSensel=System.Collections.Generic.List`1[raw2cdng_v2.point]
*****
***** [getMLVAttributes] fileData Object:
fileName=F:\DCIM\100EOS5D\M26-2005.MLV
fileNameOnly=M26-2005
fileNameShort=M262005H
fileNameNum=262005
tempPath=C:\Users\Chad\AppData\Local\Temp\
sourcePath=F:\DCIM\100EOS5D
parentSourcePath=100EOS5D
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=1/26/2015 8:04:56 PM
modificationTime=1/26/2015 8:05:04 PM
fileSize=426
VIDFBlock=
RAWBlock=
*****
***** [getMLVAttributes] lensData Object:
lens=24-70mmP4I:
focalLength=24
aperture=0
isoValue=6400
shutter=1/128s
*****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 447
[drop] hasAudio. AUDF-List created. Blocks: 27
[drop] VIDF-List created. Blocks: 121
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[drop] VIDF-Blockdata read and created.
[drop] adding M26-2005 to batchList
[showPicture] started
[drop] calculating verticalBandingCoeefs and searching hot/dead-pixels..
[drop] NO dead sensels found
[drop] verticalBanding enabled. decision: do it
[drop] verticalBanding CoEffs: 1 0.998021254016585 0.999951270393377 1.01021137184845 0.996759966744552 0.997290595487447 0.994221241876036 0.982397019246451 1
[drop] ** Item M26-2005 imported | MLV
[drop] * res 1920x1080px | 23.98fps | 121 frames | BL2048 | WL15000
[drop] * modell Canon EOS 5D Mark III | vidfblocks 121 | has audio
[batchList_Click] showPicture
[showPicture] started
[batchList_Click] read Data from M26-2005 frame 0
[batchList_Click] * M26-2005
[previewTimer_Tick] show previewframe 15 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 29 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 26 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 29 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 33 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 25 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 18 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 16 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 18 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 19 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 23 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 25 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 28 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 34 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 41 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 45 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 50 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 53 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 56 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 60 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 66 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 70 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 74 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 76 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 77 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 78 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 79 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 80 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 82 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 84 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 86 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 87 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 90 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 92 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 94 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 96 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 98 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 100 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 101 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 102 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 103 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 104 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 106 from M26-2005
[showPicture] started
[previewTimer_Tick] show previewframe 107 from M26-2005
[showPicture] started
[convert_Click] all in all there are 121 frames to convert
[doWork] started
[doWork] 12 Cores used
[doWork] -> converting item M26-2005
[doWork] settings format 4 with BL0 WL4095 with+ maximizingvalue 5
[doWork] prefix - filename generator [F]
[doWork] destinationPath -> M26-2005
[doWork] outputFilename -> M26-2005
[doWork] Directory C:\Users\Chad\Desktop\M26-2005 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
[doWork] frameconvert Done
[saveAudio] started
***** [saveAudio] audioData Object:
hasAudio=True
audioFile=
audioSamplingRate=48000
audioBitsPerSample=16
audioFormat=1
audioChannels=2
audioBytesPerSecond=192000
audioBlockAlign=4
*****
[saveAudio] bextName: Canon EOS 5D Mark III
[saveAudio] UDI: DECanMAGIC131B4DF200504714205397
[saveAudio] datetime: 2015:01:2620:05:04
[saveAudio] timeRefMultiplier: 1
[saveAudio] time2frame: 72304
[saveAudio] timeRef (samplingRate*time2frame): 3470592000
[saveAudio] fpsString: 24000/1001
[saveAudio] DF: DF
[saveAudio] subChunkSize :1010084
[saveAudio] success
[doWork] audioconvert Done -> M26-2005.wav
@ChadMuffin
thanks. havent tested yet, if they re ok inside the apps :)
Hi, chmee great job on the superfastification! :D They work great.
Crash reports: I have this two 4GB+ MLVs that kept crashing. I tried slicing them down to 55mb to send to you but they don't crash. Size is exactly 4,293,845,600 bytes on both. It's one of them multiple files MLV.
Also, on the thumbnail preview: it seems the highlights are clipped. I exposed it to the right and almost maxed out the histogram and the thumbnails are almost entirely white. (these are videos of the sky with clouds etc).
hi chmee
great tool!
didn't find your answer on question - can we get bad/hot pixel fix any time soon? :)
thanx
ps: with 6d dng's and vertical banding on with 1.6.5 i was getting some weird super "oversharp" on the edges with some (not all) sequences, so decided to turn it off. do you need these files to check what's wrong?
Converted footage with 165 and 169
Can't say how much.. But 169 is MUCH faster!
First time I enjoy watching MLV being converted ;)
Thanks Chmee.
@ChadMuffin
it seems, theres simply no frame 50.
(1) is this the only one that lacks? or lacks every 50s?
(2) does it happen on every mlv?
while im rewriting the read-functions i just found more potential pitfalls, where the chronology could be wrong, the offset could change.. so please be patient.
regards chmee
Had a file I converted with raw2cdng 1.6.9. It had a missing frame at around count 100 and one right before the shot ends.
I thought it was the MLV file but as I saw you mentioning it might be the conversion, I tried converting the same file with mlvmystic and the frames were not missing.
Its not at an interval. The shot is 4000+ frames and its only frame 101 and 4000(something).
actually i'm implementing some kind of diagnostics on rightclick, because i ve got as well one file it seems to lose one frame somewhere in the middle..
regards chmee
Hey chmee, please, don't leave out spanned MLVs, when(ever) possible :)))
Cool new update though!!!
its already implemented :) actually i finish the diagnostics and a small bug in raw-spanned files.
Hi Chmee. What is your solution for handling spanned raw files. Curious.
spanned files (raw and mlv) are supported a long time. i just found a bug in my new implementation. i dont know, how you mean that question.
(a) mlv is easy. generate list of VIDFs with file-offset, frame-number and spanfile-Number. frames are complete in one file.
(b) raw is quite the same while generating this list. caution: frames can be splitted. thus theres a delta-offset in spanned files.
i hope, today i will throw a new source on bitbucket.
regards chmee
Yea, I,m on mac so havn,t seen how extracting spanned files is done. Mlrawviewever and another app for mac here. Would you have any clue how to get it done in unix perhaps. Long shot I know but you seem to know your stuff.
WOuld you also know how white balance is recorded to MLV files? I tried setting K(kelvin) on my cam(5d mark 3) But when viewing in exiftool I get Asshotneutral. Sorry if this is a little of topic.
(a) as i know mlrawviewer exists for linux as well.
(b) colortemperatur/whitebalance isnt recorded in raw, theres a static entry for around 5600°K.
(c) whitebalance-data in mlv are recorded only as presetnumber or -if manual- as kelvin-value, but there arent whitebalance data. so raw2cdng has the ability to reconctruct wb-data from this fields. (see here (https://bitbucket.org/chmee/raw2cdng/src/e62afbaf5fc524f884b813e1f2d77510b38c4b39/csharp2010_code/raw2cdng_v2/dng.cs?at=master))
regards chmee
Thanks. Found the mlv metadata, noticed the kelvin and rgb numbers. Having a hard time applying wb multipliers to dcraw to get correct wb from mlv, now i see why.
Impressive work on your windows application by the way.
raw2cdng 1.7.0 is out - download (http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/) - sourcecode (https://bitbucket.org/chmee/raw2cdng)
* treat it as a beta :)
* massive speed up on read is working on raw and mlv, single and multifile
Quote
(tested with 1.6.9.alpha)
-- cf-card(usb3.0) to ssd - 2GB file (1.6.5 ~3min, 1.6.9 ~30seconds) 6x faster
-- hdd to hdd - 2GB file (1.6.5 ~34.9sec, 1.6.9 ~9.5sec) more than 3x faster
-- same hdd read write (1.6.5 ~32.4sec, 1.6.9 ~10.8 seconds) 3x faster
-- hdd to ssd - 2GB file (1.6.5 ~23.5sec, 1.6.9 ~11.5sec) 2x faster
-- ssd(ntfs) to ssd(exfat) - same 2GB file (1.6.5 ~10.1sec, 1.6.9 ~10.9sec) 8% slower (look beneath!)
-- ssd(exfat) to ssd(ntfs) - same 2GB file (1.6.5 ~6.6sec, 1.6.9 ~6.2sec) 6% faster (look beneath!)
-- same ssd (exFat) read/write (1.6.5 ~12.5sec, 1.6.9 10.8sec) 15% faster
-- same ssd (ntfs) read/write (1.6.5 ~9.5sec, 1.6.9 ~6.9sec) 37% faster
(exFat vs ntfs - cant explain that.. seems, write-rates on exfat here are slow(er) than expected..)
* verticalbanding only worked well with 16bit non maximized - now 12bit and 16bit maximized too.
* modelname, if not found/wrong in mlv, will be recreated from colormatrix-values @Walter_Schulz
* fps-tag - dropped frames are now well-formed (25000/1001 instead of 24975/1000)
* please test and tell
regards chmee
Testing it right now and it's as good as it gets!!!
so eager to test but i wont be able to test until next month since im away from home
cool stuff.
I tried converting a mlv from a 500D - looking at your code I would assume a colormatrix for the 500D is missing.
Excerpts from the log:
------------- 1.7.0 started at 15.03.09 08:18:57 --
[init][settings] -- ffmpeg Exists: false
[createMLVBlocklist] indexing Blocks in C:\Users\Peter Banane\Desktop\video\janarashtest\M09-0943.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] [!] modelName unknown - estimating from colormtrix
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
[getMLVAttrib] [!] NO WAVI Block
***** [getMLVAttributes] metaData Object:
xResolution=1536
yResolution=864
frames=65
bitsperSample=14
colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=
modell=Canon EOS 500D
camId=MAGIC29cc4ec
apiVersion=0
thank you for info. you could send me a super short (max 100mb) mlv-file to
[email protected]
Mail with a short sample is on the way.
BUG: 1.7.0
The 12th frame of the video being exported drops on 1.7.0. I tried deleting the settings file and it does not work. Reverting back to 1.6.1 corrects the issue and I can confirm there is a non-blank frame in the position. I tried deleting the file again after switching versions and it is still skipping the 12th frame on every different video I export.
Going to try 1.6.5 and see how that works since it seems like many have used it and liked it. The 1.6.1 has just been so rock solid for me!
ok, thanks. will re-check framenumbering and sort-order of raw-sequences.
Quote from: chmee on March 18, 2015, 09:38:58 AM
ok, thanks. will re-check framenumbering and sort-order of raw-sequences.
I've same issue with 1.7, frames drop. Was fine with 1.6.5.
i just had 2 mentions before about that. in 1.7.0 i read and convert in chunks of 50 frames, thus it could be logical it's got some hickups on beginning or on end of that data-chunk, but 12 is quite strange for me. nonetheless, give me some days to analyze and patch it.
as long as you've got those issues, please use older versions you count as stable.
Quote from: chmee on March 18, 2015, 11:56:25 AM
i just had 2 mentions before about that. in 1.7.0 i read and convert in chunks of 50 frames, thus it could be logical it's got some hickups on beginning or on end of that data-chunk, but 12 is quite strange for me. nonetheless, give me some days to analyze and patch it.
as long as you've got those issues, please use older versions you count as stable.
for me it was quite random was 24 and 65 in different MLVs, didndt try more coz of this bug.
Pulled out the old pc so I could try out this app, great work chmee! I definitely like the jpg proxy feature. Now I just need to get Windows XP to read these damn dng files.
i hav premiere resolve round triping problem if the footage is shot in 23.97. resolve keep crashing while trying to import xml. everything works fine if i shoot in 25 fps
thanks for info. will investigate. damn dropped frames :)
v1.7.1 online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
* fixed 500D support
* inputfield above the preview. write a frame-no into it, it will show you the data to the VIDF-chunk. (if debug is enabled, it will be saved into the debug-file)
* possibly framelost fixed
* still open:
* framelost fix not guaranteed, therefore this frame analyzer.
Hey chmee,
i'll give you some feedback... err i have some problem:
I made a FRSP-Timelapse with an outdated built which recorded 1 MLV File (4 GB) and 99 single picture files (.M00 to .M99), which was state of the art back then.
When i open the mlv in raw2cdng it sees 150 frames in 22 mlv-files resulting in dng-files counting from 0 - 128, 133 - 149 and 249 - 252. (see pictures below)
(https://dl.dropboxusercontent.com/u/90483500/MAGICLANTERN/M99files.jpg)
(https://dl.dropboxusercontent.com/u/90483500/MAGICLANTERN/TL-cdngs.jpg)
debug.log:
https://dl.dropboxusercontent.com/u/90483500/MAGICLANTERN/raw2cdng.2.debug.log (https://dl.dropboxusercontent.com/u/90483500/MAGICLANTERN/raw2cdng.2.debug.log)
Do you see a chance to get this working?
Thanks in advance for your generous help and keep up the good work. :)
@Glubber
thanx a lot for this insight. please look with 1.7.1 inside the frame-metadata 128, 129, 130. There should be a jump of framenumbers.. Reason: For MLV i take the framenumbers, that are written into the metadata and i dont renumber them while converting - contrary to RAW where i use an own counting.
this could be the key for "missing" frames.
edit: this 4GB- MLV should have a lot more frames, isnt it? move the files m01-m99 temporarily into another folder and do a second test.
edit2: until m99? fk. i restricted to somewhere m20. patch will come.
I might add that I tried converting FRSP MLV too, it had one MLV and three m.00 files.
But 1.7.1 crashes almost immediately. Also when only trying with the MLV file alone which contains around 100 pics. 1.6.5 was able to convert only the MLV file once, but if I put all m00 files together it also crashes immediately.
stil no luck with round triping
Thank you for your quick response chmee!
I'm afraid i don't understand " look with 1.7.1 inside the frame-metadata 128, 129, 130."
I just get:
[previewTimer_Tick] show previewframe 130 from 23550017
[showPicture] started
Clicking "show meta blocks" results in the log-file i posted.
When looking at the log-file i see no inconsistend numbering outside 0 to 150.
If i test the big mlv-file alone it shows 129 frames. -> 129*31 mb = 3999 ~ 4GB. So the math seem right.
Again many thanks for your efforts!
i really appreciate it.
above the preview-window, on the right side, there is an inputfield (filled with 12345?!). write the frames around the "glitches" into this field. test some frames, that seems not to exist.(129-133 or 150-248). but there will be another patch for longer strips until m99, by now i assume, some of these pictures are inside the splitfiles beyond m20.
(http://dl.phreekz.de/raw2cdng/raw2cdng_171_VIDFdata.jpg)
edit:
QuoteIf i test the big mlv-file alone it shows 129 frames. -> 129*31 mb = 3999 ~ 4GB. So the math seem right
yeah. i forgot, its frsp.
regards chmee
Hello chmee,
another issue that's puzzling me is the metadata of mlv-files (regular video, not FRSP):
In the raw2cdng app i see all the metadata like shutterspeed, fps, focal-length, iso etc.
But in both Lightroom and exiftoolgui i see nothing except cameramodel, framesize, colormatrix and white- and blacklevel.
Is there no way to write the first mentioned metadata inside the dng or am i missing something?
Thx again
never thought about it, 'cause i assumed its not relevant.
chmee, I'm trying to convert quite a large amount of MLV's (626 files, 1.4TB), but the app keeps crashing when I drop the files and the app starts to load them. I enabled the debug, but it sometimes crashes after loading approx 190 files, the next time after 196, etc. Everytime on another file. Is there anything I can do about it?
I tried to load less files (approx 180), it loads OK but after I clicked on convert, it crashed:
[convert_Click] all in all there are 76216 frames to convert
[doWork] started
[doWork] 12 Cores used
[doWork] -> converting item M04-1024_k01
[doWork] settings format 1 with BL2048 WL15000 with+out maximizingvalue 5,05983631871526
[doWork] prefix - filename generator [F]
[doWork] destinationPath -> M04-1024_k01
[doWork] outputFilename -> M04-1024_k01
[doWork] Directory F:\RAW VIDEO\2015-04-04 Sevara Bobur\cdng\M04-1024_k01 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert
btw, I tested older releases -- raw2cdng.1.5.0.beta7 loads all files ok, from the 1.6.0 version it starts to crash. maybe it is something related to the thumbnail generator?
Quote from: chmee on April 06, 2015, 11:28:04 AM
above the preview-window, on the right side, there is an inputfield (filled with 12345?!). write the frames around the "glitches" into this field. test some frames, that seems not to exist.(129-133 or 150-248). but there will be another patch for longer strips until m99, by now i assume, some of these pictures are inside the splitfiles beyond m20.
Aaah now i understand :)
Ok the results are:
nanual frame file nbr frame nbr
128 0 128
129 5 133
130 6 134
133 9 137
130 10 138
147 2 250
148 3 251
149 4 252
so indeed it's skipping some files.
About the metadata:
You're right, as a filmmaker you're ought to know your FPS ;)
I'm still a rookie, so i try to analize my f-stop and shutterspeed when i overexposed etc.
In my case its also helpful for timelapses with exposure-jumps.
Dual-ISO raw video would need the base ISO for deinterlacing.
Thank you once again.
@kaco @all
open while using raw2cdng the taskmanager (or the ressourcemanager) and look on ram. i assume it exceds 1,5GB and then it crashes - .net is(was?) not able to allocate more than this. and of course the thumbnails/histogramm-pictures allocate ram as well. this is the reason as well, why FRSP crashes. it needs now significantly more ram per frame - so already coded something to minimize the ability to crash :) will put that online in the german evening :)
Glad I am currently in Austria then ;)
v1.7.2 is online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
* did some tests with FSRP, worked for me, more checks needed. (if it crashes, try first to convert to 16bit)
* 70D color matrix added
* splitfiles can be more than only 20 (now until .m99 .r99)
regards chmee
I still get framedrops, how can I fix that?
@chmee, you were right, the last value I saw in task manager was 1556 Mb, then it crashed.
Tried the 1.7.2. -- still not able to load all the files. I loaded less files, allocated approx 1GB of ram, but after I hit convert (16 bit) it crashed.
So for now, I have to stay with the 1.5.x version, since I work with tons of data many times a year.
Tried same FRSP sequence that didn't work on 1.7.1 and now it works.
1 MLV file and 3 m00.
Haven't come to make any longer timelapses than that at the moment, but I will do some later today and see how it handles more.
@kaco
1.7.3 is online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
* more ram usage should be possible now.
regards chmee
You are an updating machine, Chmee.
Downloading.. :)
@chmee, getting better results, but still crashing after filling more than approx 2000 Mb of RAM
raw2cdng.1.7.3 and raw2cdng.1.7.2 Not work with RAW from 70D.
Crash then start convert.
raw2cdng.1.6.5 was OK.
need a short RAW (<50MB) from 500D and 70D to fix it.
regards chmee
Quote from: chmee on April 08, 2015, 10:58:56 AM
need a short RAW (<50MB) from 500D and 70D to fix it.
regards chmee
https://yadi.sk/d/_HIfLxdZfrd6Z
v1.7.4 online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
* :) the crash came generally on raw - my fault, thanks for the hint. fixed.
regards chmee
Hey chmee,
I was wondering if there is an easy way of adding some grade/LUT to the Program.
As I'm filming most of my stuff on a 5Dmk2 and purchased VisionLOG's Osiris a while ago.
At the Moment I have to batch convert all of those beautiful cDNG's your Programm provides,
in ACR with "VisonLOG" straight to TIF. (while doing this, I loose raw benefits in Resolve)
http://www.vision-color.com/wp-content/uploads/2015/01/VisionLOG-for-Adobe-Camera-RAW.zip
Maybe we can find a better workflow, beside using BMD Film LUT's.
Feel free to add me on Skype, wir sprechen ja die gleiche Sprache ;)
//trashnart
Cheers, Jan
ps: I just realized, that I follow ML raw for like 2 years now, and didn't have a account yet...
idea is not bad :) before that i have to solve the actual bug of leaving out some frames while converting.
@jankrueck
welcome :) geht auch whatsapp? oder threema?
@all
is this problem persistent? if you convert the file more times, are there every time the same frames, that are missing? i have to find out if its an multi-threading error or wrong filenumber-readout.
regards chmee
Hi Chmee! I, m curious if you, re app is adding specific color matrices for RAW files and if so how do you find out information about what camera has been used? I can, t see any camera related data with exiftool in the extracted dng files?
@chmee
Thanks for your efforts in making your app compatible with FRSP mlv's. Today I have tested v1.7.4 with single and multi-spanned files.
All processed without error and without dropping any frames. A few things:
Is it possible to change the number of threads used into a user definable quantity or is there an ini file that can be modified? I'm not in a position to upgrade my notebook and it works quite hard with the default 4 threads.
MLV_Dump has the ability to change the frame rate via the -A parameter. Adding this feature to raw2cdng would eliminate the need to do this as a separate workflow step when working with FRSP or any timelapse MLVs. Just a feature suggestion.
Lastly, the following images are the same image and frame using both raw2cdng and MLV_Dump apps. On the image extracted via raw2cdng there is a black border an the left and top of the image. This does not occur in the MLV_Dump extractions.
A close visual inspection of both images suggests that the image is there in its entirety within the raw2cng image. Ideally, it would be better to have the DNG created without the black borders and thereby eliminate the need to crop and possibly stretch each frame in post.
Using raw2cdng
https://www.dropbox.com/s/gdrizkeyqtk7y87/5489000000005.dng?dl=0
Using MLV_Dump
https://www.dropbox.com/s/1eab8emt0c73c3x/5489_000005.dng?dl=0
Just some feedback and thoughts. Thanks again for your ongoing support and development of this app.
Cheers.... Otto
@Danne
- MLV, according to modelname it writes the colormatrix1/2 and forwardmatrix1/2.
- on RAW its possible to define the body with the colormatrix provided inside RAW - i didnt implemented all bodies, but it works ( search for setModellOnColorMatrix in io.cs (https://bitbucket.org/chmee/raw2cdng/src/ef55e16df51fc66249f2215d0fffe269f0807766/csharp2013_code/raw2cdng_v2/io.cs?at=master))
@Ottoga
(*) threads - ok, will do something :) but there have to be at least 2.
(*) framerate - doable as well - will think about handy GUI-implementation
(*) black border on frsp - ok, not that difficult as well, but have to inspect, if real picturesize is written/implemented in MLV.
regards chmee
Nice! Will check out. Got matrices implemented in mlv already so this will be really good for raw.
Thanks
Thanks! setModell opens up a lot!
hey chmee,
Hab keine Messenger aufm dem Handy^^
@chmee.... Thanks
hi, chmee... tried 1.7.4 - keeps randomly drop frames, usually between 17 and 22 frames (1 frame drops).
same mlv with 1.6.5 works fine
will be back home ~21.04. (actually in shanghai)
@KelvinK
"randomly" means -> same MLV, more converting trials - every time another dropped frames?
@all
and please tell me, if verticalbanding-fix is working accurate in 1.7.4 or not - cause i read in another thread, 1.6.5 was the best until now..
regards chmee
chmee, no. randomly i mean: different mlvs drop random frmaes, like mlv1 - its 21 frame, mlv2 - 19, mlv3 has no drops
and since 1.7 conversion with chroma and banding on works slower, even progress bars act different
like if put 1 mlv file in 1.6.5 you can see both bars going simultaneously, in 1.7.4 lower bar goes slow when upper bar passing twice (with one mlv file) till lower (overall) bar reaches end
@KelvinK
please do me a favor, convert the same "problematic" file more than two times - just to inspect, if these dropped frames vary.
regards chmee
Ok, chmee, will do it. Actaully, I can upload (but next week only) some problematic MLVs for you, if you need to inspect it.
:) yes please.
Does anyone here have any experience with CDNG in Speedgrade?
CinemaDNG has a strong green cast in Speedgrade. This might have something to do with blacklevel, but I am not sure.. Looks a little like the ffmpg player when it importing ML Raw. And what I read in some threads on adobe, it is a known thing.. And I read that I should "counter colour grade the green".. Sounds very unintuitive to me.. But I have tried, making LogPrimaries and what not.. Basically just trying out different keywords that I read about here and there. But I can't get it to Normally normal.. Not at all as ACR does.
Would be nice if someone had a good pointer to getting the CDNG looking normal in Speedgrade, I would really like to try out some CDNG grading in Speedgrade.
The CDNG's I used were 12 bit maximized. MK III.
@Kharak - Try adding the inverse of the D65 5D3 ColorMatrix1 on a daylight shot then white balance it. This should sort out the green problem (it will require different matrices for different color temps). I'm working on a Speedgrade solution.
Ok thank you Andy, I will try that.
many entries before (nov2013 (http://www.magiclantern.fm/forum/index.php?topic=5618.msg89880#msg89880)) i talked a bit about speedgrade and handling colormatrices..
regards chmee
@Chmee.
I copied the code and made a cube file.
But frankly I don't know how to import luts into Speedgrade.
Tried your way too Andy, but I can't find the D65 inverse of the 5D MK III etc.. or what I am actually supposed to do.
I added fxInversion, but I don't think this is what you mean.. Mind you, I am a newcomer to Speedgrade, this is my 5th attempt at this software. I don't know what has changed in CC compared to CS6 but I do like it better now. But I still find it troublesome to navigate..
Need to find some tutorial to get me through the essentials.
So far I got the best results adding Gamma 2.2.ilut to the footage, still has a green cast then but the luminance seems to be at a normal level, but yeah, it ain't good.
@Kharak - I'm not a daily SpeedGrade user (actually never colored with it) so I think I over simplified things earlier regarding the matrix inversion. This might be part of a solution but not the full solution. I'll work out how to get MLV to display color and gamma correctly in SpeedGrade and build a profile for the 5D3. Fortunately, the Iridas format makes it easy to get technical :)
For reference, if you want to invert a color matrix you will need to find the D65 colormatrix in one of your DNG files using raw digger or something else that can read metadata. Then invert it - a good online matrix tool is http://www.bluebit.gr/matrix-calculator/ (http://www.bluebit.gr/matrix-calculator/)
The 5D Mark III D65 color matrix ML uses is:
0.672200 -0.063500 -0.096300
-0.428700 1.246000 0.202800
-0.090800 0.216200 0.566800
so the inverse of this would be:
1.543930 0.035363 0.249663
0.523447 0.867682 -0.221521
0.047670 -0.325303 1.888783
If my understanding of SpeedGrade is correct, this would give you the 5D3's linear raw color but would still require white balance and a color correction matrix + gamma. It needs a proper calibration profile - I'm working on a test profile for 5D3 MLV in SG and will make the rec709 version available for everyone.
here's a picture from the past, where you see the lut used inside sg - you have to put this file into a specific folder. wait.
C:\Program Files\Adobe\Adobe SpeedGrade CC 2014\settings\looks\CinemaDNG
ah, and they're named -> [name].look
try the values from andy600 - my values were just tests long ago..
(http://dl.phreekz.de/raw2cdng/5DIII_Greencast_Example.jpg)
regards chmee
renamed it to look and it sees it, but it says "invalid preset" before the name.
Maybe it has something to do with under format defaults tab (On your screenshot, that would be FormatStandards), I chose Alexa at one point, it was standard on CinemaDNG, I could choose Alexa and R3D. But now I can't choose CinemaDNG again, I can choose R3D or Alexa and it is standard on Alexa.
Same under Clip and Format Options..
Doesn't matter what footage I import, it keeps saying Alexa now.
Not sure if I wanna continue with Speedgrade at this point... Its not fun anymore :) hehe
@Kharak
in my old Versions 1.4.0 and 1.4.8 there are .look files (http://www.phreekz.de/wordpress/2013/06/magiclantern-raw2cdng-cinema-dng/) as well. try these.
Thanks, these .look files work.
With the look file from 1.4.8. The colours looked quite normal, but the image gets heavily overexposed.
Adjusting exposure in Colour Space Defaults and Blue channel gets it close to the way Windows shows my DNG's.
But its not quite normal to grade, to me it seems.. Perhaps it's the program that is that way, but seems like highlights and shadows act up very strange.. Their levels also seem to be slightly off all the time.
Definitely not the same as grading with After Effects.
@Kharak
@DFM wrote some good points about sg and the rest (2013) (http://www.magiclantern.fm/forum/index.php?topic=6021.msg76056#msg76056). Do not forget, these .look-files are from 2013, first trials to get working inside speedgrade, yes there's a lot of space to optimize them.
got a pm by @DAP , will discuss it here
Quote..just wondering what happened to the highlights fix button in 1.7.4 I am still getting pink highlights in premiere 8 cc with 16bit maximized output from raw2cdng via 5D3
i freezed the highlightfix, because it changes the rawdata, and i want to change less as possible on the sensordata - there's a fix inside of premiere (https://forums.adobe.com/thread/1500841). this problem affects other cameras/recorder as well.
regards chmee
sorry, wrong topic
Hey, i just tried to use today the raw2cdng app, by the way, thanks chmee!! :)
As i click on it i receive this error message saying that i need to install one of the following version of .NET frameworks v4.0.30319, i checked on my PC and the .NET frameworks version i have got is 4.0.4etc.
Anyone having this problem? I am running windows 7 64bit on my machine. thanks in advance
Installing Eos utilty 3 solved this problem :)
:) It wont be about eos-utility, but maybe about .net 4.5.
on win8-systems this problem shouldnt occur, because both (4.0 and 4.5) are installed already.
correct Chmee! I noticed the installer of EOS Utility 3 installing the .NET 4.5 files. now i got your splendid program working almost fine on my PC :)
I am saying Almost because something quite curious happened. basically, i tried your program yesterday night converting some footage i already converted with Magic Lantern RAW video converter 1.9.1
when i converted some shots using raw2cdng version 1.7.4 and 1.6.5 the resulting footage was all Green. The same footage, converted with Magic Lantern RAW Video converter 1.9.1 was looking fine.
The first time i converted the files, i only selected the option cdng 16bit.
i set the folder where to export them, but instead of exporting the footage in the folder i created, it exported them in the same folder as where raw2cdng was.
I just exported the same footage using the option 16bit maximized and the footage is looking fine.
Probably you already know about this "issue" (if i can call it a real issue) but i thought i would let you know. :)
Thanks again! Will donate soon! ;)
So Chmee,
I'm back home and ready to talk about some "Lut-in-baking" on cDNG ;)
Still no Skype? (or was it just because of you, not beeing home)
Cheers, Jan
ps: if no skype, what about fb?
I would like the "start TC" to be always 00:00:00:00 instead of the date of the day (that's what I see on Resolve).
I would love the folder to be named M22-1852_DNG_000001 instead of M22-1852 and the files inside something like M22-1852_000001 instead of M22-1852000001
(first 2 clips generated by raw2cdng, last 2 clips generated by MlRawViewer)
(http://41.media.tumblr.com/4730c25d7f91c55697bdcf2851ea0b60/tumblr_nnkpwma0K81s5o348o1_1280.jpg)
After all, I like raw2cndg for its batchlist properties!
Thanks
tc to 0 is something i could implement, but i dont see any benefit on it.. does it really matter for you? second question is solveable with the filenamegenerator, try [F]_DNG_00001(_)
Come to think of it I can set TC to 00:00:00:00 on Resolve. Everything is ok now.
Thanks
Short Info. With ".net Core" raises the usage of this and other .net-tools to work under mac and linux.
Link [german] - http://t3n.de/news/net-core-fuer-linux-mac-os-607819/
regards chmee
Quote from: chmee on April 16, 2015, 04:33:00 PM
@KelvinK
please do me a favor, convert the same "problematic" file more than two times - just to inspect, if these dropped frames vary.
regards chmee
sorry for delay, was away. tried convert problematic file 3-4 times, dropping same frame.
Hey Chmee,
First, thank you for everything you've done. This program works great at converting my .MLV files. However, I got a question about chroma smoothing. What exactly does it do?
The reason I ask is because I had it on when converting some footage shot on a 5dm3. When I import into AE, some of the highlights are pink. Even with using VISIONlog. I could not for the life of me figure out how to get rid of the pink without damaging the image.
After a day of searching for information on both the Adobe and ML forums, I decided to convert my .MLV files again, this time without chroma smoothing on. Whaaalaa! All of the footage is properly displaying. No more pink highlights.
So what's going on? Can you put it in layman's terms on what's happening? I read somewhere that the pink highlights are from the wrong black levels? But those threads were years ago, not sure what's the most up-to-date info.
I'm using RAW2CDNG 1.7.4
hey chmee,
I just did some raw-shooting. (mlv with sound turned off to be correct)
and when starting to convert, I realized there is some kind of brigtness-shift between 12bit(maximized) and 16bit(maximized)
on raw2cDNG 1.7.4.
Is this intended?
12Bit(max)
https://dl.dropboxusercontent.com/u/26597584/12bit(max)_M14-1620_00000.dng
16Bit(max)
https://dl.dropboxusercontent.com/u/26597584/16bit(max)_M14-1620_00000.dng
16Bit
https://dl.dropboxusercontent.com/u/26597584/16Bit_M14-1620_00000.dng
Cheerio, Jan
@all
sorry for doing nothing atm - i'm really busy.
@jankrueck
will check that. patience please.
actualy I did some new comparison between your conversion
in different modes, mlRawViewer and mlv_mystic.
Because I trie to find the mlv -> cDNG conversion wich "does" quite nothing to my material.
dunno if this is stupid, but I want completly untouched files when I start to edit.
http://i.imgur.com/x6JccSg.jpg
(http://i.imgur.com/x6JccSg.jpg)
I'm just wondering how different colors can exist.
Maybe I understand that whole mlv thing wrong,
but in my understanding it is just a container in wich some dng's are waiting to be let free :D
dng files will be here for download in some minutes.
https://dl.dropboxusercontent.com/u/26597584/compare.zip
Cheers,
Why not just regular 16-bit no smoothing? It works great.
@jankrueck:
In the end everything you see on your monitor is an interpretation of the sensor-data depending on your software (debayering).
I recommend reading the cinelog thread of Andy600. He gives a good insight of colorspace and the whole sience behind it.
It,s a pretty interesting comparison. How is mlrawviewer lossless compression working in these examples? Check the sky next to the flower.
Quote from: Danne on May 16, 2015, 04:43:59 PM
It,s a pretty interesting comparison. How is mlrawviewer lossless compression working in these examples? Check the sky next to the flower.
actualy there is no sky, its blue paper.
What you mean by lostless? dng export is always lostless in my point of view.
The Bottom pictures are all mlrawviewer. Just copy pasted them for comparison.
Quote from: glubber on May 16, 2015, 03:34:13 PM
@jankrueck:
In the end everything you see on your monitor is an interpretation of the sensor-data depending on your software (debayering).
I recommend reading the cinelog thread of Andy600. He gives a good insight of colorspace and the whole sience behind it.
I know how this works, but when you debayer dng's, in my opinion they should be all the same before I start grading.
Also cinelog is not embed into rawfiles. Its just kind of a LUT for makeing h264 files more flat to make grading easier.
This should only affect proxy files, like mov clips.
So if there is no colorspace changes while debayering, all pictures should look the same.
This comparison is a study about the debayering process itself, to ensure there is no lost or adding of information.
Quote from: theartofweb on May 16, 2015, 01:12:44 PM
Why not just regular 16-bit no smoothing? It works great.
Why no smooting? Because reducing noise on this level improves imagequality before adding Luts.
Aaaand like you see there is no difference between smooth files and not smoothed.
16bit (not maximised) is kinda dark, with smoothing and without.
----------------
mlrawviewer looks pretty much the same as I remember the scene looked like. (color & brightness)
Mlrawviewer compresses the dng files. Does it show? Said to be lossless. Could you upload the sample files?
Quote from: jankrueck on May 16, 2015, 08:34:30 PM
So if there is no colorspace changes while debayering, all pictures should look the same.
This comparison is a study about the debayering process itself, to ensure there is no lost or adding of information.
There is always a colorspace change during debayering, b/c raw has no colorspace and you must convert it to one to even be able to view the file. There is also always "lost and added" information during debayering. Debayering itself is a type of interpolation. You cannot view raw data directly, it must be synthesized to some real colorspace before you can even preview it.
There is a lot of metadata in a DNG file that tells the reader/debayer software how to read/interpret the raw file and how to convert it from the native raw "colorspace" to something standard like XYZ (this metadata can effect color as well as brightness and any manner of other image aesthetics). Also various software vary in how they interpret this data and of course debayering algorithms vary in speed/quality/artifacts/etc. Various tools that produce DNGs from MLV,RAW may vary in the particular metadata they put in the DNG file, which may affect the initial "look" of the DNG file when first opened, even if the raw data is the same. When you "edit", "grade", and "develop" the DNG file, you are likely going to be changing some of these parameters themselves, and your image is re-debayered, so even if images start out looking slightly different, you are modifying the parameters that made them look different and there will be no difference in the final output.
Quote from: Danne on May 17, 2015, 01:51:26 PM
Mlrawviewer compresses the dng files. Does it show? Said to be lossless. Could you upload the sample files?
Samplefiles are downloadable in post above
@dmilligan: yeah sure there is some change or lets call it "import" into a xyz-colorspace.
it seems there is some misinterpretation or at least some problem with the metadata embed into the 16bit dng's.
Or lets not call it problem, but if there is a brightness-shift this strong just on 16bit. It seems to be wrong.
Just wanted to tell chmee what I've found ;)
@dmilligan - I couldn't have put it better myself!
To add to that, I have long suspected that the DCRaw matrices, and for that matter, all the Adobe DNG converter matrices that I recently gleaned are really only useful for Adobe apps and more specifically, stills/print processing - the reason being is that Adobe camera calibration allows out-of-gamut color in the color matrix tags which they have to 'manipulate' with forward matrices and HSV tables to get prettier or more 'accurate' and gamut safe color reproduction. FYI, the Forward matrix tags are also important to any .ICC dependent NLE or color grading app but they are not part of the CinemaDNG standard i.e. so we can't rely on them for any gamut 'correction' outside of Adobe apps.
Some of the work we (Cinelog) are doing for forum member @janoschsimon to calibrate his DIY MiniRaw camera, plus several other custom/scientific camera projects we worked on over the last year has highlighted to me that we can do much better to conform Magic Lantern Raw Video color to fit better in relation to the big raw shooting cinema cameras - afterall, Canon sensors have great color sensitivity so let's make the most of it.
I'm planning a 'Magic Lantern Raw Color' project and will publish details of what I want to do, how I will do it and how ML users can help sometime in the next few days. Hopefully a new set of ML color matrices I plan to develop will not only provide better default color that renders identically across all apps but, if/when accepted into ML source code, will allow me some confidence in developing proper ACES IDTs for each camera - at present, it's a crap shoot and unworkable with the single DCRaw matrix and differences between commercial and non-commercial apps. There is little point basing anything on the Adobe matrices for the reasons stated above. The resulting data will be published for any app dev to use under ML licensing!
To avoid any confusion - the DCRaw matrices (which actually come from Adobe labs) and Adobe DNG matrices are not 'bad' or 'wrong', they are simply intended for another purpose - they work fine for most things but IMO, it's time for Magic Lantern Raw Video Color to becomes it's own 'thing' without reliance on incomplete or borrowed data that 'might' have legal implications.
@Andy600 -- great read once again ( along with @dmilligan's from few posts up ;) )
Looking forward to your much anticipated 'Magic Lantern Raw Color' Project and would be more than happy to help contribute and get these colors up to speed!
Keep up the excellent work and I appreciate your hustle on this -- it has definitely paid off on my part in getting used to the workflows around Adobe apps especially in AE (still can't resist from using ACR) so I must say THANK YOU for getting me on board, keeping me eager enough to stick through the bumps on the road and along with your patience I was able to absorb & learn to the fullest without giving up and still yet I'm learning!
8)
QuoteI have long suspected that the DCRaw matrices, and for that matter, all the Adobe DNG converter matrices that I recently gleaned are really only useful for Adobe apps
Thanks for calrification. I,ve been comparing these matrices with dcraw numbers and peeping dng files and came to the same conclusion recently.
QuoteI'm planning a 'Magic Lantern Raw Color' project and will publish details of what I want to do, how I will do it and how ML users can help sometime in the next few days.
How cool is that. Let me know if I can help.
@jankrueck
let raw2cdng write the according whitebalance-data into the dng's.
(*) i assume all but AWB will work.
(*) in the beginning there was the way to shot a CR2 and name it like the raw/mlv-file. raw2cdng takes the whitebalance-data from this cr2 and put it into the dng. i didnt tested it the last versions :)
but finally all was said - the raw-data are nearly untouched - in simple 16 bit the least. its up to you and your app to get out most of it. the metadata inside dng are the slightest problem..
regards chmee
Heads-up, latest Lightworks 12.5.B beta imports cinemaDNGs and LUTs even in the free version: http://www.lwks.com/index.php?option=com_lwks&view=changelog&Itemid=207&id=194 (http://www.lwks.com/index.php?option=com_lwks&view=changelog&Itemid=207&id=194)
Quotein the beginning there was the way to shot a CR2 and name it like the raw/mlv-file. raw2cdng takes the whitebalance-data from this cr2 and put it into the dng. i didnt tested it the last versions
Hi Chmee! Can I ask you what numbers you were using and applying to the dng files coming from CR2 files? I tried some with no success.
exif MeasuredRGGB - calc.getRGGBValues() (line 1287) and calc.convertToFraction() (https://bitbucket.org/chmee/raw2cdng/src/ef55e16df51fc66249f2215d0fffe269f0807766/csharp2013_code/raw2cdng_v2/calc.cs?at=master)
used in io.cs getMLVAttributes line 611 and getRAWAttributes() line 767 (https://bitbucket.org/chmee/raw2cdng/src/ef55e16df51fc66249f2215d0fffe269f0807766/csharp2013_code/raw2cdng_v2/io.cs?at=master)
old discussion with informations (http://www.magiclantern.fm/forum/index.php?topic=6779.0)
regards chmee
Thank. I keep circling. Will take a dive :).
So fiddled with dcraw. When writing dcraw -i -v CR2 it puts out the camera multiplier from the CR2 file. It seems it is almost exact to AsShotNeutral. For instance.
Numbers from CR2 via dcraw
Daylight multipliers: 2.125175 0.943985 1.338680
Camera multipliers: 1722.000000 1024.000000 2032.000000 1024.000000
Dividing by 1024/1722=0.59465737514518 1024/2032=0.50393700787402
I put in following numbers with exiftool
AsShotNeutral=0.59465737514518 1 0.50393700787402
It gives a dng with settings in ACR Temperature 3900 Tint +7
CR2 gives Temperature 4050 Tint +8
Is it possible to get an exact measuring? It is close. I try to follow your links but I am lost ;).
so far this was the simplest way to get "good" (not perfect) body-data. dont forget, you can't count on body-measured data as well, it can differ from cutters opinion of "good" colors.
regards chmee
Thanks. Finally some numbers I could use regarding wb in dng files :).
Hi
(My apologizes if this question is already asked?) I'm using raw2cdng ver 1.7.4 and it always drops 1 frame in every .mlv file conversion. so there is always 1 dng file missed! why is that?
I just read this http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/ and found out its a common issue for many and the solution is in progress... thanks.
For me I see that the dropped frames are 90% of the time within the first 24 frames. Most often frame 17, but 18, 21 and 24 occur very often too. There is some kind of pattern that drops these frames, I don't know what though.
@kharak
For me its completely random. sometime first frames sometimes last frames. i switched the 1.6.5 and its working fine so far.
Anybody else has problems with timecode?
When i import audio and dng sequences in davinci resolve the timecode of audio and video is different by a minute, maybe more maybe less, which makes syncing tedious because i have to fix every audio file timecode manually.
Nobody has this problem? I think 1.5something was working ok on that regard.
Still, latest versions are very fast and stable!
Quote from: Ali Oliya on May 28, 2015, 03:53:06 PM
@kharak
For me its completely random. sometime first frames sometimes last frames. i switched the 1.6.5 and its working fine so far.
I am also using 1.6.5, i noticed that 1.7.x gives me red frames from time to time (randomly), i dont know why because the same MLVs mounted with mlvfs have no dropped frames at all.
@chmee
Hi! Is it possible to add to the raw2cdng a possible correction hot pixels?
Like here in MLVFS Bad pixel correction?
hot pixel in r and b are corrected with chroma smooting 2x2. yes, this is inside.
wow! It is very cool! :D
tnx
Hi for some reason i cant get any version of raw2cdng working it keeps crashing every-time i press convert.. am missing something?
im basically dragging, the MLV/Raw file into the window software so it shows up in the list then press convert and it crashes.
im on a windows 8 pc, using latest build on a Cannon 5D3
if i use MLrawviewer it all works fine
Thanks for this app. It's awesome I'am using 1.6.5 and have no problems at all. Used it for over a year now.
Why DNG files that produced by raw2cdng (12bit & 16bit) are not playable through Resolve media pool?
After dropping it to the timeline playback working ok.
MLVRawViewer DNGs plays good too but they are very small in size, maybe they have a 10bit quality or something?
No, they are just lossless compressed (there is no loss in quality).
Sorry for my absence - as you see i couldnt find any time for the converter - and there's just this one big bug with lost frames while convertion. next time i will embed raw-convertion from xiaomi yi cams as a sidestep..
@Jay429
often its a problem with the filenaming and/or the path-names. did you tried without special or country specific characters (and no whitespaces)?
@chmee
Canon 70D:
-Rec RAW video : afte convert in raw2cdng.1.7.4 ACR identifies *.dng file like Canon 6D
-Rec MLV video : afte convert in raw2cdng.1.7.4 ACR identifies *.dng file like Canon 70D
what do you think, is the problem of the raw2cdng or ML?
70D and 6D shares the same color matrices. Only difference in metadata should be the name of the camera model?
I'm worried about VisonLOG
It works correctly in this case?
When a DNG file from 70D be applicable from the camera profile 6D
If it works for 6D it should behave exactly the same as for 70D I believe.
I just tried it.
Two problems to report:
1- Lens correction can't be applied with ACR.
2- When oppened with Speedgrade, I get a green cast.
The only way I have found since then, is to convert all the DNGs to a 16bit tiff (11mb per frame :-[)
Only then, I don't get a green color shift.
Ps:
I'm also using vision log... Because I'd like to beable to apply luts.
I installed bootcamp just because I thought that this software would help me to get usable DNGs on Speedgrade.. It hasn't. :-\
(*) does other Converter give you the ability for automatic lens-corrections? you can set it manually.
(*) Speedgrade needs manual adjustments. often discussed here, with examples.
Quote..[edit half a year later] speedgrade needs a color-lut/color-matrix to work accordingly with raw-files..
in raw2cdng 1.4.8 (zip) is an example - .look - file....
regards chmee
@chmee: Compared to mlv_dump your converter brings correct WB, but looses lens(model, F-number) shutter speed and ISO :( Yes, I can set lens manually, if I can remember :)
Quote from: chmee on August 27, 2015, 08:33:03 AM
(*) does other Converter give you the ability for automatic lens-corrections? you can set it manually.
It's not that you can't apply lens-corrections, it's just that they are applied wrongly (sometimes) and might not be noticeable b/c ACR makes some assumptions about the FOV. Actually, for crop mode, they will already apply correctly because the pixel size is the same as what ACR assumes.
http://www.magiclantern.fm/forum/index.php?topic=13152.msg153252#msg153252
One problem is that there's no way to handle non-centered footage. If you try to apply lens corrections to non-centered footage, they will be wrong, and there's apparently no tag to specify this to ACR.
Quote from: chmee on August 27, 2015, 08:33:03 AM
(*) Speedgrade needs manual adjustments. often discussed here, with examples.
regards chmee
I found a workaround to get the "right" color to start with in SpeedGrade is to create a Premiere project and add the footage to it first, then do a "direct link" and jump over to SpeedGrade.
You might also be interested in this: http://www.magiclantern.fm/forum/index.php?topic=13152.msg152425#msg152425
QuoteI found a workaround to get the "right" color to start with in SpeedGrade is to create a Premiere project and add the footage to it first, then do a "direct link" and jump over to SpeedGrade.
This is great!
I tried it and I confirm it works 100%. :)
big problem with the latest version. it cuts a half of files. on two parts. without sound.
it isn't known why. and so almost all the time. here through RawFlow. unpacks RawFlow correctly.
(http://s020.radikal.ru/i713/1510/ec/fd809207dd71.jpg)
That latest version looses random frames. It's known bug. Resolve treat it this way. Use 1.6.5 instead.
i never knew that raw2cdng can play mlv raw files. i just drag and drop a mlv file in raw2cdng and it starts playing like a video player
sorry once again i stopped patching raw2cdng. i just had a lot to do and i doubt, i'll find more time for the project. i dont know how many guys are still using raw2cdng :)
@andy_kh :) you're welcome. yes, thats built in since (i think) 1.6.0.
Press Keys:
O - Original Whitebalance
P - Preset Balances
Right/Left Key - change CT manualy
Up/Down - change GM-Shift
Q - change Quality
X or ESC = Exit
You can use raw2cdng as Standard App if doubleclicking .raw or .mlv. (open with..)
regards chmee
Hello again! I used to love this! I can't remember when but this app got a little buggy for my set up at one point but I look forward to trying again soon when I get a chance! Thank you for your continued work!
Hey Chmee! Glad to see you around again, I still run everything through raw2cdng 1.6.5.
Your 1.7.4 version has lightning fast conversion, but the random dropped frames made me revert back to 1.6.5.
But I tried to play mlv files in raw2cdng, but I could not succeed, I pull the Mlv's in and they just queue up for conversion, how exactly do I do it? I use MLRawViewer as default video player for my Mlv's, but would like to try out your player too. Could the problem be that everytime I run raw2cdng, windows always pops up the "The Publisher could not be verified" window and I have to click run.
I can preview in the preview window, but not play it like a video.
@kharak thanks for these infos. it turns into a player, if you drop a file on the app icon or you "open with.."/associate the file-endings (mlv and raw) with raw2cdng. regarding your "publisher"-window, look here, maybe that helps:
http://superuser.com/questions/341091/how-to-get-rid-of-the-publisher-cannot-be-verified-on-network-files
regards chmee
I use version 1.6.5 a lot! It's simple and nice! I always use 16bit maximized and highlight fix, vertical banding and chroma smoothing I let Davinci Resolve debayering solve.
I think the audio sync just fine to Davinci Resolve 11, maybe there can be some small improvment but I don't know.
Thanks again for a great app that I use frequently!
this is my fav app. i never get the chance to use version 1.6.5. never work on my laptop so i hv been using later versions.
Have had some trouble recently with raw2cdng 1.6.5, it doesn't make correct timecode all the time, 1 out of 15 clips don't sync sound properly in Davinci Resolve.
I've tried the 2 problematic clips with MLVFS on mac and they both work fine and sync well on my PC inside Resolve. Maybe someone else having the same problem with audio sync so I highlight it here.
Kind Regards
Karl Allemyr
Hi again,
raw2cdng version 1.6.1 do manage to sync the audio and make the timecode complete of these "problematic" clips.
So I use that app instead.
ah? ok. thanks for info. will hunt/collect those bugs for a final version..
@chmee
Thank you for your software.
I used it today and notice there are missing frames. Ie 00049 to 00051.
When importing to Davinci Resolve 12, the split in file sequence is recognized as a new clip.
I'm using 1.7.4. Should I try a different version?
I noticed one other comment about it here.
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
"mohan
Meint am 19. August 2015 um 18:33
The newest raw2cdng 1.7.4 version is discarding some frames. But with 1.6.1 it renders out perfectly. Haven't tested those in between versions."
I also ran the same MLV through Mystic without the missing frame. To me the dng looked better from Raw2CDNG rather than Mystic, but I also know it can be edited to be the same.
Thank you!
Take another older Version. 1.6.1 someone said has better audiosync for resolve, 1.6.5 is an often used version with the old io-code. after that i made massive improvements on conversion-speed (http://www.magiclantern.fm/forum/index.php?topic=5618.msg141242#msg141242) but couldnt find the missing-frames-bug. so i will do this step to find some bugs and release a last version. but please, be gentle and patient :)
regards chmee
oooh a new version! Hope you can fix the missing frames bug and keep the conversion speed as high as it is in 1.7.4.
I'll be patient!
new version!! sounds kool
Thanks for this great application chmee!
I would be overjoyed if you were able to get the audio timecode to play nicely with Resolve 12 on windows! I have yet to find a converter for windows 10 that spits out audio files that resolve will correctly recognize.
EDIT: I tried version 1.6.1 per your advice, and I found that audio conversion was much better - with v1.6.1, the audio works seamlessly with Resolve 12.1 on windows. When the same clip is converted with 1.7.4, Davinci keeps the audio and video separate (in old versions of Resolve, it would import an empty audio track with the video when you drag the clip onto the timeline).
Whatever you were doing with audio/timecode in 1.6.1 works much better with resolve than the latest version of raw2cdng.
Hope this helps with development of the latest update!
Is there any version ofraw2cdng that converts MLV to cDNG without any corrupt frames.
I use version 1.6.5 and 1.7.4. Which one is best for you folks?
@nlhenry
thanks for info
@ibrahim
raw2cdng cant correct "corrupt" frames. but 1.7.4 has a bug, that frames doesnt exist - it wasnt there in versions until 1.6.5
Quote from: nlhenry1 on December 17, 2015, 01:36:39 AM
EDIT: I tried version 1.6.1 per your advice, and I found that audio conversion was much better - with v1.6.1, the audio works seamlessly with Resolve 12.1 on windows. When the same clip is converted with 1.7.4, Davinci keeps the audio and video separate (in old versions of Resolve, it would import an empty audio track with the video when you drag the clip onto the timeline).
I believe this is due to the versions after 1.6.5 are dropping frames and Resolves reads from dropped frame until next dropped frame and makes a new clip everytime.
Try 1.6.5 and see if you have the same problem. In Premiere the audio is correct (to my eyes and ears).
@Chmee, is 1.7.4 the last version ever? Christmas is now.. you know.. :)
@kharak
santa claus will tell you on 24. :)
I've been a good boy this year!
@all
could someone give me a mlv with the "dropped-frames"-symptom? dropbox or google or onedrive? mail me: raw2cdng[at]phreekz.de
how do you mean dropped frame? When recording or running through 1.7.4 ?
a file that's not trancoded properly with my tool (lost frames). i didnt had this bug with my recordings.
File (https://dl.dropboxusercontent.com/u/76052543/DualISOFlickering/M28-1449_fl.MLV)
Program settings (https://dl.dropboxusercontent.com/u/76052543/RawCDNG174.jpg)
The result (https://dl.dropboxusercontent.com/u/76052543/FolderContents.jpg)
thx
@mothaibaphoto
It transcodes (00-10) 11 Pictures on my PC. There were ~360 Frames in this MLV and the 12. is not complete i assume. So the problem you have is the flickering, right?
regards chmee (btw. merry christmas)
Any other converter transcodes 12 pictures as this is result of running "mlv_dump.exe -f 1-12 " against a big file.
It seems the last picture is omitted, but this is not always the same with 1.7.4 - with big files it can drop random frames.
I believe flickering has nothing to do with this case - I just tried already uploaded file.
thanks for clarification. in this case i think i just left the last frame (generally) and its not under this "lostframe"-symptom. If you say, this bigger MLV has problems in my converter - could you leave me a bigger part, that loses frames somewhere in the middle? maybe about 250 frames? zip'ped?
I tried about couple of dozen files - and every time just last frame was dropped :(
But, I remember for sure, there was something in between.
Then, I found more than 2 gig file which loses № 494 out of 501. And, finally, i found video with lost frame № 128 out of 135. It's a 550 mb mlv and 80 mb m00. I can upload these files tomorrow (after my daily traffic limit renew).
Merry Christmas !!!
UPD: video with lost frame № 128 out of 135 (https://dl.dropboxusercontent.com/u/76052543/M23-1018.zip)
Hi
Whwn I open the mlrawviewer and check the number of frames compared to raw2cdng I notice that many frames are missing after export in raw2cdng compared to mlrawviewer. I use both raw2cdng version 1.7.4 and 1.6.5.
Does this mean that I am getting drop frames in raw2cdng?
@mothaibaphoto
thanks. just downloaded. will test now.
@ibrahim
this bug shouldnt occur with 1.6.1 or 1.6.5.
Thanks chmee ;)
@chmee raw2cdng v.1.7.4 and 1.6.5 have been crashing.
If I load raw2cdng once after a complete system restart or shutdown, it is fine. I can load RAW files into the batch by dragging and dropping and then convert. Works perfectly... One time.
After the files have converted, if I drag and drop another set of files or a single file, raw2cdng crashes. If I relaunch raw2cdng, same crash activity happens.
The debug report contains only this:
[drop] ------
Any ideas? The program was working fine until I installed DaVinci Resolve 11 Lite.
Running Asus G750 - Windows 8.1 - Intel i7 - 32GB RAM
Thanks!
Hi there
@all
good news, missing-frame bug resolved, needs to be tested. but before yelling "new version out" i need to optimize/fix some other things.
(a) audio-glitch - its said, 1.6.1 did good audio-files for resolve, newer ones not, right?
(b) vertical banding. sometimes it was ok sometimes not. give me some more short video-examples to fix it.
@bobberesh
maybe a problem with folder names? try to dragdrop from other place..
regards chmee
waiting for new version
Audio Glitch (Resolve Problem) - could someone tell me if its working with raw2cdng 1.6.3?
1.6.1 - working audio under resolve
1.6.3 - audiolength fixed
1.6.5 - adjusted timeRef
(i dont know what to revert, audiolength or timeRef)
regards chmee
Hello there,
when I convert MLVs shot with my 6D with raw2cdng 1.7.4 I always get one or two red frames flashing towards the beginning of the resulting DNG file. I tried many times with different settings but once I import the DNG in premiere I always find these red frames. I tried converting the MLVs with MLVRawviewer (which I don't like) and I don't get the red frames, but I'd prefer to use raw2cdng. Any help?
Edit: wooops I noticed only now this issue is already being discussed... I thought it wasn't because the search didn't show anything.
Hi,
I notice that raw2cdng 1.6.5 and even 1.7.4 is not exporting my entire mlv file to cDNGs only 1180 out of 2875 frames. My MLV file has .M01 and .M02 associated to it.
Missing .M00
After my recording there were never any M00. How do I solve this?
I have .wav, M01 M02 MLV and .MRX
Load File_man.mo and browse files.
Sorry I don't understand Walter, could you please elaborate what you mean. :)
Modules tab -> file_man.mo ON, will start
Restart cam
Debug tab -> File Manager -> DCIM -> 100CANON
Is there a file with extension M00?
Quote from: Walter Schulz on February 03, 2016, 02:53:56 AM
Modules tab -> file_man.mo ON, will start
Restart cam
Debug tab -> File Manager -> DCIM -> 100CANON
Is there a file with extension M00?
After File manager I get > A:/ or B:/
Choosing A:/ does not lead me to any M00 however:
Choosing B:/ > DCIM > 100EOS5D > find several M00 files (52 B file size).
What do I do with this file?
Your conversion tool doesn't convert M01 (and all following chunks) because M00 is missing ... (hinthinthint)
See reply #1288, too.
Hi. I've just recently started using raw2cdng.
Is it currently stable and safe to use? I've seen there were some issues with missing frames. Cuz it's just TL;DR for me for now. :)
The tool will get an update next days. until now 1.6.5 is recommended.
@walterschulz
a pitty :) thats not fair :) kind of "dontlikethat" /thx
Hello,
I have an issue with Raw2cdng. I can open it, import MLV but when I try to convert IT, I have a "stopped working" message.
In the windows LOG, the issue seems to come from KERNELBASE.dll.
I tried several version of Raw2cdng and I still have the same issue.
I'm on Windows 10 64 bits, and it was working correclty when I was on Windows 8.
Any idea?
1. what are the last lines inside the debug-file?
2. starting as administrator?
1. Here is the entire debug file:
------------- 1.7.4 started at 16.02.11 20:47:07 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file D:\M17-1410.MLV.
[drop] ------
[drop] -- file D:\M17-1410.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] filesize counted (624MB)
[createMLVBlocklist] indexing Blocks in D:\\M17-1410.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
[getMLVAttrib] [!] NO WAVI Block
***** [getMLVAttributes] metaData Object:
xResolution=1824
yResolution=764
frames=268
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=
lostFrames=0
fpsNom=25000
fpsDen=1000
duration=00:10:18
dropFrame=False
fpsString=25.00
stripByteCount=2438688
stripByteCountReal=2438688
modell=Canon EOS 6D
camId=MAGIC3942748
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0.540084388185654
wb_G=1
wb_B=0.547008547008547
whiteBalance=4500
whiteBalanceMode=0
whiteBalanceGM=5200
whiteBalanceBA=5200
blackLevelOld=2047
blackLevelNew=2047
whiteLevelOld=13000
whiteLevelNew=13000
maximize=True
maximizer=5.98338354788642
previewPic=
previewFrame=0
errorString=
isMLV=True
isFRSP=False
DNGHeader=
version=1.7.4
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
deadSensel=System.Collections.Generic.List`1[raw2cdng_v2.point]
*****
***** [getMLVAttributes] fileData Object:
fileName=D:\M17-1410.MLV
fileNameOnly=M17-1410
fileNameShort=M171410G
fileNameNum=171410
tempPath=D:\TEMP\
sourcePath=D:\
parentSourcePath=
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=11/02/2016 20:31:58
modificationTime=17/01/2016 14:10:52
fileSize=624
VIDFBlock=
RAWBlock=
*****
***** [getMLVAttributes] lensData Object:
lens=EF35mm f/2 IS USM2
focalLength=35
aperture=1820
isoValue=110
shutter=1/50s
*****
[drop] MLV Attributes and Blocklist created and sorted. Blocks: 558
[drop] VIDF-List created. Blocks: 268
[readVIDFBlockData] started
[readVIDFBlockData] done: [0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
[readVIDFBlockData] done: [25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40][41][42][43][44][45][46][47][48][49]
[readVIDFBlockData] done: [50][51][52][53][54][55][56][57][58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74]
[readVIDFBlockData] done: [75][76][77][78][79][80][81][82][83][84][85][86][87][88][89][90][91][92][93][94][95][96][97][98][99]
[readVIDFBlockData] done: [100][101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118][119][120][121][122][123][124]
[readVIDFBlockData] done: [125][126][127][128][129][130][131][132][133][134][135][136][137][138][139][140][141][142][143][144][145][146][147][148][149]
[readVIDFBlockData] done: [150][151][152][153][154][155][156][157][158][159][160][161][162][163][164][165][166][167][168][169][170][171][172][173][174]
[readVIDFBlockData] done: [175][176][177][178][179][180][181][182][183][184][185][186][187][188][189][190][191][192][193][194][195][196][197][198][199]
[readVIDFBlockData] done: [200][201][202][203][204][205][206][207][208][209][210][211][212][213][214][215][216][217][218][219][220][221][222][223][224]
[readVIDFBlockData] done: [225][226][227][228][229][230][231][232][233][234][235][236][237][238][239][240][241][242][243][244][245][246][247][248][249]
[drop] VIDF-Blockdata read and created.
[drop] adding M17-1410 to batchList
[showPicture] started
[drop] calculating verticalBandingCoeefs and searching hot/dead-pixels..
[drop] NO dead sensels found
[drop] verticalBanding enabled. decision: not needed
[drop] verticalBanding CoEffs: 1 1.00430185684813 1.00268723386271 1.00576654109846 1.00111753703342 1.00459834482779 1.00234481080323 1.00587428639912 0 13.0811428649475
[drop] ** Item M17-1410 imported | MLV
[drop] * res 1824x764px | 25.00fps | 268 frames | BL2047 | WL13000
[drop] * modell Canon EOS 6D | vidfblocks 268 | has no audio
[convert_Click] all in all there are 268 frames to convert
[doWork] started
[doWork] 4 Cores used
[doWork] -> converting item M17-1410
[doWork] settings format 4 with BL0 WL4095 with+ maximizingvalue 5
[doWork] prefix - filename generator F:\Vidéos\Projets\2015-05 NYC\1-CDNG
[doWork] destinationPath -> F:\Vidéos\Projets\2015-05 NYC\1-CDNG
[doWork] outputFilename -> F:\Vidéos\Projets\2015-05 NYC\1-CDNG
2. I don't understand why, but when I open raw2cdng in Administrator mode, I can't insert MLV file in it. I have a "forbidden" mouse pointer...
It's only in adminstrator mode, it works in normal mode.
Edit: Nevermind, I'm such an idiot...
In reading the last line of the debug log, I reminded I have deleted the output foler...
It works perfectly now !
However, I often have missing frames in DNG output, is there a reason for that?
Hi,
I'm having issues with the lack of M00 for one of my large MLV footage. I have M01, M02 and M03.
I've tried to merge them with its associated .MLV in the Cmd on windows.
I notice that on mlRAwviewer (prior to merging and prior to export) there should be 3596 frames.
However after merging and then exporting the merged mlv on raw2cdng I get 3588 frames. Any idea what's wrong and why I'm not getting 3596 frames instead?
Merging files is contraproductive.
Merging files won't create missing file *.M00
-> I don't really care why there are 8 frames missing.
EDIT: If you wanted to ask if there is a way to convert frames stored in M01, M02 and M03 without M00: Yes, that's possible.
v1.7.5 is out
* please! treat it as a beta! i have no splitted files here (on my laptop).
* main patch - framebug solved (i think)
* toDo: Audiofix for Resolve (at home, then, in ~10days) | MLVlite-Support(need Samples) | tolerance on missing Files (see above M00-Example)
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
regards chmee
oooh snap!
I am testing this right now!
EDIT: So far so good! Converted 3 .MLV files, 2 of them 10 Gig, about 3000 Frames each and another 8.5 Gig and not one frame missing! :)
Good speed also! Very nice! Thank you so much for fixing this Chmee!
.mlv+audio
1920x1038
24 fps
non-Crop
5d3 113
May I request one tiny fix? The Convert button says "Converting" always, its not a big deal, but would be nice if it just said "Convert". It says "Converting" before and after clicking it, so might aswell just be "Convert" :)
Thanks chmee for the new build ,
5D2 MLV+audio (1872x936) 3 spanning file total of 3:41 (4590 frames)
Check and all frames are there no missing frames at this point .
Tested New MLV Lite Support is there for spanning files (check out just only2 spanning files) everything seems to be OK
not missing any frames , but there was a single first frame was total black , it wasn't there in other app thou , not a big deal thou.
In 3xCrop mode the cdng's all have pink cast it's like the black level is off .
Be on that, No crashes very fast at extraction even on my very old Dell Inspiron 1764 i5 Laptop. ;)
Here is raw2cdng_mlv_lite_1856x1044 48-Cdng Frames (https://www.dropbox.com/s/iau8bqy3gwgcd7v/Raw2Cdng_MLVLite_1856x1044.zip?dl=0) there are normal looking and here is
raw2cdng_mlv-lite_1920x1038_3xCrop - 48 frames (https://www.dropbox.com/s/b7k0gfqxib5crao/raw2cdng_mlv-lite_1920x1038_3xCrop.zip?dl=0) they have pink cast
I made a mini MLV for your mlvdiag.1.0.1 tool M23-1525.MLV.55mb.mlv (https://www.dropbox.com/s/p12vut2ivpbaxp5/M23-1525.MLV.55mb.mlv?dl=0) this the 1856x1044 one &
here is the M23-1531.MLV.55mb.mlv (https://www.dropbox.com/s/8zjc3odu4ilgopj/M23-1531.MLV.55mb.mlv?dl=0) for the 3xCrop 1920x1038 , it's the one that produces pink cast over the image .
Quote from: Walter Schulz on February 24, 2016, 04:31:16 PM
...
EDIT: If you wanted to ask if there is a way to convert frames stored in M01, M02 and M03 without M00: Yes, that's possible.
Does this mean that the M00 only contains 8 frames?
No.
Processed a number of std and mlv_lite mlv files through v1.7.5.
All files processed successfully. No lost frames and spot checks of various frames showed no abnormalities.
@reddeercity
will check that when i'm back home. net is awfully slow here in geneva.
@ottoga
thats good news.
@Kharak
will look into it. (because i revert after converting to "convert" but it doesnt..)
thanks for testing. regards chmee
How do I recover the M00? or fix this issue so that all my recordings include the M00.
M00 is not on the card anymore? Do you have reused the card (=writing to card) after M00 got lost?
@ibrahim
rename the last one to M00.. because all VIDF-Chunks (aka frames) have timestamps, its not necessary to hold the files (and frames) in the right order. but no one can recover files, you dont know, if they existed ;)
Quote from: Walter Schulz on February 26, 2016, 04:52:47 PM
M00 is not on the card anymore? Do you have reused the card (=writing to card) after M00 got lost?
I've noticed that M00 have never been recorded on the CF-card all the files (which are long) start with M01.
Does the absence of M00 mean that I'm lacking some frames from my filmed footage? In case, I've missunderstod the issue of lacking M00...
Quote from: chmee on February 26, 2016, 08:10:43 PM
@ibrahim
rename the last one to M00.. because all VIDF-Chunks (aka frames) have timestamps, its not necessary to hold the files (and frames) in the right order. but no one can recover files, you dont know, if they existed ;)
Ok, I'll try that to see if the application exports all the frames.
Quote from: ibrahim on February 28, 2016, 10:55:55 PM
I've noticed that M00 have never been recorded on the CF-card all the files (which are long) start with M01.
Sorry, but that's not what you described earlier. In reply #1293 you located M00 files in the cam. Filemanager does not give correct file sizes for files > 4GB. So I suggest you take a good look again using your cardreader.
the Raw2CDNG stops working as soon as I drop my .RAW file on it.
here is the debug file.
I really need help on this.
PS. I use 5D mark III shot at 1920x1080
it's my first time shooting ML Raw with my 5D.
------------- 1.7.5 started at 59.03.14 09:58:11 --
[init][settings] -- ffmpeg Exists: false
[drop] started
[drop] Files dropped:
[drop] ------
[drop] -- file F:\5D3 Lantern RAW test\M13-2359.RAW.
[drop] ------
[drop] -- file F:\5D3 Lantern RAW test\M13-2359.RAW will be analyzed now.
[drop] is RAW
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[getRAWAttrib] started
[getRAWAttrib] counting Splitfiles
[getRAWAttrib] found 1 Files
[getRAWAttrib] counted 4095 MB
[getRAWAttrib] reading RAW Footer and Attribs
[getRAWAttrib,RAW] deciding model on first colorMatrix-Value
***** [getMLVAttributes] metaData Object:
xResolution=61282
yResolution=11464
frames=-1862980394
bitsperSample=14
bitsperSampleChanged=0
isLog=False
colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=
lostFrames=47369192
fpsNom=1110098057
fpsDen=1000
duration=255:32:01:164
dropFrame=False
fpsString=1110098.00
stripByteCount=1217323566
stripByteCountReal=155697660
modell=Canon EOS 7D
camId=MAGIC199e3d3
apiVersion=0
splitCount=1
photoRAW=False
photoRAWe=False
RGGBValues=System.Int32[]
RGBfraction=System.Int32[]
wb_R=0.540084388185654
wb_G=1
wb_B=0.547008547008547
whiteBalance=4500
whiteBalanceMode=0
whiteBalanceGM=0
whiteBalanceBA=0
blackLevelOld=1745391186
blackLevelNew=1745391186
whiteLevelOld=15000
whiteLevelNew=15000
maximize=True
maximizer=-3.75477793988877E-05
previewPic=
previewFrame=0
errorString=
isMLV=False
isFRSP=False
DNGHeader=
version=1.7.5
versionString=System.Byte[]
propertiesString=
verticalBandingCoeffs=
verticalBandingNeeded=False
deadSensel=System.Collections.Generic.List`1[raw2cdng_v2.point]
*****
***** [getMLVAttributes] fileData Object:
fileName=F:\5D3 Lantern RAW test\M13-2359.RAW
fileNameOnly=M13-2359
fileNameShort=M132359P
fileNameNum=132359
tempPath=C:\Users\asutt\AppData\Local\Temp\
sourcePath=F:\5D3 Lantern RAW test
parentSourcePath=5D3 Lantern RAW test
basePath=
destinationPath=
extraPathInfo=
_changedPath=
outputFilename=
creationTime=14/3/2559 0:02:02
modificationTime=14/3/2559 0:00:14
fileSize=4095
VIDFBlock=
RAWBlock=
*****
***** [getMLVAttributes] lensData Object:
lens=
focalLength=0
aperture=0
isoValue=0
shutter=
*****
[drop] RAW Attributes read and set.
[createRAWBlocklist] started
[drop] RAW Blocklist created and sorted. Blocks: 0
[drop] adding M13-2359 to batchList
[showPicture] started
You used a 5d3 ? but according to the log file it's a 7d
Quotemodell=Canon EOS 7D
Try a older version of raw2cdng , the stable version I use from time to time when I have troubles is 1.6.5 (http://dl.phreekz.de/raw2cdng/raw2cdng.1.6.5.zip)
Oh I just saw it said 7D. That's very weird! [I used the 5D3]
Thank you for pointing that out!
Will try the 1.6.5
Thanks a lot @Reddeercity
Quote from: reddeercity on March 14, 2016, 08:10:02 AM
You used a 5d3 ? but according to the log file it's a 7dTry a older version of raw2cdng , the stable version I use from time to time when I have troubles is 1.6.5 (http://dl.phreekz.de/raw2cdng/raw2cdng.1.6.5.zip)
@Ake A.
Please use MLV or MLVLite. i know, raw is not working well in this version.
@Chmee
Is the Vertical Banding fix disabled (in the latest version or 1.6.5)?
I have some footage with more visible Vertical stripes than normal due to the Shadows having to be pushed more and I can not see any difference with Vertical Banding ON and OFF
Or is the Vertical Banding tab something else than the Vertical Stripe Fix from A1ex?
@Kharak
Please give me some more footage. Generally its the Fix coded by a1ex, but sometimes my code inverts the analyse-decision.
@Chmee,
Sorry for the late reply, I have been busy this weekend with Easter and everything.
It didn't get worse with the Vertical Banding, I did ask you about this in this thread many many pages back. Yes in that case it inverted the Vertical banding making it extremely visible for some reason.
Alas, I think it was the mistake of AE that it interpreted two CinemaDNG sequences as the same, because I had already imported the footage without Vertical Stripe Fix and then converted the same sequence with Vertical Stripe Fix and there was no difference, I assume After Effects saw the Sequences as identical and just re-imported the previous CDNG sequence, because I only changed the name of the folders for my sake to see the difference. I have since separated the two sequences and performed Vertical Stripe Fix on all the footage and imported in to a new composition and everything is A- OKEY!
But if you still require footage I can send you some?
:) easy :)
So, lets put some small footage into the samples thread.
http://www.magiclantern.fm/forum/index.php?topic=11899.0
Sure thing!
But could you post a link or explain to me how it is I "Cut" or reduce a MLV file to a more reasonable size?
@Kharak
Its an option in mlv_dump.
okey, where do I get MLV_Dump?
Searching on forum the does not lead me to a download.
Also some commands ?
Sorry for being so demanding.
@Kharak
A link to mlv_dump can be found here. http://www.magiclantern.fm/forum/index.php?topic=7122.msg59525#msg59525
If you are a Windows or Linux user:
- Copy the relevant binary into your source mlv files directory
- Open a command window (I always do this in administrator mode)
- Change directory to your source MLV files directory
- As a first time user, run mlv_dump without any parameters to display a list of available parameters.
As you are trying to create some cut down versions of your MLV files the following run string should achieve this:
mlv_dump -o output_filename -f [nr of frames] -c -l compression_level(0-9) <inputfile_name>
eg: mlv_dump -o M20-1234_new.mlv -f 50 -c -l 9 m20-1234.mlv
The value of the number of frames in the -f parameter is optional. The more frames that you extract the larger the output file.
The -c and -l parameters are optional but will reduce the size of the output file and as such, the time required to download it for @chmee.
@Kharak , Use chmee's mlvdiag.1.0.1 (http://dl.phreekz.de/raw2cdng/mlvdiag.1.0.1.zip) No need to use cmd line .
You can make a mini MLV file (55Mb) from any size , there a reason why chmee made this program.
thx @reddeercity.
So I took at closer look at the difference between Vertical Fix ON and OFF.
First thing I noticed, (think I've read someone else say this too) is that Vertical Fix ON, makes the DNG's darker than the ones with Vertical Banding off. I have to raise the exposure with about +0.35 EV's to match them.
2nd thing I noticed is that the shadows of the Vertical Fix ON DNG's have a Magenta/purple cast. Which is not so easy to remove. Looking back and forth between Non-Vertical Fix and Fixed DNG's, the Non-vertical fix DNG's look much nicer because the shadows are truly black and overall shadow colour look natural.
Maybe this scene is not the best to illustrate the Vertical Stripes because I have a hard time seeing them. I deleted the other ones I was working with which had clearly visible vertical stripes... Stupid me.
But I will be on a lookout for shots with more visible vertical stripes. I think shooting in to light will emphasize the Vertical Stripes.
oh well:
https://mega.nz/#!REQFSLBC!1uvZCODlMYh2udDqeskf_dj-Uh_E8SQPs83-S52-KkU
5D MK III
MLV+Audio
1920x1038
2016-02-29 Build
113
what are min system requirements for working with ml raw dng files from canon 50d or other video work? I was thinking of using lightroom for grading and editing offline then grading. is that possible. can i get away with core2duo and 4g of ram?
Hello,
I have a problem. Im using 5DMIII , ML 1.2.3 , win 8.1 64x. I recorded 2:13 min/sec video - 3331 frames (shown by MlRawViewer 1.3.3) .
When I downloaded the CF card (SanDisk Extreme PRO, 160 mb/s, 64GB) there were only these files:
M08-2042.MLV (3,99GB)
M08-2042.M01 (3,99GB)
M08-2042.M02(3,27GB)
M08-2042.MRX(74KB)
After searching the forum I think Im missing the *.M00 file.
When Im trying to convert the MLV files through MLRV Converter 1.9.1 - I could get only 1182 frames out of it.
After conversion through MlRawViewer v 1.3.3 - DNG s are lossless Compressed (2MB DNG s instead of 4MB) - I could get only 2520 frames out of the footage.
raw2cdng 1.7.5 - was recognizing and exporting only 1182 frames.
After I renamed the *.M01 file to *.M00 and *.M02 to *.M01 raw2cdng 1.7.5 could recognize all 3331 frames and could export them to 16bit maximized CDNG format.
Questions:
Is this the right way how to do it?
Is there some wrong setting in my 5DMIII camera that does not create the *.M00 file?
Im missing the *.M00 file in about 90% of all footages.
Why is this random creating of *.M00 happening?
Thanx a lot for answer.
Check your SD card, if you have Split CF/SD recording enabled, you're missing m00 file might be on the SD card.
If you play the file in MLRawViewer, do you get random black frames? If so, then that is a good indicator that you have been split recording to the CF and SD card, just copy all the files from the SD and CF card to the same folder and do it again.
And the .mrx file is just a profile for MLrawviewer for each MLV. No need to keep it.
Thanx a lot for a quick answer.
Now I know what to do helped a lot. :)
Hey Guys,
I drop 30 video's into the RAW2CDNG 1.7.4 on windows 10. And click export to a specific folder. But afterwards I see there are no seperate folders created for each file. Offcours Resolve and Premiere are seeing it just as one big file. How can I convert each file to a a separte folder, so I can select sepatere files/video in my NLE.
Thanks!
@chmee
Do you think it's possible for you if you have time to design a new version of raw2cdng with the "bad pixels" fix option ?
I'm asking you this feature request in reference to those links and the new vertical stripes fix :
https://bitbucket.org/hudson/magic-lantern/pull-requests/732/5d3-attempt-to-fix-vertical-stripes-so-it/diff
QuoteSince the bad pixels are easier and faster to correct in postprocessing, what about choosing the image with bad pixels, but without stripes?
http://www.magiclantern.fm/forum/index.php?topic=17021.msg171179#msg171179
Your converter software is for me the more practice and fast in personnal and pro use, hope it will possible !
Thanks you !
@zerocool22
lately.. generally it should export into seperate folders. did you changed the folder/filenaming-pattern?
@GutterPump
i'm very sorry. i hadnt had the time since half a year to patch the code - and i believe, i wont anymore. sorry sorry.
regards chmee
Hi! How can I rename the output file through raw2cdng v.1.6-1.7? Davinchi Resolve converts individual clips as one file and it constantly rewrites. For exemple, i use raw2cdng v.1.4.9.7 - with those converted output names i have no problem.
File: M05-1441
raw2cdng v.1.6-1.7 = M05-151000001
raw2cdng v.1.4.9.7 = M051441H_C0001_B16.000001
I can't undestand - why after "deliver" in Resolve I get only one file - M05-000000000???
Okey Chmee! Thanks again for a great app think RAW2CDNG 1.6.1 is very reliable thanks for that! Us it all the time! Afraid of windows 10 though :)
Hello, a i am young operator from Russia
i really like your software but there is a problem
why are different colors preview raw2cdng and worspace Adobe Premier Pro CC?
https://drive.google.com/file/d/0By3Yfa2Y4BnSYXNBUE1NQzZENWc/view
@snep
i dont see a problem. sorry. i coded a simple previewer. demosaicing and color rendering is not that easy with raw-files. so finally my output does its job not that bad :)
Hi again Chmee,
I'am about to upgrade to Windows 10 Professional from Windows 7 Professional. Don't really want to do that but I have to upgrade/downgrade :-[
Have used your Raw2CDNG app for a really long time now, I think my 5D3 workflow is quite easy to use because of your app.
I use version 1.6.1 since some time back and just for fun tested the speed between that and the 1.7.5 application.
1.6.1: 15 seconds for a bunch of MLV files
1.7.5: 50 seconds for the same bunch of MLV files.
Maybe doing something wrong though.
The settings i always use is only ticking the CDNG 16bit maximized button. I think I don't need "Chroma smoothing 2x2", "no proxy" or "vertical banding" feutures.
I'am not working with cinematography at al just using to making short films just for fun. And yes I can't think of any other camera right now to use then the 5D3 with ML RAW :)
https://vimeo.com/172286896 (https://vimeo.com/172286896)
Any tips for using RAW2CDNG version 1.6.1 on Windows 10 is warmly welcome. How many hours days would it take to code it for Windows 10? I guess that it's quite time consuming.
Best
/ Karl Allemyr
@allemyr
is v1.6.1 not working under win10? will try that for myself.. (but by now im in paris, back home in two days)
@chmee versions 1.4.9 to 1.7.4 work under windows 10 . I tested different PC app's under Windows 10 Here (http://www.magiclantern.fm/forum/index.php?topic=15654.msg152335#msg152335)
Sorry for all the hassel. I did just read in a thread that one had problem with Windows 10 and RAW2CDNG so I thought it was a issue.
RAW2CDNG works like a charm in Windows 10 Pro don't know what the other user did wrong. Was a main thing to check before upgrading to Windows 10.
Ok then, thanks for a great app again, really sorry that I made big thing about it.
i never thought, i ll do something on the code. and i dont know, if someone's still using this tool. Now in v1.7.9 there is 10 and 12bit support. Tested with the samples from @danne (thx)
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
regards chmee
I still have raw2cdng at the core of my workflow. This update is a warm welcome.
If you ever decide to break your own promise again, could you take a look at what happens to the Metadata after conversion?
i'm sure you wrote it once. could you give me a short info, what metadata you want to see inside the dngs?
I find it very useful to have all the lens information, lens model, focal lenght, @focal lenght, @focus, f-stop, iso, shutter speed etc. Its very helpful with older footage.
Quote from: chmee on December 21, 2016, 03:31:28 PM
i'm sure you wrote it once. could you give me a short info, what metadata you want to see inside the dngs?
FPS and Color temperature. P.S. thanks a lot for adding support of 10 12 bit
Wow what a Christmas gift of an update. This came JUST in time for a little holiday personal project! Thank you so much!!!
Yes we are still using raw2cdng. Thank q for the update chmee. Btw my 70D cant shoot 10/12 bit
+1 for metadata. Knowing the ISO, focal length, and aperture would be very helpful and seems to be the only thing missing that I can think of. Thanks again!
@zalbnrum
could you upload/post a small raw-file that shows this 1st-frame-bug?
@chmee
your raw2cdng 1.7.9. works very well, the only thing I missing the automatic Lens identification in ACR so is always one extra step to make the correction settings manual in ACR
Exported *.dng have alway fixed size 2,97 MB (Full HD 10bpp converted to 12bpp).
When I use Adobe DNG Converter 9.8 to exported files, new size is only +-1,5 MB.
Is it possible use more effective Adobe DNG Converter in this tool?
I am trying to import cDNG footage into Adobe Pr after editing them in Lightroom. Why are they not seen as modified after the import?
It is as if I haven't made any changes at all. ???
I have used raw2cdng 1.6.5.
Lightroom is a "non-destructive" editor. When you "edit" in Lr, all you are really doing is changing some metadata used by ACR to "develop" the DNG. You are not changing any image data. Premiere does not use ACR to develop DNG files (b/c ACR is slow and not intended for video) and so therefore all of the ACR metadata you've created in Lr is simply ignored.
Premiere doesn't use ACR to import DNGS as it's expecting cinema DNGs which are different to "photo" DNGs. The metadata fields where the Lightroom adjustments are stored are not recognised in the Cinema DNG format.
Your best bet is to import them as image sequences in After Effects, where ACR will open for each sequence. It will apply the settings of the first file in the sequence to the whole clip. From there you can export to a lossless codec and edit in Premiere. For maximum flexibility use Cinelog DCP to generate 444 log masters, rather than using ACR's controls which are adaptive and not absolute and therefore introduce flickering. So you have to be careful about most of the shadow/highlights/contrast sliders if you want to use ACR.
Alternatively put all the Cinema DNG clips in a sequence in Premiere and do the same in After Effects. Nest that sequence to edit in Prem. When you're done, do the colour correction in After Effects and import that comp into Premiere, placing it on the track above your first sequence. Your edit will now have your colour correction applied.
Neither of these workflows are particularly straightforward, or fast. Another option is to use Resolve to generate log masters, which is pretty fast. Set your timeline to Cineon gamma and Alexa gamut. But definitely don't colour your CDNGs directly in Premiere as its debayering is pretty poor. ACR gives best results but is very slow.
Thanks Hyalinejim.
That might be why I got terrible flickering while rendering from AE as a test sample. I already had a bit of flickering. I managed to find a plugin that removed it in Pr.
I am working on a green screen which I filmed in RAW. Since I use scopes while color correcting in Pr I only intend to color correct in Pr instead of AE. I find it much easier than AE which doesn't seem to have scopes.
For this purpose I wonder if I should first remove green screen, then denoise, then deflicker (flicker free plugin) and then color correct
OR
First denoise, then deflicker (flicker free plugin), remove green screen, and then color correct
OR
Deflicker, denoise, remove green screen and at last color correct
OR
denoise, deflicker, then color correct and at last remove green screen.
I really need an expert advice since I having hard time maintaining the high quality of the footage intact.
I use 5d3 (1.1.3).
Here is a little something which will output a more scene linearish output. Thankfully without flicker. To speed up acr workflow you can export in batches with AE command line options.
http://www.magiclantern.fm/forum/index.php?topic=13512.msg172443#msg172443
I've been using raw2cdng 1.6.5 for almost 2 years now but since last week I not longer can export any MLVs to cDNGs without getting green cast.
What is the reason behind this. I haven't made any changes to the settings on my 5d3 1.1.3.
Since I use 5d3 I select highlightfix and 16 bit nothing else is checked.
The same green cast results from v1.7.4.
Hi
I'm a new user of magic lantern and I did a few test shots of raw 2.0 mlv footage this morning. I've converted them in to cdng files in raw2cdng. But I can't get them into premiere pro! It's been frustrating me all day!
How do you guys do it?
Cheers, Stuart
Quote from: StuartCox on March 05, 2017, 07:20:33 PM
Hi
I'm a new user of magic lantern and I did a few test shots of raw 2.0 mlv footage this morning. I've converted them in to cdng files in raw2cdng. But I can't get them into premiere pro! It's been frustrating me all day!
How do you guys do it?
Cheers, Stuart
It should work with your premiere, which version of Premiere do you use?
Do you get a popup error of some kind or maybe you-re doing drag-and-drop which won't work. You'll have to import the cDNGs internally through the program itself using the browser.
I recommend using after effects if you want to work with cDNGs OR since CC 2015 came out use the proxy function which is great. Then you won't have to go through AE render and then import the prores/cineform/animation quicktime files to Pr for post-editing. Istead everything can be done in premiere with the proxy function. Just make sure to select a significantly lower resolution and choose something like gopro cineform on PC. And make sure you have enabled proxy function in your preferences. There are s of everal tutorials on Youtube on how to use proxies which I recommend for all ML users.
Premiere pro cc supports cdng. Older version wil not support.
Open your cdng folder. You will see those cdng sequence. Click the first top one only, drag and drop it in your premiere pro media folder. It should work.
Warnin: dont select all the sequence or premiere pro will crash
Hello guys, I have an issue with MLVlite files, the highlights of exported cDNG are pink, no issue with RAW files.
How can I fix it? v1.7.9
Thanks.
I tried 1.7.9 First frame is always black after convert to DNG(((
both @JackDaniel412 and @Surin Dmitriy
please check, if its fine with an older version. fi v.1.6.1
http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/
The last working solution without this bug is 1.6.5.
Hello chmee! Love to use raw2cdng!
I'm on canon 600d with 10bit RAW nightbuild.
Raw2cdng sometimes gives me GREEN output.
In about 1/7 of my clips it sets random black level. After 5 days research I found solution in fixing BlackLevel to 2048 in mvl_dump (--black-fix=2048 --white-fix=15000) or to set BlackLevel to 128 in MVLProducer. But i like raw2cdng best and would love to use only it!
I downloaded early raw2cdng ver 1.6.1 - it doesn't support 10bit, but there is HIGHLIGHT FIX box.
Is there a way to put BlackLevel FIX inside? I know I'm not alone with that problem, it would help us all:)
I also think that a play button in preview window would be so practical! - than we would see which take is less shaky and convert only that one.
It would save hours of rendering unwanted clips and another hours of watching them and than deleting from a hard drive:(
I'm sorry for my english;)
I am very grateful for your work -
For the creator of this converter (you are awesome!) and all ML community - thanks for making it possible.
@szafran
it seems, this has to be fixed :) so, give me some days.. (then i will look on firstframe-bug as well. please use all the latest ml-firmware for those errors. maybe it was an early-version-bug inside the ml-firmware)
i have been waiting for this fix
...Metadata..? :)
Hey now, I'm trying to install this thing on my mac (running mavericks) using wineskin winery (granted I know absolutely nothing about what I'm doing...)—I performed a test run, and it produced two .txt files, one indicating that it could read the contents of raw2cdng, and another saying that:
QuoteThe entry point method could not be loaded due to Could not load file or assembly 'PresentationFramework, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. assembly:PresentationFramework, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35 type:<unknown type> member:<none>
Idk how much of this information is needed (if at all), but here's the result of the test:
QuoteWineskinX11: main(): argc=12
Waiting for startup parameters via Mach IPC.
WineskinX11: do_start_x11_server(): argc=12
Attempting to use pixel depth of 24
[258379.097] WineskinX11 starting:
[258379.097] X.Org X Server 1.13.0
[258379.097] Build Date: 20120921
[258379.098] _XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.
[258379.101] Initializing built-in extension Generic Event Extension
[258379.101] Initializing built-in extension SHAPE
[258379.101] Initializing built-in extension MIT-SHM
[258379.101] Initializing built-in extension XInputExtension
[258379.101] Initializing built-in extension XTEST
[258379.101] Initializing built-in extension BIG-REQUESTS
[258379.101] Initializing built-in extension SYNC
[258379.101] Initializing built-in extension XKEYBOARD
[258379.101] Initializing built-in extension XC-MISC
[258379.101] Initializing built-in extension XINERAMA
[258379.101] Initializing built-in extension PseudoramiX
[258379.101] Initializing built-in extension XFIXES
[258379.101] Initializing built-in extension RENDER
[258379.101] Initializing built-in extension RANDR
[258379.101] Initializing built-in extension DAMAGE
[258379.101] Initializing built-in extension MIT-SCREEN-SAVER
[258379.101] Initializing built-in extension DOUBLE-BUFFER
[258379.101] Initializing built-in extension RECORD
[258379.101] Initializing built-in extension X-Resource
[258379.101] Initializing built-in extension XVideo
[258379.101] Initializing built-in extension XVideo-MotionCompensation
[258379.101] Initializing built-in extension GLX
[258379.101] x: 0, y: 0, w: 1280, h: 778
[258379.115] (II) GLX: Initialized Core OpenGL GL provider for screen 0
[258379.115] [dix] Could not init font path element /opt/X11/share/fonts/75dpi, removing from list!
[258379.116] [dix] Could not init font path element /opt/X11/share/fonts/100dpi, removing from list!
[258379.116] [dix] Could not init font path element /opt/X11/share/fonts/cyrillic, removing from list!
[258379.116] [dix] Could not init font path element /opt/X11/share/fonts/misc, removing from list!
[258379.116] [dix] Could not init font path element /opt/X11/share/fonts/OTF, removing from list!
[258379.116] [dix] Could not init font path element /opt/X11/share/fonts/Speedo, removing from list!
[258379.116] [dix] Could not init font path element /opt/X11/share/fonts/TTF, removing from list!
[258379.116] [dix] Could not init font path element /opt/X11/share/fonts/Type1, removing from list!
[258379.116] [dix] Could not init font path element /opt/X11/share/fonts/util, removing from list!
[258379.223] noPseudoramiXExtension=0, pseudoramiXNumScreens=1
Engine Used: WS9Wine2.6
Hardware:
Hardware Overview:
Model Name: MacBook Pro
Model Identifier: MacBookPro9,2
Processor Name: Intel Core i5
Processor Speed: 2.5 GHz
Number of Processors: 1
Total Number of Cores: 2
L2 Cache (per Core): 256 KB
L3 Cache: 3 MB
Memory: 4 GB
Boot ROM Version: MBP91.00D3.B0C
SMC Version (system): 2.2f44
Sudden Motion Sensor:
State: Enabled
Graphics/Displays:
Intel HD Graphics 4000:
Chipset Model: Intel HD Graphics 4000
Type: GPU
Bus: Built-In
VRAM (Dynamic, Max): 1024 MB
Vendor: Intel (0x8086)
Device ID: 0x0166
Revision ID: 0x0009
Displays:
Color LCD:
Display Type: LCD
Resolution: 1280 x 800
Pixel Depth: 32-Bit Color (ARGB8888)
Main Display: Yes
Mirror: Off
Online: Yes
Built-In: Yes
Can somebody help me out step by step, or point me to some resource or whatever? Thanks.
This app uses WPF, you won't be able to use it on a Mac.
Quote from: dmilligan on May 16, 2017, 01:29:44 PM
This app uses WPF, you won't be able to use it on a Mac.
So, at the mono project website (http://www.mono-project.com/docs/gui/wpf/ (http://www.mono-project.com/docs/gui/wpf/)) it says:
QuoteSilverlight implement a subset of the WPF APIs and is available on Windows, MacOS X and through our own open source effort Moonlight it is available on Linux and other Unix systems.
Silverlight does not have all the capabilities of WPF, but both technologies are converging API-wise.
I found a tutorial online, seemingly, about how to get silverlight working in wine (https://www.youtube.com/watch?v=C3q9ECt1rAg (https://www.youtube.com/watch?v=C3q9ECt1rAg)), will that work?
Quotewill that work?
Sure, if you want to port this app from WPF to Silverlight (probably not trivial).
Ok. So again, bearing in mind I don't really know what I'm doing or what to look for, I am willing to give it a shot
(unless someone else who knows what they're doing wants to).
Here's what I've found so far:
WPF–Silverlight compatibility: https://msdn.microsoft.com/en-us/library/cc903925(v=VS.95).aspx (https://msdn.microsoft.com/en-us/library/cc903925(v=VS.95).aspx)
Some porting speculations: http://stackoverflow.com/questions/2948770/convert-wpf-application-to-silverlight (http://stackoverflow.com/questions/2948770/convert-wpf-application-to-silverlight)
I also found out that Wine has up to ver. 4.0 of the .net framework bundled with the .dmg. There are more recent versions which I can download and install into my copy of wine via winehq, but I don't know what's necessary and what is not.
I also just found what seems, albeit from my ignorant perspective, to be a good resource for this exact process:
https://www.codeproject.com/Articles/111356/Porting-from-WPF-to-Silverlight-The-Missing-Pieces (https://www.codeproject.com/Articles/111356/Porting-from-WPF-to-Silverlight-The-Missing-Pieces)
And then, I also found something about this on microsoft's website: https://blogs.msdn.microsoft.com/silverlight_sdk/2007/04/30/converting-a-wpfe-application-to-a-silverlight-scripting-application/ (https://blogs.msdn.microsoft.com/silverlight_sdk/2007/04/30/converting-a-wpfe-application-to-a-silverlight-scripting-application/)
Any pointers?
Why are you so bent on getting this app to run on mac when there are a plethora of alternatives that already run on mac and actually support more of the very latest features (like lossless compression)?
If you really want to port this app to Mac, I would recommend first decoupling the MLV processing code from the UI specific stuff and into its own assembly. Such an assembly should have minimal dependencies and therefore would run directly under mono or .net core. Then you can re-write the UI to be able to support only Silverlight stuff, while you're at it I would highly recommend adopting the MVVM design pattern (which is really how WPF was intended to be used). Finally you can eliminate the WPF specific code, and re-implement all of that functionality using only stuff available in Silverlight.
My friend just told me that cdngs and dng files are virtually the same, metadata notwithstanding. It was simply a fool's ignorant whim :) (meaning myself)
Mlvfs, cr2hdr.app and mlv_dump for steroids produces cdng files.
However I wouldn't mind you porting chmees fine tool to mac so be my guest :).
Quote from: Danne on May 19, 2017, 11:54:11 AM
However I wouldn't mind you porting chmees fine tool to mac so be my guest :).
Just make it happen already @spec42! :)
I am getting non-stop crashes from raw2cdng, which was working fine before... here is a screenshot and cut & page of the problem signature. Ideas? Suggestions? Alternatives?
In full production mode and pulling my hair out... :-[
https://ibb.co/iKpOfv
Problem signature:
Problem Event Name: CLR20r3
Problem Signature 01: raw2cdng.exe
Problem Signature 02: 1.7.9.0
Problem Signature 03: 585a6be9
Problem Signature 04: mscorlib
Problem Signature 05: 4.0.30319.34209
Problem Signature 06: 534894cc
Problem Signature 07: 4527
Problem Signature 08: 23b
Problem Signature 09: System.IO.IOException
OS Version: 6.1.7601.2.1.0.256.48
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789
remove and reinstall.
Try different version.
For me 1.7.5 just works and fast too.
CCleaner registry cleaner once got me out of a pickle with DNG's being none viewable and MLRawViewer did not start.
Good luck!
IO-Exception will be sort of
* Pathnaming or Filenaming is confusing (resulting file not found)
* Destination has ran out of space :)
raw2cdng doesnt write data into the registry. Often it helped erasing the raw2cdng.settings.json in /Users/[user]/AppData/roaming
How about first black frame issue? Do you find the problem?
Quote from: DeafEyeJedi on May 19, 2017, 07:01:45 PM
Just make it happen already @spec42! :)
@Danne: we'll see. I literally have no experience coding or scripting, so this would certainly be a project. I've wanted an excuse to learn about this kind of stuff though, so I'm going to make the attempt, at least.
to be honest, not the best way. because of so much hurdles, that are somewhere inside the os and some wrappers. Why dont u try to start from the beginning with the help of my sourcecode and (of course!) the code from ml.
@surin Dmitriy
hadnt had enough time until now. while i prepare to touch the code :D please upload some mlv/raw-examples with this bug. use my mlvdiag (http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/) to shorten the files, we only need the first frames for that.
Quote
to be honest, not the best way. because of so much hurdles, that are somewhere inside the os and some wrappers. Why dont u try to start from the beginning with the help of my sourcecode and (of course!) the code from ml.
@chmee Ok. Can you be a little more specific? I've tried compiling from source code before, but I didn't have too much luck. Where should I begin exactly? Perhaps we could continue this via PM?
@chmee I have a small request , would if be possible to add the reading/displaying the bit-rate block or blocks in mlvdiag ?
I find this program very useful for my development efforts & repairing damage mlv's :D A must have tool .
@spe24
Not really. I have nearly no time to fix/refresh this tool, cant afford being a tutor in coding. what did you tried? compiling my tool?
specific: as a coding beginner, you cant tell if a bug is because of you or because of a external codepiece like a wrapper (mono fi). no one guarantees, that these wrappers are bugfree.
@reddeercity
will try to - (bitrate blocks.. have to look what they are :)
with new built rec_crop4k, We aint able to convert cDNG, right?
Or do you have any solution? MLV to cDNG? Because cDNG better than DNG
I am photographer and video maker and I have a problem with the new version (MLV_lite 12bits). Raw magic no longer work.
just a question
Where can I get raw2cdng 1.7.9. for Mac ?
Quote from: jean-paul34 on June 30, 2017, 10:47:10 AM
I am photographer and video maker and I have a problem with the new version (MLV_lite 12bits). Raw magic no longer work.
just a question
Where can I get raw2cdng 1.7.9. for Mac ?
raw2cdng is Windows only and the app you mention that no longer works is not supported here as it violates Magic Lantern GPL. Please look at the sticky topics in the 'Raw Video Postprocessing' section for suggestions.
Quote from: hyalinejim on February 10, 2017, 02:44:10 AM
Premiere doesn't use ACR to import DNGS as it's expecting cinema DNGs which are different to "photo" DNGs. The metadata fields where the Lightroom adjustments are stored are not recognised in the Cinema DNG format.
Your best bet is to import them as image sequences in After Effects, where ACR will open for each sequence. It will apply the settings of the first file in the sequence to the whole clip. From there you can export to a lossless codec and edit in Premiere. For maximum flexibility use Cinelog DCP to generate 444 log masters, rather than using ACR's controls which are adaptive and not absolute and therefore introduce flickering. So you have to be careful about most of the shadow/highlights/contrast sliders if you want to use ACR.
Alternatively put all the Cinema DNG clips in a sequence in Premiere and do the same in After Effects. Nest that sequence to edit in Prem. When you're done, do the colour correction in After Effects and import that comp into Premiere, placing it on the track above your first sequence. Your edit will now have your colour correction applied.
Neither of these workflows are particularly straightforward, or fast. Another option is to use Resolve to generate log masters, which is pretty fast. Set your timeline to Cineon gamma and Alexa gamut. But definitely don't colour your CDNGs directly in Premiere as its debayering is pretty poor. ACR gives best results but is very slow.
Hi.
I've working for a while with a loseless workflow wtih Premiere Pro and AE (with ACR).
I read everywhere things like this:
"But definitely don't colour your CDNGs directly in Premiere as its debayering is pretty poor. ACR gives best results but is very slow." Re: The CinemaDNG Discussion (raw2cdng). « Reply #1359 on: February 10, 2017, 02:44:10 AM »
I just want to have the BEST results, and I don't care how complicated or slow it can get, so for me is no problem to use ACR despite it might be very slow.
The thing is that every time a open ACR and replace de clip (a proxy clip from Premiere that is linked to AE), ACR adds an incredible amount of sharpness to the clip. I hate that. The overall look of my clips after that, is just ugly. I have great colors but I loose that distinctive look that we all to see on clips shooted with hi ended camera: flat and subtle soft images.
Any ideas why that this happens?
Thanks in advance guys!
I believe the sharpness in ACR is turned up by default. You've already checked and turned that down?
Thanks man kyrobb!
I know it might sound a little stupid of me to ask this, but, I can't find any options for sharpness in ACR; just contrast.
I'm missing something?
I'm from Uruguay and I have spanish version of ACR isntalled, and it's only "Contraste", no option of "Nitidez", which is the translation of sharpness.
Thanks in advance!
It's been a while since I tried Magic Lantern the last time. But today I tried it again with my 50D in RAW and MLV and was amazed how fast and easy the conversion
was with raw2cdng on Windows 7.
I also read about the crop_rec_4k with the 5D Mark III. I don't have this camera yet but think about buying a used one if I can find a good deal.
Would the raw2cdng application work also with 12 or 14-bit lossless compressed MLV files?
I used to only use raw2cdng. But no, raw2cdng does not work with the new Lossless MLV's or 8-12 Bit.
But Danne has made a very nice piece of "program" that is very straight forward and actually delivers in higher quality than raw2cdng. With Darkframe Automation the DNG's simply come out cleaner than raw2cdng.
It is basically a script with a Menu.
http://www.magiclantern.fm/forum/index.php?topic=10526.100
Last build on page 5 is as of now the latest.
No not so raw2cdng supports 10 , 12 & of course 14bit
Just not compressed raw .
MLVProducer will soon support compressed raw which is a far
Better app just drag & drop the file and you have ACR like adjustments
For windows it the best app nothing else comes close .
Thank you both very much for your information. That was very helpfull.
Is the H264 4:4:4 on MLVProducer in 10 bit?
for me MLVProducer not the best App if you are editing cdng in premiere pro. you will have to correct white balance for all the clips which is time consuming. you will get discrete amount of magenta. some people including me raised the issue sometime back and there was no fix which is not the case with raw2cdng. go to reply number #802 in MLVProducer forum if you want to know what i mean. some people also complaint about not able to import h264 to davinci resolve converted with MLVProducer. its just my opinion. please go to its forum, downnload and test for yourself.
I was not aware that raw2cdng supports 10 and 12 bit. I never use it, just thought its so long since Chmee touched it.
If you want to convert straight to a compressed codec, MLVProducer is very good, it has many options.
You were asking about raw2cdng so I assumed you were looking at a DNG converter. If you want high quality DNG's, I think the Script Danne made is the best choice. For as far as I know, currently Danne's batch script is the only way to batch process MLV's with Darkframe Averaging on a Windows machine.
Hi Chmee!
After 4 years I still use this 5D3 camera and 14bit raw together with Raw2CDNG, I found the 1.6.1 app to be the best and least complicated, it always synced sound correct from problemetic clips etc. Today I realised that under Windows 10 newest framework in the Fall Creators Update the 1.6.1 app doesn't work. Had that problem aswell during when I tried Windows Insiders Program to get early updates. Later realeases is fine like the 1.7.9. Will try that one instead for a while but I will always think that the 1.6.1 is superior in function.
Will have to read what's new in 1.7.9, but any chance of getting a 1.6.1 app working in latest Windows Framework? I guess that's the problem for me?
Best Regards
Karl
I tried all different version of raw2cdng. The 1.5.0 BETA7 works just fine, flawlessly. And it's fast too!!
In fact the BETA7 is the only one that works for me with multiple MLV clips around that time period in both my Windows 8.1 Pro and Windows 10 Pro with latest Windows updates.
@allemyr
Try MLV App (http://www.magiclantern.fm/forum/index.php?topic=20025.0) project started by @Ilia3101.
Recently I've implemented DNG export into it. You can even export any available bit depth DNGs with lossless copression. And it works on Win32/64, Mac and Linux.
Quote from: bouncyball on December 23, 2017, 12:18:57 PM
@allemyr
Try MLV App (http://www.magiclantern.fm/forum/index.php?topic=20025.0) project started by @Ilia3101.
Recently I've implemented DNG export into it. You can even export any available bit depth DNGs with lossless copression. And it works on Win32/64, Mac and Linux.
Will check that when 1.5.0 Beta 7 stops to work in future Windows 10 and 8.1 updates. Shouldn't say anything about other MLV apps but generally they are to complicated with too much included.
The only option some of users need is fast MLV to CDNG with synced sound export. Maybe I will sell my 5D3 when and use another camera when the raw2cdng app doesn't work anymore.
I use in the raw2cdng app is 16 bit CDNG, no hightlight fix or chroma smoothing.
Even though 5D3 ML RAW might be "unsupported" later I ofcourse love the quailty this camera has to offer, it's crazy cheap and I've been using it for 4 years now.
No story in this video but the quality is insane in my opinion, upscaled to 4k 14-bit on youtube it's also interesting how much you do progress in quality and grading over the years with the same camera.
https://www.youtube.com/watch?v=bxm2WjVu2DY&t=3s
Update. Now the 1.6.1 works again for me, reinstalled Windows 10 Pro and now it works. Was at a Windows Insider Program earlier. Will continue to use this 1.6.1 app instead of the 1.5.0 Beta7 app. Have no clue what I did wrong earlier.