The CinemaDNG Discussion (raw2cdng)

[chmee]

never thought about it, 'cause i assumed its not relevant.

[kaco]

chmee, I'm trying to convert quite a large amount of MLV's (626 files, 1.4TB), but the app keeps crashing when I drop the files and the app starts to load them. I enabled the debug, but it sometimes crashes after loading approx 190 files, the next time after 196, etc. Everytime on another file. Is there anything I can do about it?

[kaco]

I tried to load less files (approx 180), it loads OK but after I clicked on convert, it crashed:

Code: [Select]

[convert_Click] all in all there are 76216 frames to convert
[doWork] started
[doWork] 12 Cores used
[doWork] -> converting item M04-1024_k01
[doWork] settings format 1 with BL2048 WL15000 with+out maximizingvalue 5,05983631871526
[doWork] prefix - filename generator [F]
[doWork] destinationPath -> M04-1024_k01
[doWork] outputFilename  -> M04-1024_k01
[doWork] Directory F:\RAW VIDEO\2015-04-04 Sevara Bobur\cdng\M04-1024_k01 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert

[kaco]

btw, I tested older releases -- raw2cdng.1.5.0.beta7 loads all files ok, from the 1.6.0 version it starts to crash. maybe it is something related to the thumbnail generator?

[glubber]

above the preview-window, on the right side, there is an inputfield (filled with 12345?!). write the frames around the "glitches" into this field. test some frames, that seems not to exist.(129-133 or 150-248). but there will be another patch for longer strips until m99, by now i assume, some of these pictures are inside the splitfiles beyond m20.

Aaah now i understand

Ok the results are:

nanual frame    file nbr    frame nbr
    128                  0               128
    129                  5               133 
    130                  6               134

    133                  9               137
    130                 10              138

    147                  2               250
    148                  3               251
    149                  4               252

so indeed it's skipping some files.

About the metadata:
You're right, as a filmmaker you're ought to know your FPS 

I'm still a rookie, so i try to analize my f-stop and shutterspeed when i overexposed etc.
In my case its also helpful for timelapses with exposure-jumps.
Dual-ISO raw video would need the base ISO for deinterlacing.

Thank you once again.

[chmee]

@kaco @all
open while using raw2cdng the taskmanager (or the ressourcemanager) and look on ram. i assume it exceds 1,5GB and then it crashes - .net is(was?) not able to allocate more than this. and of course the thumbnails/histogramm-pictures allocate ram as well. this is the reason as well, why FRSP crashes. it needs now significantly more ram per frame - so already coded something to minimize the ability to crash will put that online in the german evening

[Kharak]

Glad I am currently in Austria then

[chmee]

v1.7.2 is online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* did some tests with FSRP, worked for me, more checks needed. (if it crashes, try first to convert to 16bit)
* 70D color matrix added
* splitfiles can be more than only 20 (now until .m99 .r99)

regards chmee

[wyrlyn]

I still get framedrops, how can I fix that?

[kaco]

@chmee, you were right, the last value I saw in task manager was 1556 Mb, then it crashed.
Tried the 1.7.2. -- still not able to load all the files. I loaded less files, allocated approx 1GB of ram, but after I hit convert (16 bit) it crashed.

So for now, I have to stay with the 1.5.x version, since I work with tons of data many times a year.

[Kharak]

Tried same FRSP sequence that didn't work on 1.7.1 and now it works.

1 MLV file and 3 m00.

Haven't come to make any longer timelapses than that at the moment, but I will do some later today and see how it handles more.

[chmee]

@kaco

1.7.3 is online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* more ram usage should be possible now.

regards chmee

[Kharak]

You are an updating machine, Chmee.

Downloading..

[kaco]

@chmee, getting better results, but still crashing after filling more than approx 2000 Mb of RAM

[X-STATE]

raw2cdng.1.7.3 and raw2cdng.1.7.2 Not work with RAW from 70D.
Crash then start convert.
raw2cdng.1.6.5 was OK.

[chmee]

need a short RAW (<50MB) from 500D and 70D to fix it.

regards chmee

[X-STATE]

need a short RAW (<50MB) from 500D and 70D to fix it.

regards chmee

https://yadi.sk/d/_HIfLxdZfrd6Z

[chmee]

v1.7.4 online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* the crash came generally on raw - my fault, thanks for the hint. fixed.

regards chmee

[jankrueck]

Hey chmee,

I was wondering if there is an easy way of adding some grade/LUT to the Program.
As I'm filming most of my stuff on a 5Dmk2 and purchased VisionLOG's Osiris a while ago.

At the Moment I have to batch convert all of those beautiful cDNG's your Programm provides,
in ACR with "VisonLOG" straight to TIF. (while doing this, I loose raw benefits in Resolve)
http://www.vision-color.com/wp-content/uploads/2015/01/VisionLOG-for-Adobe-Camera-RAW.zip

Maybe we can find a better workflow, beside using BMD Film LUT's.
Feel free to add me on Skype, wir sprechen ja die gleiche Sprache
//trashnart

Cheers, Jan

ps: I just realized, that I follow ML raw for like 2 years now, and didn't have a account yet...

[chmee]

idea is not bad before that i have to solve the actual bug of leaving out some frames while converting.

@jankrueck
welcome geht auch whatsapp? oder threema?

@all
is this problem persistent? if you convert the file more times, are there every time the same frames, that are missing? i have to find out if its an multi-threading error or wrong filenumber-readout.

regards chmee

[Danne]

Hi Chmee! I, m curious if you, re app is adding specific color matrices for RAW files and if so how do you find out information about what camera has been used? I can, t see any camera related data with exiftool in the extracted dng files?

[Ottoga]

@chmee

Thanks for your efforts in making your app compatible with FRSP mlv's.  Today I have tested v1.7.4 with single and multi-spanned files.

All processed without error and without dropping any frames. A few things:

Is it possible to change the number of threads used into a user definable quantity or is there an ini file that can be modified? I'm not in a position to upgrade my notebook and it works quite hard with the default 4 threads.

MLV_Dump has the ability to change the frame rate via the -A parameter. Adding this feature to raw2cdng would eliminate the need to do this as a separate workflow step when working with FRSP or any timelapse MLVs. Just a feature suggestion.

Lastly, the following images are the same image and frame using both raw2cdng and MLV_Dump apps. On the image extracted via raw2cdng there is a black border an the left and top of the image. This does not occur in the MLV_Dump extractions.

A close visual inspection of both images suggests that the image is there in its entirety within the raw2cng image. Ideally, it would be better to have the DNG created without the black borders and thereby eliminate the need to crop and possibly stretch each frame in post.

Using raw2cdng
https://www.dropbox.com/s/gdrizkeyqtk7y87/5489000000005.dng?dl=0

Using MLV_Dump
https://www.dropbox.com/s/1eab8emt0c73c3x/5489_000005.dng?dl=0

Just some feedback and thoughts.  Thanks again for your ongoing support and development of this app.

Cheers.... Otto

[chmee]

@Danne
- MLV, according to modelname it writes the colormatrix1/2 and forwardmatrix1/2.
- on RAW its possible to define the body with the colormatrix provided inside RAW - i didnt implemented all bodies, but it works ( search for setModellOnColorMatrix in io.cs)

@Ottoga
(*) threads - ok, will do something but there have to be at least 2.
(*) framerate - doable as well - will think about handy GUI-implementation
(*) black border on frsp - ok, not that difficult as well, but have to inspect, if real picturesize is written/implemented in MLV.

regards chmee

[Danne]

Nice! Will check out. Got matrices implemented in mlv already so this will be really good for raw.
Thanks

[Danne]

Thanks! setModell opens up a lot!