Author Topic: The CinemaDNG Discussion (raw2cdng)  (Read 984999 times)

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1150 on: April 06, 2015, 11:33:20 AM »
never thought about it, 'cause i assumed its not relevant.

kaco

  • Member
  • ***
  • Posts: 101
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1151 on: April 06, 2015, 04:44:38 PM »
chmee, I'm trying to convert quite a large amount of MLV's (626 files, 1.4TB), but the app keeps crashing when I drop the files and the app starts to load them. I enabled the debug, but it sometimes crashes after loading approx 190 files, the next time after 196, etc. Everytime on another file. Is there anything I can do about it?

kaco

  • Member
  • ***
  • Posts: 101
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1152 on: April 06, 2015, 04:49:01 PM »
I tried to load less files (approx 180), it loads OK but after I clicked on convert, it crashed:
Code: [Select]
[convert_Click] all in all there are 76216 frames to convert
[doWork] started
[doWork] 12 Cores used
[doWork] -> converting item M04-1024_k01
[doWork] settings format 1 with BL2048 WL15000 with+out maximizingvalue 5,05983631871526
[doWork] prefix - filename generator [F]
[doWork] destinationPath -> M04-1024_k01
[doWork] outputFilename  -> M04-1024_k01
[doWork] Directory F:\RAW VIDEO\2015-04-04 Sevara Bobur\cdng\M04-1024_k01 created
[doWork] took DNGtemplate and changed values
[doWork] * init frameCount for multithreaded Convert

kaco

  • Member
  • ***
  • Posts: 101
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1153 on: April 06, 2015, 04:57:54 PM »
btw, I tested older releases -- raw2cdng.1.5.0.beta7 loads all files ok, from the 1.6.0 version it starts to crash. maybe it is something related to the thumbnail generator?

glubber

  • Member
  • ***
  • Posts: 118
  • 550D
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1154 on: April 06, 2015, 06:19:11 PM »
above the preview-window, on the right side, there is an inputfield (filled with 12345?!). write the frames around the "glitches" into this field. test some frames, that seems not to exist.(129-133 or 150-248). but there will be another patch for longer strips until m99, by now i assume, some of these pictures are inside the splitfiles beyond m20.

Aaah now i understand :)

Ok the results are:

nanual frame    file nbr    frame nbr
    128                  0               128
    129                  5               133 
    130                  6               134

    133                  9               137
    130                 10              138

    147                  2               250
    148                  3               251
    149                  4               252

so indeed it's skipping some files.

About the metadata:
You're right, as a filmmaker you're ought to know your FPS  ;)

I'm still a rookie, so i try to analize my f-stop and shutterspeed when i overexposed etc.
In my case its also helpful for timelapses with exposure-jumps.
Dual-ISO raw video would need the base ISO for deinterlacing.

Thank you once again.
EOS 550D // Sigma 18-200 // Sigma 18-70 // Canon 10-18 STM

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1155 on: April 06, 2015, 06:57:43 PM »
@kaco @all
open while using raw2cdng the taskmanager (or the ressourcemanager) and look on ram. i assume it exceds 1,5GB and then it crashes - .net is(was?) not able to allocate more than this. and of course the thumbnails/histogramm-pictures allocate ram as well. this is the reason as well, why FRSP crashes. it needs now significantly more ram per frame - so already coded something to minimize the ability to crash :) will put that online in the german evening :)

Kharak

  • Hero Member
  • *****
  • Posts: 1019
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1156 on: April 06, 2015, 08:46:31 PM »
Glad I am currently in Austria then ;)
once you go raw you never go back

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1157 on: April 06, 2015, 10:07:30 PM »
v1.7.2 is online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* did some tests with FSRP, worked for me, more checks needed. (if it crashes, try first to convert to 16bit)
* 70D color matrix added
* splitfiles can be more than only 20 (now until .m99 .r99)

regards chmee

wyrlyn

  • New to the forum
  • *
  • Posts: 43
  • 700D user
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1158 on: April 07, 2015, 11:02:31 AM »
I still get framedrops, how can I fix that?

kaco

  • Member
  • ***
  • Posts: 101
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1159 on: April 07, 2015, 11:08:38 AM »
@chmee, you were right, the last value I saw in task manager was 1556 Mb, then it crashed.
Tried the 1.7.2. -- still not able to load all the files. I loaded less files, allocated approx 1GB of ram, but after I hit convert (16 bit) it crashed.

So for now, I have to stay with the 1.5.x version, since I work with tons of data many times a year.

Kharak

  • Hero Member
  • *****
  • Posts: 1019
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1160 on: April 07, 2015, 12:53:20 PM »
Tried same FRSP sequence that didn't work on 1.7.1 and now it works.

1 MLV file and 3 m00.

Haven't come to make any longer timelapses than that at the moment, but I will do some later today and see how it handles more.
once you go raw you never go back

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1161 on: April 07, 2015, 06:28:19 PM »
@kaco

1.7.3 is online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* more ram usage should be possible now.

regards chmee

Kharak

  • Hero Member
  • *****
  • Posts: 1019
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1162 on: April 07, 2015, 06:41:12 PM »
You are an updating machine, Chmee.

Downloading.. :)
once you go raw you never go back

kaco

  • Member
  • ***
  • Posts: 101
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1163 on: April 07, 2015, 07:17:42 PM »
@chmee, getting better results, but still crashing after filling more than approx 2000 Mb of RAM

X-STATE

  • Freshman
  • **
  • Posts: 76
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1164 on: April 08, 2015, 08:25:45 AM »
raw2cdng.1.7.3 and raw2cdng.1.7.2 Not work with RAW from 70D.
Crash then start convert.
raw2cdng.1.6.5 was OK.
Body: Canon EOS 70D Lenses: Sigma 18-35mm f1.8 DC HSM Canon EF 85 f/1.8 USM
70D.112

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1165 on: April 08, 2015, 10:58:56 AM »
need a short RAW (<50MB) from 500D and 70D to fix it.

regards chmee

X-STATE

  • Freshman
  • **
  • Posts: 76
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1166 on: April 08, 2015, 08:12:49 PM »
need a short RAW (<50MB) from 500D and 70D to fix it.

regards chmee
https://yadi.sk/d/_HIfLxdZfrd6Z
Body: Canon EOS 70D Lenses: Sigma 18-35mm f1.8 DC HSM Canon EF 85 f/1.8 USM
70D.112

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1167 on: April 08, 2015, 08:41:06 PM »
v1.7.4 online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* :) the crash came generally on raw - my fault, thanks for the hint. fixed.

regards chmee

jankrueck

  • New to the forum
  • *
  • Posts: 33
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1168 on: April 10, 2015, 11:18:59 AM »
Hey chmee,

I was wondering if there is an easy way of adding some grade/LUT to the Program.
As I'm filming most of my stuff on a 5Dmk2 and purchased VisionLOG's Osiris a while ago.

At the Moment I have to batch convert all of those beautiful cDNG's your Programm provides,
in ACR with "VisonLOG" straight to TIF. (while doing this, I loose raw benefits in Resolve)
http://www.vision-color.com/wp-content/uploads/2015/01/VisionLOG-for-Adobe-Camera-RAW.zip

Maybe we can find a better workflow, beside using BMD Film LUT's.
Feel free to add me on Skype, wir sprechen ja die gleiche Sprache ;)
//trashnart

Cheers, Jan

ps: I just realized, that I follow ML raw for like 2 years now, and didn't have a account yet...

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1169 on: April 10, 2015, 11:23:19 AM »
idea is not bad :) before that i have to solve the actual bug of leaving out some frames while converting.

@jankrueck
welcome :) geht auch whatsapp? oder threema?

@all
is this problem persistent? if you convert the file more times, are there every time the same frames, that are missing? i have to find out if its an multi-threading error or wrong filenumber-readout.

regards chmee

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7297
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1170 on: April 10, 2015, 02:45:46 PM »
Hi Chmee! I, m curious if you, re app is adding specific color matrices for RAW files and if so how do you find out information about what camera has been used? I can, t see any camera related data with exiftool in the extracted dng files?

Ottoga

  • Member
  • ***
  • Posts: 203
  • The Ox is Slow but the Earth is Patient
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1171 on: April 10, 2015, 03:07:04 PM »
@chmee

Thanks for your efforts in making your app compatible with FRSP mlv's.  Today I have tested v1.7.4 with single and multi-spanned files.

All processed without error and without dropping any frames. A few things:

Is it possible to change the number of threads used into a user definable quantity or is there an ini file that can be modified? I'm not in a position to upgrade my notebook and it works quite hard with the default 4 threads.

MLV_Dump has the ability to change the frame rate via the -A parameter. Adding this feature to raw2cdng would eliminate the need to do this as a separate workflow step when working with FRSP or any timelapse MLVs. Just a feature suggestion.

Lastly, the following images are the same image and frame using both raw2cdng and MLV_Dump apps. On the image extracted via raw2cdng there is a black border an the left and top of the image. This does not occur in the MLV_Dump extractions.

A close visual inspection of both images suggests that the image is there in its entirety within the raw2cng image. Ideally, it would be better to have the DNG created without the black borders and thereby eliminate the need to crop and possibly stretch each frame in post.

Using raw2cdng
https://www.dropbox.com/s/gdrizkeyqtk7y87/5489000000005.dng?dl=0

Using MLV_Dump
https://www.dropbox.com/s/1eab8emt0c73c3x/5489_000005.dng?dl=0

Just some feedback and thoughts.  Thanks again for your ongoing support and development of this app.

Cheers.... Otto
EOS 7D.203, EFS 55-250mm, EF 75-300 III, Tamron 16-300 DiII VC PZD Macro, SpeedLite 580EX II.

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1172 on: April 11, 2015, 10:10:13 AM »
@Danne
- MLV, according to modelname it writes the colormatrix1/2 and forwardmatrix1/2.
- on RAW its possible to define the body with the colormatrix provided inside RAW - i didnt implemented all bodies, but it works ( search for setModellOnColorMatrix in io.cs)

@Ottoga
(*) threads - ok, will do something :) but there have to be at least 2.
(*) framerate - doable as well - will think about handy GUI-implementation
(*) black border on frsp - ok, not that difficult as well, but have to inspect, if real picturesize is written/implemented in MLV.

regards chmee

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7297
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1173 on: April 11, 2015, 10:58:14 AM »
Nice! Will check out. Got matrices implemented in mlv already so this will be really good for raw.
Thanks

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7297
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1174 on: April 11, 2015, 01:04:48 PM »
Thanks! setModell opens up a lot!