Author Topic: The CinemaDNG Discussion (raw2cdng)  (Read 975005 times)

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1125 on: February 17, 2015, 09:08:27 AM »
(a) as i know mlrawviewer exists for linux as well.
(b) colortemperatur/whitebalance isnt recorded in raw, theres a static entry for around 5600°K.
(c) whitebalance-data in mlv are recorded only as presetnumber or -if manual- as kelvin-value, but there arent whitebalance data. so raw2cdng has the ability to reconctruct wb-data from this fields. (see here)

regards chmee

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7252
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1126 on: February 17, 2015, 09:38:48 AM »
Thanks. Found the mlv metadata, noticed the kelvin and rgb numbers. Having a hard time applying wb multipliers to dcraw to get correct wb from mlv, now i see why.
Impressive work on your windows application by the way.

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1127 on: February 28, 2015, 02:48:29 PM »
raw2cdng 1.7.0 is out - download - sourcecode

* treat it as a beta :)

* massive speed up on read is working on raw and mlv, single and multifile
Quote
(tested with 1.6.9.alpha)
-- cf-card(usb3.0) to ssd - 2GB file (1.6.5 ~3min, 1.6.9 ~30seconds) 6x faster

-- hdd to hdd - 2GB file (1.6.5 ~34.9sec, 1.6.9 ~9.5sec) more than 3x faster
-- same hdd read write (1.6.5 ~32.4sec, 1.6.9 ~10.8 seconds) 3x faster
-- hdd to ssd - 2GB file (1.6.5 ~23.5sec, 1.6.9 ~11.5sec) 2x faster

-- ssd(ntfs) to ssd(exfat) - same 2GB file (1.6.5 ~10.1sec, 1.6.9 ~10.9sec) 8% slower (look beneath!)
-- ssd(exfat) to ssd(ntfs) - same 2GB file (1.6.5 ~6.6sec, 1.6.9 ~6.2sec) 6% faster (look beneath!)

-- same ssd (exFat) read/write (1.6.5 ~12.5sec, 1.6.9 10.8sec) 15% faster
-- same ssd (ntfs) read/write (1.6.5 ~9.5sec, 1.6.9 ~6.9sec) 37% faster

(exFat vs ntfs - cant explain that.. seems, write-rates on exfat here are slow(er) than expected..)
* verticalbanding only worked well with 16bit non maximized - now 12bit and 16bit maximized too.
* modelname, if not found/wrong in mlv, will be recreated from colormatrix-values @Walter_Schulz
* fps-tag - dropped frames are now well-formed (25000/1001 instead of 24975/1000)

* please test and tell

regards chmee

jose_ugs

  • Member
  • ***
  • Posts: 243
  • 5DM3
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1128 on: February 28, 2015, 11:56:38 PM »
Testing it right now and it's as good as it gets!!!

andy kh

  • Hero Member
  • *****
  • Posts: 537
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1129 on: March 01, 2015, 07:12:32 AM »
so eager to test but i wont be able to test until next month since im away from home
5D Mark III - 70D

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7252
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1130 on: March 01, 2015, 11:32:05 AM »
cool stuff.

rbrune

  • Contributor
  • Freshman
  • *****
  • Posts: 64
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1131 on: March 09, 2015, 09:23:09 AM »
I tried converting a mlv from a 500D - looking at your code I would assume a colormatrix for the 500D is missing.

Excerpts from the log:
Code: [Select]
------------- 1.7.0 started at 15.03.09 08:18:57 --
[init][settings] -- ffmpeg Exists: false

[createMLVBlocklist] indexing Blocks in C:\Users\Peter Banane\Desktop\video\janarashtest\M09-0943.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] [!] modelName unknown - estimating from colormtrix
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
[getMLVAttrib] [!] NO WAVI Block
 ***** [getMLVAttributes] metaData Object:
xResolution=1536
yResolution=864
frames=65
bitsperSample=14

colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=

modell=Canon EOS 500D
camId=MAGIC29cc4ec
apiVersion=0


chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1132 on: March 09, 2015, 10:47:08 AM »
thank you for info. you could send me a super short (max 100mb) mlv-file to raw2cdng@phreekz.de

rbrune

  • Contributor
  • Freshman
  • *****
  • Posts: 64
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1133 on: March 09, 2015, 05:29:55 PM »
Mail with a short sample is on the way.

evanbuzzell

  • New to the forum
  • *
  • Posts: 10
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1134 on: March 18, 2015, 02:05:48 AM »
BUG: 1.7.0
The 12th frame of the video being exported drops on 1.7.0. I tried deleting the settings file and it does not work. Reverting back to 1.6.1 corrects the issue and I can confirm there is a non-blank frame in the position. I tried deleting the file again after switching versions and it is still skipping the 12th frame on every different video I export.

Going to try 1.6.5 and see how that works since it seems like many have used it and liked it. The 1.6.1 has just been so rock solid for me!

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1135 on: March 18, 2015, 09:38:58 AM »
ok, thanks. will re-check framenumbering and sort-order of raw-sequences.

KelvinK

  • Senior
  • ****
  • Posts: 267
  • Less words, more films
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1136 on: March 18, 2015, 10:36:31 AM »
ok, thanks. will re-check framenumbering and sort-order of raw-sequences.

I've same issue with 1.7, frames drop. Was fine with 1.6.5.
6D - 5D - NEX - M50!

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1137 on: March 18, 2015, 11:56:25 AM »
i just had 2 mentions before about that. in 1.7.0 i read and convert in chunks of 50 frames, thus it could be logical it's got some hickups on beginning or on end of that data-chunk, but 12 is quite strange for me. nonetheless, give me some days to analyze and patch it.

as long as you've got those issues, please use older versions you count as stable.

KelvinK

  • Senior
  • ****
  • Posts: 267
  • Less words, more films
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1138 on: March 18, 2015, 03:17:10 PM »
i just had 2 mentions before about that. in 1.7.0 i read and convert in chunks of 50 frames, thus it could be logical it's got some hickups on beginning or on end of that data-chunk, but 12 is quite strange for me. nonetheless, give me some days to analyze and patch it.

as long as you've got those issues, please use older versions you count as stable.

for me it was quite random was 24 and 65 in different MLVs, didndt try more coz of this bug.
6D - 5D - NEX - M50!

N/A

  • Hero Member
  • *****
  • Posts: 576
  • Dreaming in 14 bit
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1139 on: March 18, 2015, 10:37:27 PM »
Pulled out the old pc so I could try out this app, great work chmee! I definitely like the jpg proxy feature. Now I just need to get Windows XP to read these damn dng files.
7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

andy kh

  • Hero Member
  • *****
  • Posts: 537
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1140 on: April 03, 2015, 10:21:58 AM »
i hav premiere resolve round triping problem if the footage is shot in 23.97. resolve keep crashing while trying to import xml. everything works fine if i shoot in 25 fps
5D Mark III - 70D

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1141 on: April 03, 2015, 10:30:58 AM »
thanks for info. will investigate. damn dropped frames :)

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1142 on: April 04, 2015, 03:49:40 PM »
v1.7.1 online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* fixed 500D support
* inputfield above the preview. write a frame-no into it, it will show you the data to the VIDF-chunk. (if debug is enabled, it will be saved into the debug-file)
* possibly framelost fixed

* still open:
* framelost fix not guaranteed, therefore this frame analyzer.

glubber

  • Member
  • ***
  • Posts: 118
  • 550D
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1143 on: April 04, 2015, 08:01:05 PM »
Hey chmee,

i'll give you some feedback... err i have some problem:

I made a FRSP-Timelapse with an outdated built which recorded 1 MLV File (4 GB) and 99 single picture files (.M00 to .M99), which was state of the art back then.
When i open the mlv in raw2cdng it sees 150 frames in 22 mlv-files resulting in dng-files counting from 0 - 128, 133 - 149 and 249 - 252. (see pictures below)





debug.log:
https://dl.dropboxusercontent.com/u/90483500/MAGICLANTERN/raw2cdng.2.debug.log

Do you see a chance to get this working?

Thanks in advance for your generous help and keep up the good work. :)
EOS 550D // Sigma 18-200 // Sigma 18-70 // Canon 10-18 STM

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1144 on: April 04, 2015, 08:10:43 PM »
@Glubber
thanx a lot for this insight. please look with 1.7.1 inside the frame-metadata 128, 129, 130. There should be a jump of framenumbers.. Reason: For MLV i take the framenumbers, that are written into the metadata and i dont renumber them while converting - contrary to RAW where i use an own counting.

this could be the key for "missing" frames.

edit: this 4GB- MLV should have a lot more frames, isnt it? move the files m01-m99 temporarily into another folder and do a second test.
edit2: until m99? fk. i restricted to somewhere m20. patch will come.

Kharak

  • Hero Member
  • *****
  • Posts: 1019
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1145 on: April 04, 2015, 08:39:00 PM »
I might add that I tried converting FRSP MLV too, it had one MLV and three m.00 files.


But 1.7.1 crashes almost immediately. Also when only trying with the MLV file alone which contains around 100 pics. 1.6.5 was able to convert only the MLV file once, but if I put all m00 files together it also crashes immediately.

once you go raw you never go back

andy kh

  • Hero Member
  • *****
  • Posts: 537
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1146 on: April 06, 2015, 10:09:38 AM »
stil no luck with round triping
5D Mark III - 70D

glubber

  • Member
  • ***
  • Posts: 118
  • 550D
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1147 on: April 06, 2015, 11:11:37 AM »
Thank you for your quick response chmee!

I'm afraid i don't understand " look with 1.7.1 inside the frame-metadata 128, 129, 130."

I just get:
[previewTimer_Tick] show previewframe 130 from 23550017
[showPicture] started


Clicking "show meta blocks" results in the log-file i posted.
When looking at the log-file i see no inconsistend numbering outside 0 to 150.

If i test the big mlv-file alone it shows 129 frames. -> 129*31 mb = 3999  ~ 4GB. So the math seem right.

Again many thanks for your efforts!
i really appreciate it.
EOS 550D // Sigma 18-200 // Sigma 18-70 // Canon 10-18 STM

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1148 on: April 06, 2015, 11:28:04 AM »
above the preview-window, on the right side, there is an inputfield (filled with 12345?!). write the frames around the "glitches" into this field. test some frames, that seems not to exist.(129-133 or 150-248). but there will be another patch for longer strips until m99, by now i assume, some of these pictures are inside the splitfiles beyond m20.



edit:
Quote
If i test the big mlv-file alone it shows 129 frames. -> 129*31 mb = 3999  ~ 4GB. So the math seem right
yeah. i forgot, its frsp.

regards chmee

glubber

  • Member
  • ***
  • Posts: 118
  • 550D
Re: The CinemaDNG Discussion (raw2cdng)
« Reply #1149 on: April 06, 2015, 11:32:00 AM »
Hello chmee,

another issue that's puzzling me is the metadata of mlv-files (regular video, not FRSP):

In the raw2cdng app i see all the metadata like shutterspeed, fps, focal-length, iso etc.
But in both Lightroom and exiftoolgui i see nothing except cameramodel, framesize, colormatrix and white- and blacklevel.

Is there no way to write the first mentioned metadata inside the dng or am i missing something?

Thx again
EOS 550D // Sigma 18-200 // Sigma 18-70 // Canon 10-18 STM