The CinemaDNG Discussion (raw2cdng)

Started by chmee, May 23, 2013, 10:46:55 AM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

chmee

(a) as i know mlrawviewer exists for linux as well.
(b) colortemperatur/whitebalance isnt recorded in raw, theres a static entry for around 5600°K.
(c) whitebalance-data in mlv are recorded only as presetnumber or -if manual- as kelvin-value, but there arent whitebalance data. so raw2cdng has the ability to reconctruct wb-data from this fields. (see here)

regards chmee
[size=2]phreekz * blog * twitter[/size]

Danne

Thanks. Found the mlv metadata, noticed the kelvin and rgb numbers. Having a hard time applying wb multipliers to dcraw to get correct wb from mlv, now i see why.
Impressive work on your windows application by the way.

chmee

raw2cdng 1.7.0 is out - download - sourcecode

* treat it as a beta :)

* massive speed up on read is working on raw and mlv, single and multifile
Quote
(tested with 1.6.9.alpha)
-- cf-card(usb3.0) to ssd - 2GB file (1.6.5 ~3min, 1.6.9 ~30seconds) 6x faster

-- hdd to hdd - 2GB file (1.6.5 ~34.9sec, 1.6.9 ~9.5sec) more than 3x faster
-- same hdd read write (1.6.5 ~32.4sec, 1.6.9 ~10.8 seconds) 3x faster
-- hdd to ssd - 2GB file (1.6.5 ~23.5sec, 1.6.9 ~11.5sec) 2x faster

-- ssd(ntfs) to ssd(exfat) - same 2GB file (1.6.5 ~10.1sec, 1.6.9 ~10.9sec) 8% slower (look beneath!)
-- ssd(exfat) to ssd(ntfs) - same 2GB file (1.6.5 ~6.6sec, 1.6.9 ~6.2sec) 6% faster (look beneath!)

-- same ssd (exFat) read/write (1.6.5 ~12.5sec, 1.6.9 10.8sec) 15% faster
-- same ssd (ntfs) read/write (1.6.5 ~9.5sec, 1.6.9 ~6.9sec) 37% faster

(exFat vs ntfs - cant explain that.. seems, write-rates on exfat here are slow(er) than expected..)
* verticalbanding only worked well with 16bit non maximized - now 12bit and 16bit maximized too.
* modelname, if not found/wrong in mlv, will be recreated from colormatrix-values @Walter_Schulz
* fps-tag - dropped frames are now well-formed (25000/1001 instead of 24975/1000)

* please test and tell

regards chmee
[size=2]phreekz * blog * twitter[/size]

jose_ugs

Testing it right now and it's as good as it gets!!!

andy kh

so eager to test but i wont be able to test until next month since im away from home
5D Mark III - 70D

Danne


rbrune

I tried converting a mlv from a 500D - looking at your code I would assume a colormatrix for the 500D is missing.

Excerpts from the log:

------------- 1.7.0 started at 15.03.09 08:18:57 --
[init][settings] -- ffmpeg Exists: false

[createMLVBlocklist] indexing Blocks in C:\Users\Peter Banane\Desktop\video\janarashtest\M09-0943.MLV
[getMLVAttrib] started
[getMLVAttrib] reading RAWI-Block
[getMLVAttrib] reading MLVI-Block
[getMLVAttrib] reading IDNT-Block
[getMLVAttrib] [!] modelName unknown - estimating from colormtrix
[getMLVAttrib] reading WBAL-Block
[getMLVAttrib] reading EXPO-Block
[getMLVAttrib] reading LENS-Block
[getMLVAttrib] reading WAVI-Block
[getMLVAttrib] [!] NO WAVI Block
***** [getMLVAttributes] metaData Object:
xResolution=1536
yResolution=864
frames=65
bitsperSample=14

colorMatrixA=System.Byte[]
colorMatrixB=
forwardMatrixA=
forwardMatrixB=

modell=Canon EOS 500D
camId=MAGIC29cc4ec
apiVersion=0


chmee

thank you for info. you could send me a super short (max 100mb) mlv-file to [email protected]
[size=2]phreekz * blog * twitter[/size]

rbrune

Mail with a short sample is on the way.

evanbuzzell

BUG: 1.7.0
The 12th frame of the video being exported drops on 1.7.0. I tried deleting the settings file and it does not work. Reverting back to 1.6.1 corrects the issue and I can confirm there is a non-blank frame in the position. I tried deleting the file again after switching versions and it is still skipping the 12th frame on every different video I export.

Going to try 1.6.5 and see how that works since it seems like many have used it and liked it. The 1.6.1 has just been so rock solid for me!

chmee

ok, thanks. will re-check framenumbering and sort-order of raw-sequences.
[size=2]phreekz * blog * twitter[/size]

KelvinK

Quote from: chmee on March 18, 2015, 09:38:58 AM
ok, thanks. will re-check framenumbering and sort-order of raw-sequences.

I've same issue with 1.7, frames drop. Was fine with 1.6.5.
6D - 5D - NEX - M50!

chmee

i just had 2 mentions before about that. in 1.7.0 i read and convert in chunks of 50 frames, thus it could be logical it's got some hickups on beginning or on end of that data-chunk, but 12 is quite strange for me. nonetheless, give me some days to analyze and patch it.

as long as you've got those issues, please use older versions you count as stable.
[size=2]phreekz * blog * twitter[/size]

KelvinK

Quote from: chmee on March 18, 2015, 11:56:25 AM
i just had 2 mentions before about that. in 1.7.0 i read and convert in chunks of 50 frames, thus it could be logical it's got some hickups on beginning or on end of that data-chunk, but 12 is quite strange for me. nonetheless, give me some days to analyze and patch it.

as long as you've got those issues, please use older versions you count as stable.

for me it was quite random was 24 and 65 in different MLVs, didndt try more coz of this bug.
6D - 5D - NEX - M50!

N/A

Pulled out the old pc so I could try out this app, great work chmee! I definitely like the jpg proxy feature. Now I just need to get Windows XP to read these damn dng files.
7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

andy kh

i hav premiere resolve round triping problem if the footage is shot in 23.97. resolve keep crashing while trying to import xml. everything works fine if i shoot in 25 fps
5D Mark III - 70D

chmee

thanks for info. will investigate. damn dropped frames :)
[size=2]phreekz * blog * twitter[/size]

chmee

v1.7.1 online - http://www.phreekz.de/wordpress/2014/04/magiclantern-raw2cdng-1-5-0/

* fixed 500D support
* inputfield above the preview. write a frame-no into it, it will show you the data to the VIDF-chunk. (if debug is enabled, it will be saved into the debug-file)
* possibly framelost fixed

* still open:
* framelost fix not guaranteed, therefore this frame analyzer.
[size=2]phreekz * blog * twitter[/size]

glubber

Hey chmee,

i'll give you some feedback... err i have some problem:

I made a FRSP-Timelapse with an outdated built which recorded 1 MLV File (4 GB) and 99 single picture files (.M00 to .M99), which was state of the art back then.
When i open the mlv in raw2cdng it sees 150 frames in 22 mlv-files resulting in dng-files counting from 0 - 128, 133 - 149 and 249 - 252. (see pictures below)





debug.log:
https://dl.dropboxusercontent.com/u/90483500/MAGICLANTERN/raw2cdng.2.debug.log

Do you see a chance to get this working?

Thanks in advance for your generous help and keep up the good work. :)
EOS 550D // Sigma 18-200 // Sigma 18-70 // Canon 10-18 STM

chmee

@Glubber
thanx a lot for this insight. please look with 1.7.1 inside the frame-metadata 128, 129, 130. There should be a jump of framenumbers.. Reason: For MLV i take the framenumbers, that are written into the metadata and i dont renumber them while converting - contrary to RAW where i use an own counting.

this could be the key for "missing" frames.

edit: this 4GB- MLV should have a lot more frames, isnt it? move the files m01-m99 temporarily into another folder and do a second test.
edit2: until m99? fk. i restricted to somewhere m20. patch will come.
[size=2]phreekz * blog * twitter[/size]

Kharak

I might add that I tried converting FRSP MLV too, it had one MLV and three m.00 files.


But 1.7.1 crashes almost immediately. Also when only trying with the MLV file alone which contains around 100 pics. 1.6.5 was able to convert only the MLV file once, but if I put all m00 files together it also crashes immediately.

once you go raw you never go back

andy kh

5D Mark III - 70D

glubber

Thank you for your quick response chmee!

I'm afraid i don't understand " look with 1.7.1 inside the frame-metadata 128, 129, 130."

I just get:
[previewTimer_Tick] show previewframe 130 from 23550017
[showPicture] started


Clicking "show meta blocks" results in the log-file i posted.
When looking at the log-file i see no inconsistend numbering outside 0 to 150.

If i test the big mlv-file alone it shows 129 frames. -> 129*31 mb = 3999  ~ 4GB. So the math seem right.

Again many thanks for your efforts!
i really appreciate it.
EOS 550D // Sigma 18-200 // Sigma 18-70 // Canon 10-18 STM

chmee

above the preview-window, on the right side, there is an inputfield (filled with 12345?!). write the frames around the "glitches" into this field. test some frames, that seems not to exist.(129-133 or 150-248). but there will be another patch for longer strips until m99, by now i assume, some of these pictures are inside the splitfiles beyond m20.



edit:
QuoteIf i test the big mlv-file alone it shows 129 frames. -> 129*31 mb = 3999  ~ 4GB. So the math seem right
yeah. i forgot, its frsp.

regards chmee
[size=2]phreekz * blog * twitter[/size]

glubber

Hello chmee,

another issue that's puzzling me is the metadata of mlv-files (regular video, not FRSP):

In the raw2cdng app i see all the metadata like shutterspeed, fps, focal-length, iso etc.
But in both Lightroom and exiftoolgui i see nothing except cameramodel, framesize, colormatrix and white- and blacklevel.

Is there no way to write the first mentioned metadata inside the dng or am i missing something?

Thx again
EOS 550D // Sigma 18-200 // Sigma 18-70 // Canon 10-18 STM