Messages

+ it would be perfect if folder drop will be working. Now I use 3 cameras -- I have to copy the mlvs into folders "card1, card2,.." because 2 cameras can create same named files. Then I convert MLVs to cdngs with namespace "[D2]_[F]_[P]_". But I have to manually drop every folder, which is time consuming if I have 40 cards and so.

@chmee: in beta7 there is still present the bug that the app will crash when scrolling all loaded MLV files during convert

btw: i implemented the WBAL-usage and resolve, premiere and photoshop behaves quite well. did some more exception-trapping and debug-writing. beta7 is near.

very nice!

For me it's totally OK to use some "average" values. This values will still be used only when creating proxies and then finetuned in the final color grade.

Hey there,

unless I did not make any mistake, the values are different on my 5D mark III's:

1. Older 5D mark III, bought in 2012, SN 013021001***, FW 1.1.3

Code Select Expand

2500,1182,1024,1024,31392600,1226,1024,1024,30222700,1274,1024,1024,29212800,1326,1024,1024,28192900,1369,1024,1024,27523000,1413,1024,1024,26893100,1450,1024,1024,25763200,1489,1024,1024,24673300,1526,1024,1024,23993400,1565,1024,1024,23303500,1606,1024,1024,22603600,1638,1024,1024,22083700,1672,1024,1024,21533800,1708,1024,1024,20973900,1739,1024,1024,20564000,1768,1024,1024,20164100,1802,1024,1024,19784200,1833,1024,1024,19384300,1859,1024,1024,19074400,1886,1024,1024,18724500,1913,1024,1024,18434600,1942,1024,1024,18114700,1967,1024,1024,17804800,1990,1024,1024,17534900,2009,1024,1024,17275000,2028,1024,1024,16995100,2048,1024,1024,16755200,2068,1024,1024,16495300,2089,1024,1024,16315400,2110,1024,1024,16135500,2131,1024,1024,15965600,2153,1024,1024,15775700,2171,1024,1024,15635800,2189,1024,1024,15475900,2208,1024,1024,15336000,2226,1024,1024,15176100,2241,1024,1024,15076200,2255,1024,1024,14946300,2270,1024,1024,14836400,2284,1024,1024,14716500,2300,1024,1024,14586600,2315,1024,1024,14486700,2330,1024,1024,14366800,2346,1024,1024,14276900,2362,1024,1024,14157000,2378,1024,1024,14047100,2394,1024,1024,13967200,2405,1024,1024,13897300,2416,1024,1024,13807400,2427,1024,1024,13727500,2439,1024,1024,13647600,2450,1024,1024,13577700,2461,1024,1024,13487800,2473,1024,1024,13417900,2485,1024,1024,13328000,2497,1024,1024,13268100,2509,1024,1024,13178200,2521,1024,1024,13118300,2533,1024,1024,13038400,2545,1024,1024,12988500,2551,1024,1024,12938600,2558,1024,1024,12878700,2570,1024,1024,12828800,2576,1024,1024,12768900,2583,1024,1024,12719000,2595,1024,1024,12669100,2602,1024,1024,12609200,2608,1024,1024,12569300,2615,1024,1024,12509400,2628,1024,1024,12459500,2635,1024,1024,12419600,2641,1024,1024,12359700,2655,1024,1024,12319800,2661,1024,1024,12259900,2668,1024,1024,122110000,2675,1024,1024,1215

Code Select Expand

Daylight,5200,2068,1024,1024,1649
Shade,7000,2378,1024,1024,1404
Cloudy,6000,2226,1024,1024,1517
Tungsten,3200,1489,1024,1024,2467
Fluorescent,3714,1817,1024,1024,2372
Flash,6320,2305,1024,1024,1492

1. Newer 5D mark III, bought in 2014, SN 268021002***, FW 1.1.3

Code Select Expand

2500,1159,1024,1024,30132600,1201,1024,1024,29052700,1247,1024,1024,28042800,1295,1024,1024,27092900,1334,1024,1024,26483000,1376,1024,1024,25833100,1413,1024,1024,24793200,1450,1024,1024,23783300,1485,1024,1024,23153400,1522,1024,1024,22503500,1558,1024,1024,21853600,1589,1024,1024,21363700,1621,1024,1024,20853800,1654,1024,1024,20323900,1683,1024,1024,19934000,1713,1024,1024,19564100,1745,1024,1024,19204200,1774,1024,1024,18834300,1799,1024,1024,18534400,1824,1024,1024,18244500,1846,1024,1024,17924600,1872,1024,1024,17654700,1896,1024,1024,17364800,1917,1024,1024,17114900,1935,1024,1024,16865000,1956,1024,1024,16595100,1975,1024,1024,16365200,1993,1024,1024,16115300,2013,1024,1024,15945400,2032,1024,1024,15775500,2052,1024,1024,15605600,2072,1024,1024,15425700,2093,1024,1024,15295800,2110,1024,1024,15135900,2127,1024,1024,15006000,2144,1024,1024,14856100,2158,1024,1024,14756200,2171,1024,1024,14646300,2185,1024,1024,14526400,2198,1024,1024,14426500,2212,1024,1024,14316600,2226,1024,1024,14216700,2241,1024,1024,14096800,2255,1024,1024,14006900,2270,1024,1024,13897000,2284,1024,1024,13787100,2300,1024,1024,13717200,2310,1024,1024,13647300,2320,1024,1024,13557400,2330,1024,1024,13487500,2341,1024,1024,13417600,2351,1024,1024,13327700,2367,1024,1024,13267800,2378,1024,1024,13177900,2389,1024,1024,13118000,2399,1024,1024,13048100,2411,1024,1024,12968200,2422,1024,1024,12908300,2433,1024,1024,12828400,2444,1024,1024,12778500,2450,1024,1024,12738600,2461,1024,1024,12668700,2467,1024,1024,12628800,2473,1024,1024,12578900,2485,1024,1024,12519000,2491,1024,1024,12479100,2497,1024,1024,12429200,2509,1024,1024,12379300,2515,1024,1024,12329400,2521,1024,1024,12289500,2533,1024,1024,12229600,2539,1024,1024,12189700,2545,1024,1024,12149800,2558,1024,1024,12089900,2564,1024,1024,120410000,2570,1024,1024,1198

Code Select Expand

Daylight,5200,1993,1024,1024,1611
Shade,7000,2284,1024,1024,1378
Cloudy,6000,2144,1024,1024,1485
Tungsten,3200,1450,1024,1024,2378
Fluorescent,3714,1751,1024,1024,2280
Flash,6320,2217,1024,1024,1460

For instance also the shutter sounds a bit different on these, so maybe there's some little design difference?

I also doublechecked that no WB shift was set on both and that both had same settings, same standard profile, disabled all "enchance" functions, etc.

OKay, not any  0xff or 0x01 phrazes found, so I'd say we'll done here

JimmyD30-- how can i change thr headers?

To all: If that shot could have been done with smaller file we would have done that. I am okay with the risks. Just wanted to know, if it's completely or partly corrupted. (Btw, we shoot more that 2-3tb/day and have filmed many many days)

G3gg0: i will search for it in a week, currently away from office

Hi,

I've got one 82,2 GB (88 354 624 096 bytes) MLV file and I cannot read it. Could someone please help me to investigate if the file is totally empty, or just partially corrupted?

We shoot whole day on many cards. Card containing this file had previous and next MLV without a problem. We even do not remember any on location problem. Maybe the battery run out during recording? We really can't remember. We shoot with the some build from feb to may, but don't know exactly which (we updated the cards in the meantime.

Here's some details:

Code Select Expand


K:\>mlv_dump -v M07-1702.MLV

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: 'M07-1702.MLV'
   - Verbose messages
   - Verify file structure
   - Dump all block information
File M07-1702.MLV opened
File M07-1702.M00 not existing.
Processing...
[ERROR] Invalid block size at position 0x00000000
Processed 0 video frames
Done


Here's the screenshot from HxD:
https://drive.google.com/file/d/0B-7-kTVW5sAqSjNaaHM2eHQ3bUU/edit?usp=sharing

How could the camera create 82 gigs of zeros?

@chmee: now tested with mlv convertor 1.9.2 ... the files open, but won't extract... since it's a 80gb of data I think there's something important in it.. oh, god...
Do you think there is a chance to get the data from the file?

tested it also with mlv2dng:

Code Select Expand

mlv2dng.exe 002_M0336.MLV
Exception! Header not recognized: ''

Exception! Header not recognized: ''

mlv_dump:

Code Select Expand


mlv_dump.exe 002_M0336.MLV

MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: '002_M0336.MLV'
   - Verify file structure
File 002_M0336.MLV opened
File 002_M0336.M00 not existing.
Processing...
[ERROR] Missing file header
Processed 0 video frames
Done


Hi, from my 2TB shooting I have a trouble opening one of my file. The loader bar is still loading, both raw2cdng and mlvdiag.

What i did -- downloaded all the mlv's, deleted those 1 bytes and batch renamed all the mlv's (because of Resolve round-trip method).

The debug info:

Code Select Expand


[drop] ------
[drop] -- file K:\xxx\002_M0336.MLV will be analyzed now.
[drop] is MLV
[setFileinfoData] set/changed Filedata
[drop] FileinfoData set
[createMLVBlocklist] started
[createMLVBlocklist] splitFiles counted (1)
[createMLVBlocklist] indexing Blocks in K:\xxx\002_M0336.MLV


My theory is that the end of this file was splitted into those 1byte chunks and sice I deleted them, raw2cdng don't know where to find them? The progress bar is still active, but the disc activity is zero.

EDIT: But I deleted many 1 byte files and from 399 MLV's only with this I am getting error. The file exctracts with MLV Converter 1.9.2 fine.

please try latest nightly (or from there)

Tested the version from jenkins, also got error as SergePlotkin. Tested your build of mlv_rec, the console error has gone. But now I am missing the option to allocate space with LV off (some kind of extra hack). And on record start/stop i got message:
Hacked error at 0:
expected e3a00032, got e1a00000

g3gg0, is it safe now to start using .123 for the MLV raw?

[git with the sourcecode]

so. i've done a git with the sourcecode. therefore its now free software (this software will never cost a dime, its up to you to donate me some cents). i hope, some of you will help me to optimize code, to put some new options into it and to still let me learn the next stage of coding tried to comment most points.

https://bitbucket.org/chmee/raw2cdng

(i'm now a little bit tired, have to take my time for my children and for two other projects)

regards chmee

Bravo! Thanks a lot! I hope will see the vertical banding checkbox soon!

I also put some donation for the start. But I thought you can't start porting while there is no firmware update from Canon. This is not true now?

can someone explain to me what chroma smoothing option does in raw video postprocessing? whats the diference between none, 2x2, 3x3, etc... does it have the same effect for all the cameras, what's the recommended setting for the 5d mark iii shots?

sorry but i coudn't find any post regarding this concerns, i think the search function in this forum kind of sucks.


cheers.

I was also searching for this. Look here: http://www.gna-productions.com/tutos/how-to-process-magic-lantern-raw-videos Part 2: 2) REMOVE PINK DOTS USING PINKDOTREMOVER

vertical banding and chroma smoothing are under gpl-license. i have to talk with @g3gg0/@a1ex/@1% about using their code, because in this case i'm too lazy to solve it myself.. but, they're mostly recoded and inside

@DavidSh As i've read, ThomasWorth is preparing his new RawMagic-Version.

I'm OK with you using my code for free software under the terms of GPL (nothing to do with price).

Maybe you can tell Thomas Worth too; he is currently using my GPL'ed code in his proprietary app.

If I understand this correctly, does this mean that we won't have vertical banding correction soon?

the new interface is perfect. great job

it doesnt work in resolve

CinemaDNG + wav from Blackmagic Design cameras work. So it's probably just a question of correct filename convetions and maybe some metadata.

@chmee -- I can provide you some BMD footage for the "reverse engineering" if you are interested.

@kako

i dont understand why dont you guys import cdng in cc and start your cut? you guys neednt sync audio manaully in premiere pro thats the reason why i love raw2cdng. after your cut you can grade in resolve or after effects without having any problem

It's because I am still on CS6. If there would be ACR involved into Premiere CC I would definetely upgrade. But just because of CDNG "proxies" and then also round tripping to Resolve it's not worth the money for me.

If the CDNG audio works with Premiere, I think it might somehow work in AE and Resolve too. If it works with Blackmagic CDNG files. I would say that the Premiere just checks the filenames while AE and Resolve also timecode or what -- that's why the audio and video imports as two separate files.

Also, using Premiere CC to grade raw makes no sense to me when you can use Resolve which is for free. The process is not so much more complicated, isn't it?

@chmee, it is very slow and complicated process (AE or resolve), but it will give you more production options. For me, the basic color grading tools in Premiere CC are not enough (I think they are more color correction than color grading). So even the process is slow and complicated, it is sometimes worth it.

I usually export with raw2cdng (or MLV converter if I need to fix vertical bandings). Then depending on the type of a project:

1. Create proxies in Resolve (faster then realtime). Import into Premiere. Make my edit, export XML to resolve, create final grade.

2. Create proxies in Resolve. Import into Premiere. Make my edit. Then duplicate the edit in the another layer. Nest every clip. Replace with AE composition (Adobe dynamic link) and import CDNG sequence. Then I make color grading on the first frame with Lightroom, save metadata and reload footage in AE. It will also update the clip in Premiere. Very slow, but higher quality debayer process then in Resolve.

3. Grade with Lightroom, import all into AE and directly export Prores/DNxHD -- all clips graded.

The second workflow is highest quality and with ability to render just the clips used in the final edit with ACR. But very annoying that I have to manually nest and relink every clip to AE.

Still though, I have problems with audio You should really look on that. If you have 1TB project with 250 shots, I spend day or two linking just the audio.

@chmee: we have discussed the wav+dng import into resolve, so I know you are working on this. Have you tested this in Adobe AE? When I drop the CDNG folder into AE it imports just the video, not audio in my case. Maybe I'm doing smth wrong or it is not working? But it would save a lot of time if this will work.

[after effects]

Guys, I select the cdng folder, drag it into after effects. The ACR will open, but after importing, I get no audio. I have to manually select every wav and then move it into the compostion. What am I doing wrong? Thanks!

oh sorry, thought i already described that error.
but it seems i typed it (from my phone) and i didnt submit that message.
doh!

ok then let me explain again:

that message is from the metadata thread.
mlv_rec tried to put some metadata into that (usually) free area in front of a frame.
(EDMAC needs 4KiB alignment, so there is up to 4KiB empty space in front of a frame)

but there was not enough free space in the last few frames, so the metadata
has to be requeued and to be transmitted next time.

if you get only one message (failed: 1) or now and then one more, it doesnt matter.
but if the "failed" counts up permanently and the "queued" stays constant, then something is totally wrong.
(either "queued" or "failed" count up)


so: your footage should be fine without any information loss.
if not, then we have to talk

Thanks. I usually got about 5-7 messages per 128Gb file. Wouldn't it be better to write this information to console, which I am able to turn off?