Messages

GREAT JOB!!! this version resolves the bad pixel problem in Davinci Resolve importing (cdng) with some camera. (60D)
Thanks. 

That's good to hear! There is a very simple dead pixel detection/hiding I put in a while ago when I was doing something with raw from a 60D. I'm glad it's still working.

I just tried the new version 1.1.3 of MlRawViewer, exported a couple of DNG sequences, and then imported the DNG sequences into Premiere Pro CC.  Unfortunately, the highlights are pink, making footage unusable in Premiere unless there are no strong highlights in the footage.  I tried both MLV and RAW files and both had the same issue.  I can send you sample DNGs exhibiting this issue if you find it useful to your development.  You are soooo close to providing a post processing tool that allows working directly with the DNGs in Premiere.

I know about the pink highlights. As I understand chmee is also seeing similar results with his raw2cdng and Premiere.

The same files works fine in ACR and Resolve. In an overexposed area (with at least one channel - usually green first - clipped) I see those tools reconstructing highlight detail as white levels. In MlRawViewer itself, I take a slightly different approach to try and recover some of the original colour in the highlight areas (though that depends on having the right white balance for the highlight).

Premiere appears to suffer a similar problem that MlRawViewer had before it had highlight recovery - the white balance colour correction is applied afterwards so the clipped channel(s) take on the wrong colour. This is usually pink/magenta if green was clipped. 

Reconstructing the raw data is the task of the tool reading the DNG files. In this case, MlRawViewer is not doing anything to the original raw bayer data apart from padding it with 2 zero high bits and writing it to a series of DNGs with the correct black and white levels, along with other meta data taken from the MLV file if available.

To me, it's appears the problem is with Premiere's highlight reconstruction and white balance. But since that's a tool you are paying money for, you shouldn't have any trouble raising the defect with the developer and getting it quickly fixed. Right? ;-)

1.1.3 now available

• Now exports 16bit DNGs with per-frame metadata. Should directly import into PremierePro as well as Resolve/AE.
• S key switches between different anamorphic squeeze ratios (viewing only)
• MOV/DNG icon made clearer
• Mark line red instead of white so it shows up in bright scenes

See top post: http://www.magiclantern.fm/forum/index.php?topic=9560.msg91165#msg91165

@ baldand, could you add the keyboard keys functions to your first post please? somewhere at the end of your first post, for example ... The way you have them in your Readme file(archive) gives a much easier view of which does what.

Done now.

I shot a video on the Canon 50D.
I played it off the cf reader and the playback was smooth.

Now that I took the files off the cf card and put them on a hard drive, the play back is slow, skipping, and lagging.

It might help to have a few more details:
- What kind of computer you have (ago, CPU, GPU, memory, hard disk size & interface type - IDE or SATA)
- Is the card reader USB2 or USB3?
- What resolution and frame rate are you recording at?

It sounds as though your hard disk could be slow at reading for some reason. The needed read speed depends on the video resolution and frame rate, but if you have less than 50Mbyte/second reads speed, you can experience the symptoms you describe.

Another possibility could be too little RAM, e.g. causing the system to use swap files, though that seems less likely if it works from card reader ok.

hi Guys,

I start to use mlraw viewer to export to .mov,its worked perfectly.
since yesterday,its impossible to export anymore,the program freeze or crashes.
In the meantime I changed the preset on camera,and installed some program on the mac,like raw2dng.Since it doesnt work I re-install Mlraw viewer but no better.
I use a macbook air last version,I was wondering if its something to do with CPU?
the files to convert are on the local SSD and I try from an USB3 HDD.
the export button sometimes is vivid red,sometimes I cant click it,any case nothing exports.
Also when I play the mlv,it freeze after 3-4 seconds...Any help is welcome,cheers!

If you are using 1.1.2, check your target directory is valid -> Press W key to bring up the target directory or choose a new one.
Also if you want MOV exports, make sure the "MOV" icon is displayed, not the "DNG" icon.

If nothing else works, you could try deleting the hidden directory ".mlrawviewer" in your home directory and see if that helps.

My dngs are coming out Canon EOS instead of canon EOS 7D. Also 2 files got stuck converting... I tried to run a bunch of MLrawviewers at once, bad idea.

Changing the tag with exiftool gui causes a lockup after a certain number of files, gah. Its > ~420 files, exiftool hangs, then gui hangs.

Re the model, I have put the correct make/model writing into git (however I have a personal problem - most of my MLVs made with TL for 7D have "Failed to get properties." as the model name 

I wouldn't recommend multiple DNG exports from different mlrawviewer instances at the same time unless they are all using different disks for both read and write (or if the read disks together would be much slower than the write disk).The operation is mostly disk I/O bound because it uses different threads for reading and writing. Because of that it should work best when the read & write disks are different. Of course if you are converting between 2 fast SSDs it might become CPU bound.

the dng-support in premiere is a shame - its fixed on bmcc files. so best would be converting to 12bit/log (with the inverted table i linked) and leaving reconvertion to 16bit inside premiere with the same delinearizationtable..

Actually it seemed to work fine for me with the changes I listed. There was no need for any actual data conversion apart from the cheap and lossless 14->16bit padding. The LinearizationTable I made was just integer values from 0 to 2^14-1, so, still linear. 

That's the way RAWMagic handles it, but despite all the CDNG files being 16bit, Premiere still reads the files wrongly (they will import and playback, but there is a nasty pink overcast on all the footage which makes it kinda unusable).

Something is going on with the way Premiere is reading files generated by the Blackmagic Cameras vs. the files generated by ML CDNG.

PM me if you want me to give you some Blackmagic CDNG files to study.

I got hold of PremierePro for testing and I have the basics working now thanks to chmee's tips. All that was needed was 16bit DNG, a non-default LinearizationTable (PP seems to need this even though it *should* be optional - surely Adobe's bug) and BlackLevel/WhiteLevel tags to be Short instead of Long (another Adobe bug - both Long and Short are allowed in the DNG spec).

@baldand
the mystery behind cdng for premiere are not really the cdng-tags, but debiasing and maximizing values. possibly the delinearizationtable from bmcc-files can help. AND: of course the dng-files should be converted to 16, 12 or 10bit, because 14bit is no typical video-bitdepth, only supported by resolve (and acr as a photo->converter)

https://infogr.am/bmcc-delinearization-curve?src=web
and
http://www.magiclantern.fm/forum/index.php?topic=7122.msg105445#msg105445

regards chmee

Thanks chmee. That should be enough info for me to work out what's needed for Premiere. I expect it's the 14bit data. I can easily make an option to pad to 16bit (though that will waste disk space)

However, as you move forward on refining this, could I lobby you to consider CDNG instead of regular DNG? The reason why is that Adobe is likely about to fully support CDNG in Premiere Pro, and thanks to Blackmagic it's quickly becoming the de-facto standard for raw video files. My understanding is that the difference between DNG and CDNG is just in the header information which includes specific information as it relates to video.

Of course. I don't see any reason not to aim for making CDNG-compatible DNG sequences. I already set the FrameRate tag from the CDNG spec, and IIRC there are only 3 other (optional) tags there that might be needed.

(DNGs embedded in MXF files  - the alternate format of CDNG - is a totally different story I won't be doing that.)

@midphase could I ask you (or somebody else) to try out the DNG sequences being produced currently in Premiere Pro (which I don't have access to) and tell me what doesn't work or what needs adding? I've already tested in Resolve 10 and After Effects and they seem to import the sequences ok.

i have the latest win 7 ver 1.1.2. Is it just me or is anyone else having trouble reading the new mov/dng icon? I have yo lean in closer to read it all the time.  Can you make the text whiter and a little larger? The other items are easy to see since the are bigger simple graphic symbols.

Yes, I think you are right there. Plus there is not much other difference between the 2 states. I'll try to improve that in the next release.

It would be awesome if you could include a JPEG preview for those people on mac 10.9 mavericks. Not entirely sure how that works

Do you mean thumbnails in the individual DNG files, or something else?

i dont have that problem.  my pro is i cant see the mark In and Out white bar clearly when i play clips that i shoot in bright sunlight.

Ouch. A workaround for now would be to reduce the brightness of the clip while you are marking. I'll try to give the mark bar more contrast in a later release.

[*EXPERIMENTAL*]

1.1.2 is now available for Mac and Win: http://www.magiclantern.fm/forum/index.php?topic=9560.msg91165#msg91165

This includes a couple of new features:

1. Drag and drop MLV/RAW/DNG files to the existing player window to play the new video. Drag a WAV file to play with the current video.

2. *EXPERIMENTAL* DNG export as alternative to MOV.
To use DNG export, click the MOV/DNG icon to say DNG, or press D. Then press E or the red button to export the current marked range to a target directory. DNG exports are added to a queue which is processed immediately. There can be multiple items in the queue either from the same file or different files. A WAV file is created in the same directory as the DNG sequence.
The DNGs are currently light on metadata and always claim to be 25FPS. I will add more detailed and correct metadata in a later release. Please let me know if you have any need for any specific tags in the DNGs.
To set the target directory press W. The setting is persisted so you don't need to set it every time if you always use the same one.

3. If you run the program without giving it a file, it will not launch a file dialog asking you to choose one. This avoids the problem many had seen where the app could not be run just by clicking on it. Contributed by ItsMeLenny

4. Mark setting logic has been slightly refined - when you try to move the current range you can set the new marks in any order.

thanx for the reply. about the play/pause icons, it is obvious that the clip is playing by seeing it playing in the window and the frame numbers are changing as it plays to also confirm it is playing. So, do we also need to see the play arrow icon to know if a clip is playing? :-)

Yes, for the play icon this is the case. But it isn't for the other state icons like frame dropping & colour mapping. Those need to show the current state. The design decision I made was for all the icons to consistently show the current state.

Questions. Anybody experienced problems with "unexpected error -10813" when trying to "change all" ponting to the app?
I,m on osx 10 mavericks on a macbook pro. I have older builds present as well but can,t seem to find any cache preferenses to clean. Maybe that,s what needed? Suggestions?
How do I play dng files by he way? Tried drag all of the to the app and also only the first one. Didn,t work.

*tried to "kill all" preferences and then open with but the app still doesn,t seems to be able to "change all" for a simplier workflow. Maybe not working on mac

Oh sounds bad. Have you tried moving or deleting the old versions of the tool? I'm not a heavy mac user, but this associating has been working for me with Mavericks, also with DNGs.

You should be able to drag one DNG file from a sequence onto the app icon, or doing "Open with". If that's not working, it may indicate another problem with the file associations or installation, since the app had to declare it handles .DNG files.

Maybe you should just delete all copies of MlRawViewer (including emptying the trash..) and reinstall 1.1.0?

Sound play in sync when playing in the app, but after exporting to prores it looses sync.

Thanks for the report. Some more information could be useful:
- Is it obviously out of sync from the start (offset), or does it get more out of sync as the file plays (stretched)?
- Have you tried playing the ProRes file with different players and see the same thing?
- If you process the same file with e.g. mlv_dump and some DNG tool, is it in sync without any manual offset?
- Did you convert the whole file, or just a marked subsection? Is it out of sync in both cases?
- Which camera was the file made with?

(You can use the V/B/N/M keys to slide the audio, but I know that's not ideal if every track is out of sync for you)

[current app state]

the play and pause icons seem backwards to me. i expect the pause icon to show while the clip is playing and the play icon to show when the  clip is paused. most app works like that.

I understand, as I had to think for a while when designing this.

The model I went for is for all icons to consistently show the current app state rather than another state you can get to by clicking that icon. That means the play icon shows the arrow when playing, not when paused.  It also means the icons can have more than two states (like the colour mapping).

when I click on the exe file, i expect the app to open up and nothing happens. i see that i can drop a file onto that exe and then it works. am i missing something on the way this works? i have mlv files in separate folders. do i have to copy and paste your exe to every mlv folder and then drag n drop on it? that seems to be a tedious way. isn't here a way to just open your app and then use an import function of some sort for single files? groups of files?

On windows you should set the file associations for .MLV, .RAW and .DNG to the mlrawviewer.exe so that you can just click on the files to be opened automatically in the the viewer.

Make one copy of the MlRawViewer_1_1_0 directory, for instance in "c:\Program Files\"

Then find an example MLV/RAW/DNG file, right click, select "Open with" then "Choose default programs..." from the sub menu. Click "browse" on the dialog that opens, then find and select the mlrawviewer.exe.

In case this doesn't work for you (I've had problems in Win7), find the mlrawviewer.exe, right click and "Pin to taskbar", and drag the MLV/RAW/DNG file to the taskbar icon to teach Windows about the association. After that, you should be able to set the defaults.

again, excuse my lack of understanding on this, but what else does the app do? It seems to only play back mlv raw files, some brightness control and color adjust. but then what? can i extract dng's? make prores proxies? i don't see any more menu items , drop downs or right click menus anywhere.

It's primarily a real time player for MLV/RAW/CDNG files, which does not require the files to be converted first.
It can also convert marked regions of the files to ProRes (click the red button or E key).

More features may come in later versions.

Very cool!!Can somebody explain what the third icon from top does?Only the pattern needs fixing,after that no more adobe for me!

When the icon is a clock it will try to play in real time, dropping frames if needed. This mode is needed for playing audio
When the icon is a frame, it will play all frames so you don't miss anything (useful for checking if any frames are bad). This mode cannot play audio.

When you say "pattern", do you mean fixed pattern noise, or stripes, pink dots? Or something else?

why does it keep connecting to bit bucket?

To check if a newer binary release is available. If there is, it shows an update icon. It shouldn't check more than once a day though.

Heads up: I had a go at 1.1.0 binary builds for Mac and Windows. See the top post of the thread for links: http://www.magiclantern.fm/forum/index.php?topic=9560.msg91165#msg91165

Headline feature is a real UI (with icons).

DNG sequences should now be playable on Mac.

I sneaked in IN/OUT marks, so encoding now uses the marked range rather than the current frame.

Please report problems to the bitbucket issues pages: https://bitbucket.org/baldand/mlrawviewer/issues

Thanks for the great feedback!

General comment: the bitbucket page has an issues list for defects and enhancements, and you can also vote up other issues already posted. See https://bitbucket.org/baldand/mlrawviewer/issues

Testing with MLV and audio (ML Feb 10 build) on OSX (latest): for a ~23GB recording (> 4GB+exFat), during playback the audio cuts out at 1:50; when encoding to ProRes it cuts out around the same time. The fully extracted .wav file is OK (4m30s). Full audio file matched video correctly in PPro CC.

There are 2 wav files written. One has the name of the original MLV, and the other the name of the MOV. Were they both of the correct length?

Unfortunately my 7D doesn't support exFat, so I may have some problems reproducing this one, but I'll see what I can do.

CPU utilization shows ffmpeg running around 143% and the host app running at 97%. If there's any way to multithread deBayer, that would help speed up ProRes output (tested on 4-core MBP (late 2013- fast CPU option)). My 2010 MacPro 12-core with AE+ACR runs 1/4 real-time (full CPU utilization): I believe this MBP has faster CPUs and SSD, so I don't think the MacPro will run your tool faster (will test). Would be nice if this tool ran as fast or faster than ACR+AE (MlRawViewer is about 1/2 as fast currently).

Linux uses OpenMP for this, but neither of the Mac or Win compilers I have support OpenMP for multithreading the AMaZE debayering. I hope to rework the multithreading in a later version so that it's cross platform and demosaicing is faster on multicore machines. Also there is one post processing step I can push to the GPU in most cases, which might speed it up slightly.

The AMaZE deBayer looks very good- appears to do a better job than ACR (AHD variant?), especially on super high contrast edges (where ACR looks strange- color/edge issue).

Really?! That's interesting to hear. I haven't closely compared ACR output.

Vertical line removal, denoise, and perhaps sharpening during deBayer would be nice to see (looks reasonably sharp as is, and sharpens OK in post; might be possible to get higher quality sharpening as part of deBayer step (e.g. ACR sharpening looks better than post sharpening).

Vertical line removal I will try to address in the future for sure, since I suffer a lot of that on 7D. Sharpening might be quite easy to add later. Denoise would be great, but may be too big a challenge to do well.

A film-like highlight roll-off option would be more useful than the current tone-mapping option...

Much more flexible colour handling is a target for a future version.

A command line option to allow direct converting to ProRes would allow for scripting/batching (tried calling the compiled exe from the shell- didn't work).

That should be posted to the issues list. I think there may be a few different users for that so it would be good to capture all the requirements for command line options different people would like.

Your tool brings the 5D3 one step closer to being a (useful) baby Alexa

Thanks! 

Hi Baldand. Really like your app. Looking forward for your upcoming achievements. One question on my mind is about batchfunctionality. If It,s like now a simple inpoint and to start encoding from current frame would do fine. However, the question is what to do next when the conversion starts. Instead of waiting for conversion to finish I want to get on with the rest of my files. Would be awesome to maybe queue the files and when all settings are applied to all files being able to start the conversion queue from there. This might be too hard to implement and maybe this little "dream" have been discussed already.
Thanks again for all your work on this. Still the best rawviewer by far.

Batch encoding or DNG conversion will first need a browser to allow files to be picked.

The browser is on the enhancements list https://bitbucket.org/baldand/mlrawviewer/issue/19/browser and will probably be the major focus of the release after the coming 1.1.0

After that is in, then I can look at a batch queue, and I agree that this will need the in/out point recorded in the queue entry.

I am also looking forward to the next release, and please add in and out points!! I think that would be a main feature for converting stuff

I was thinking about this - would it be enough to be able to add a single mark in the timeline and then use the current frame as the second (implicit) mark when starting encoding?

If the current frame would be before the mark, encoding would run from current frame until the mark.
If the current frame would be after the mark, encoding would run from the mark to the current frame.

Or do you see some use case for having both in and out marks independent of the current frame?