raw2mlv processing

Started by Danne, November 10, 2019, 03:45:44 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Danne

Yes, white balance is incorrectly added to the dng file. I checked a dng and this is what in the As Shot Neutral tag from my particular file:
As Shot Neutral                 : 0.0004656612873 1 0.0004656612873
Good thing is that it´s a nice, usable dng otherwise.

ilia3101

Quote from: cmh on November 14, 2019, 02:58:32 PM
YCbCr according to this:
http://lclevy.free.fr/cr2/#sraw
edit: it is encoded in YCbCr 4:2:2 and needs interpolation for the Cb and Cr values.



Thanks

Quote from: cmh on November 14, 2019, 05:51:41 PM
I'm reading this right now:
https://github.com/lclevy/libcraw2

"Libcraw2 has two kind of interfaces:
an C API, mainly used by crawtool, to offer CR2 data handling in a Camera agnostic way
an Python extension, PyCraw2 a simpler API which goal is to offer all required functions to render RAW data into a 16bits RGB image, either from RAW (RGGB) or sraw/mraw (YCbCr) data."

Looks liek a nice library. Still need to decide on an official way of including other types of image data like RGB in MLVs, only bayer images are properly supported.

cmh

Libcraw2 doesn't seem maintained but the author is aware of magic lantern's forum. Bugging him on github or tweeter could be worth the shot.  Maybe Rawtherapy's dcraw instead of libraw is a better solution since it is an active project (it's not a lib tho, not sure).

masc

I tried to integrate RAW2MLV to MLVApp. To use this, compile and install raw2mlv like this (path of MLVApp executable):

If MLVApp finds this executable, it will show this action:

Triggering brings:

Select single picture or a sequence folder, then click "Transcode and Import", choose a destination folder. Then it will need some time - WAIT! - and the converted files show up in the session.

TESTED ON OSX ONLY! Linux might work too, on Windows I expect some additions. AND: only the "good" use case works - no error handling at all. So if you do s*** you will get s***. :P
5D3.113 | EOSM.202

Levas

Found out about this raw2mlv today  :o
Haven't tried it...yet, but already:
Great, nice & thanks  :D 8)

cmh

Darktable use rawspeed and is able to open sraw/mraw just fine btw.

@masc
Nice. People will use a gui, that's great.

cmh

@masc Yeah, it's working great on Windows if I change line 1344 in platform/qt/MainWindow.cpp to

#ifdef Q_OS_WIN
    //raw2mlv available?
    if( !QFileInfo( QString( "%1/raw2mlv.exe" ).arg( QCoreApplication::applicationDirPath() ) ).exists() )
        ui->actionTranscodeAndImport->setVisible( false );
#endif
#ifdef Q_OS_LINUX
    //raw2mlv available?
    if( !QFileInfo( QString( "%1/raw2mlv" ).arg( QCoreApplication::applicationDirPath() ) ).exists() )
        ui->actionTranscodeAndImport->setVisible( false );
#endif


Easy peasy.

IDA_ML

Just tested RAW2mlv.exe with a CR2 file from the 100D.  Works perfectly on my M6600.  I can't believe my eyes, how well the skin tone white ballance works on portrait stills, just with a single mouse click.

Obviously, sRAW and mRAW files are the trouble makers.  It would be nice if these two formats could also be processed with RAW2mlv - very convenient for wedding and event shooting where the huge full-size RAW files are not needed.  Workflow is much faster and saves a lot of disk space.

Masc,

I think, now is the time for adding two more features for stills processing to MLVApp:

1)  Export to JPEG with minimum compression, something like 11 in Photoshop.  An adjustable compression ratio would even be better.  Right now, we only have the PNG and DNG export formats for current frame stills export which requires JPEG conversion in an external software.

2)  Adding a crop tool with adjustable aspect ratio similar to the one in ACR.   

Is this doable?

garry23

...and don't forget the idea of being able to do statistics, eg mean, i.e. create an image from statistically merging the x images in an mlv, as can be done in Ps.

This, of will help in creating an mlv focused tool to simulate LEs in post.

If this could be done, it would be a fantastic feature.

;)

masc

Quote from: cmh on November 15, 2019, 12:01:54 AM
@masc Yeah, it's working great on Windows if I change line 1344 in platform/qt/MainWindow.cpp to

#ifdef Q_OS_WIN
    //raw2mlv available?
    if( !QFileInfo( QString( "%1/raw2mlv.exe" ).arg( QCoreApplication::applicationDirPath() ) ).exists() )
        ui->actionTranscodeAndImport->setVisible( false );
#endif
#ifdef Q_OS_LINUX
    //raw2mlv available?
    if( !QFileInfo( QString( "%1/raw2mlv" ).arg( QCoreApplication::applicationDirPath() ) ).exists() )
        ui->actionTranscodeAndImport->setVisible( false );
#endif


Easy peasy.
That's not all... OSX doesn't work this way. And about Linux I am not sure.

Quote from: IDA_ML on November 15, 2019, 01:44:57 AM
I think, now is the time for adding two more features for stills processing to MLVApp:

1)  Export to JPEG with minimum compression, something like 11 in Photoshop.  An adjustable compression ratio would even be better.  Right now, we only have the PNG and DNG export formats for current frame stills export which requires JPEG conversion in an external software.

2)  Adding a crop tool with adjustable aspect ratio similar to the one in ACR.   

Is this doable?
1) I can offer the standard Qt jpg export for single frame export.
2) This is still very difficult. And I don't think it will take longer than 3 days until someone asks for free turning, perpective stretching, etc. ... and all this is a very huge amount of work.

@garry23: could you please explain a bit more? I don't understand what you are asking. What are LEs? Do you mean an averaged frame? This is possible already.
5D3.113 | EOSM.202

IDA_ML

Quote from: masc on November 15, 2019, 08:15:42 AM
1) I can offer the standard Qt jpg export for single frame export.

Thanks a lot, Masc, for considering this.  Yes, please implement it.  It would be an important step towards single CR2 files processing in MLVApp.

garry23

@masc

Forgive me, I feel an interloper here, as I don't do any video stuff.

Currently, if I don't have my NDs with me or the wrong one, I can simulate a longer exposure (LE) by taking x images of y sec exposure, to cover the same motion blur time as one exposure at x times y seconds.

I then send all the images to Photoshop as layers and use the Ps statistics feature to blend them all together with mean or median.

I'm hopeful that what is going on with raw2mlv will give me a single tool that allows me to do the same in the mlv post processing app, eg and ideally

- as with Dual post processing, seamlessly select x images in LR and export to the mlv post processing app, via the raw2mlv tool
- post process the mlv file, ie merge with mean or median statistics
- return to LR

Or do the above via a standalone method, ie not coupled to LR.

I also hopeful the current developments will soon be available without compiling on winX  ;)

Bottom line: I'm really looking forward to using this approach.

Thanks for all those involved in pushing the various tools along.

Cheers

Garry

cmh

Quote from: masc on November 15, 2019, 08:15:42 AM
That's not all... OSX doesn't work this way. And about Linux I am not sure.

That was to be expected, I know nothing about qt or c++. QString QFileInfo::baseName () const seems to be a good way to strip the extension but I wouldn't know where to put any of that. I know you'll have to test for input validation later anyway. Meanwhile, I'll just replace raw2mlv by raw2mlv.exe and reupload it real quick.


histor

Quote from: garry23 on November 15, 2019, 10:39:23 AM
- post process the mlv file, ie merge with mean or median statistics

Already implemented in mlv_dump. Simple average only. Not yet included in the GUI.

-- Image manipulation --
-a                  average all frames in <inputfile> and output a single-frame  MLV from it
--avg-vertical      [DARKFRAME ONLY] average the resulting frame in vertical direction, so we will extract vertical banding
--avg-horizontal    [DARKFRAME ONLY] average the resulting frame in horizontal direction, so we will extract horizontal banding
-s mlv_file         subtract the reference frame in given file from every single frame during processing
-t mlv_file         use the reference frame in given file as flat field (gain correction)

masc

@garry23:
You can do all this with this MLVApp revision I commited yesterday.
- transcode and import all your cr2 pics into one MLV
- export the MLV as averaged MLV
- open the averaged MLV (this one has one single frame)
- apply dualiso (or not)
- do your settings
- export e.g. PNG or DNG (for LR)
5D3.113 | EOSM.202

garry23

@masc

I owe you a beer  :)

I guess the only thing I now need is to wait for standalone, compiled Win version  ;)

Once again, thanks for all your work.

Cheers

Garry

masc

Haha. Try the one from @cmh. Thanks.
5D3.113 | EOSM.202

garry23

@masc @cmh

As I say, I'll wait.

Just tried it, but running MLVapp.exe and I get the qt platform plugin error.

As I say, I'll wait until someones says it runs without 'fiddling'  ;)

Cheers

Garry

cmh

Quote from: garry23 on November 15, 2019, 12:37:32 PM
As I say, I'll wait until someones says it runs without 'fiddling'  ;)
I respect that.

I'll still create a virtual machine real quick just to see if I can reproduce that error.

Edit: in the meantime, you can still use Ilia3101's build of raw2mlv and use it as shown in the video you've requested.

Edit: nvm the video isn't about long exposure.

Quentin


cmh

You'll probably have to wait for an official release unfortunately.
I might post another link later (but it requires recompiling qt to get a static version, which takes some time).

masc

No need for a static version. Just search and copy the missing dlls from your Qt directory.
5D3.113 | EOSM.202

cmh

Yeah, but it won't solve garry23's error, I tried on a fresh vm (dropping qwindows.dll doesn't work).
"could not find or load the Qt platform plugin "windows" in"
If people have another qt app installed they might not get this error, I don't know for sure.
So there's few potential options that could solve this issue that I'm aware of:
- using an installer to add QT_QPA_PLATFORM_PLUGIN_PATH to their system environment (and include the content of qt's \plugins\platforms with the rest of the DLLs) and I'm not sure it would work
- maybe using visual studio and I'd rather not.
- building a static win64 version.

masc

You need about 10 or more dlls from Qt dir. Get some inspiration from offical (non static) version. MLVApp won't compile with VisualStudio.
5D3.113 | EOSM.202

cmh

Sure.
https://github.com/cmhamiche/mlvapp_win/raw/master/MLV.App.v1.9.Win64_raw2mlv.zip
edit: added missing imageformats and platforms folders plus dfort's pixel maps.
I hope it works fine. Meanwhile qt's still compiling.

edit: New build with single frame jpg export available.
I'll put the link in MLV App's thread (Sony raw will have to wait).
https://www.magiclantern.fm/forum/index.php?topic=20025.msg222468#msg222468