Magic Lantern Forum

Developing Magic Lantern => General Development Discussion => Topic started by: Ilia3101 on August 08, 2019, 12:09:00 AM

Title: The MLV format
Post by: Ilia3101 on August 08, 2019, 12:09:00 AM
Some time ago I remember hearing that the MLV format is getting extended with more features in Google summer of code for Apertus cine camera project. Did that ever happen?

A couple more questions:

1. Would developers consider adding new blocks to the official MLV format?

2. Where can I see most up to date version of mlv.h?
Title: Re: The MLV format
Post by: Ilia3101 on August 16, 2019, 03:42:11 PM
Ok never mind about the new blocks idea, raw_info has the matrix field I wanted. Though a "TEXT" block seems like a nice idea to me.

Anyway, I found mlv_structures.h on MLV post, and it is under LGPL (what I was hoping to find). However it does not contain raw_info structure, is that because raw_info is from CHDK and is full GPL? (I think I heard about this ages ago)

I am thinking there needs to be an mlv library (somthing like "libmlv"), that developers can use to add MLV support to their propreitary software. It would need to be licensed with LGPL or MIT, and written from scratch. But the GPL raw_info block is currently the main concern. Could it be done using byte offsets to access fields? - someone else calculates the offsets and communicates them to me.

I genuinely want to spread MLV as much as possible. No, I am not planning to sell any propreitary software with MLV support. I just want MLV to be a more widespread raw video format, I think it is perfect suited for that.

But first, I want to make a raw to MLV converter that converts other raw video formats or photo sequences to MLVs, so I can use MLV App with them. No licensing issues here as it will simply be GPL.


Anyone have any thought or advice about this?? About anything... The licensing issues, MLV for other cameras... bad or good ideas?
Title: Re: The MLV format
Post by: Ilia3101 on August 18, 2019, 01:27:07 AM
Ok let me condense the last verbal spaghetti post down to a couple of questions:

1. Is the MLV format suitable to use as a container for other raw video cameras' footage? (thinking of making a converter)
2. Would magic lantern community support the idea of a library (something like "libmlv") being created, with a license that does allow it to be used in proprietary software
Title: Re: The MLV format
Post by: Danne on August 18, 2019, 08:53:59 AM
But first, I want to make a raw to MLV converter that converts other raw video formats or photo sequences to MLVs, so I can use MLV App with them.
How cool is that. I guess dng or tiff into mlv would be really convenient here. Or even jpg h264 maybe?

The library also seems like a good idea but I have little experience i this field. Maybe contact g3gg0 and a1ex directly too? Would be nice to see resolve and premiere implementing mlv support ;).
Title: Re: The MLV format
Post by: Ilia3101 on August 18, 2019, 05:17:44 PM
How cool is that. I guess dng or tiff into mlv would be really convenient here. Or even jpg h264 maybe?
I saw in MLV headers that it can have h.264, but IDK if it has ever been tested or put to use (MLV App certainly doesn't handle h264 MLVs :D). DNG or cr2 would be easiest to convert to MLV though, especially if it is one of the already supported ML cameras, and masc found some nice libraries we could use.

The library also seems like a good idea but I have little experience i this field. Maybe contact g3gg0 and a1ex directly too?
Yeah I guess I should reach out to them. I'm just not sure about the licensing, I remember loads of something about Martin Herring's Footage app (Footage was completely closed, this library WILL be open source, but I would want it to be usable in proprietary software). As MLV headers are LGPL it should techincally be allowed... as long as raw_info can be dealt with, maybe using byte offsets

Would be nice to see resolve and premiere implementing mlv support ;).
It would be so satisfying if any of the industry giants were to implement MLV support. Maybe this is wishful thinking though :-\
Title: Re: The MLV format
Post by: Ilia3101 on August 21, 2019, 05:45:03 PM
Ok I found this https://github.com/apertus-open-source-cinema/opencine/blob/master/Source/OCcore/Image/mlv_structure_mod.h

It says it's LGPL, but contains the raw_info structure. Is that a mistake on their part?

Also I will forget the library idea for a bit, as I have heard from Apertus that their GSoC student will soon be finished with MLV format improvements... don't wanna make something already obsolete.
Title: Re: The MLV format
Post by: g3gg0 on August 25, 2019, 07:51:26 PM
back from holiday,

anything i could help with?

It says it's LGPL, but contains the raw_info structure. Is that a mistake on their part?

IIRC we aligned that that little part of the structure is fine to made LGPL to get the exchange format accepted.
only if even professional tools are allowed to use the structures, you can undercut commercial products with open source mindset ;)
Title: Re: The MLV format
Post by: Ilia3101 on August 26, 2019, 02:49:27 PM
back from holiday,

anything i could help with?

I'll probably have something soon :)

IIRC we aligned that that little part of the structure is fine to made LGPL to get the exchange format accepted.

Glad to hear

only if even professional tools are allowed to use the structures, you can undercut commercial products with open source mindset ;)

Yep, always fun to do that :D
Title: The MLV format
Post by: DeafEyeJedi on September 07, 2019, 08:59:09 AM
Following this thread as it seems an important topic to keep up with. Fun facts. Thanks guys!
Title: Re: The MLV format
Post by: extremelypoorfilmaker on September 08, 2019, 06:44:24 PM
Following :)
Title: Re: The MLV format
Post by: Ilia3101 on October 01, 2019, 08:13:42 PM
The new(ish) RAWC block does not appear to be in the official mlv_structure.h LGPL file
Title: Re: The MLV format
Post by: Ilia3101 on October 11, 2019, 10:12:59 PM
(https://i.ibb.co/QnVLCYV/Screenshot-2019-10-11-at-21-11-30.jpg)
Title: Re: The MLV format
Post by: masc on October 11, 2019, 10:15:36 PM
Wicked! What have you done?  ;D
Title: Re: The MLV format
Post by: Walter Schulz on October 11, 2019, 10:17:32 PM
Please compress this file. It doesn't need 6 MByte to show what you want us to see and mobile users data plans will suffer.
Title: Re: The MLV format
Post by: Ilia3101 on October 11, 2019, 10:24:01 PM
Wicked! What have you done?  ;D

Almost ready to release, a tool for converting raw photo sequences to MLV. Should be useful for timelapses, or converting BlackMagic CDNG footage to MLV.

This will work much better once a colour matrix block finally gets added to the MLV format. It is currently relying on RAWI.raw_info.color_matrix1 and this commit (https://github.com/ilia3101/MLV-App/commit/9f1377b802a0c5a72e0849392b645e2337749c3c).


@a1ex @g3gg0 can I call a new MLV reading/writing library "libMLV"?
Title: Re: The MLV format
Post by: Danne on October 12, 2019, 05:17:43 AM
Almost ready to release, a tool for converting raw photo sequences to MLV. Should be useful for timelapses, or converting BlackMagic CDNG footage to MLV.

This will work much better once a colour matrix block finally gets added to the MLV format. It is currently relying on RAWI.raw_info.color_matrix1 and this commit (https://github.com/ilia3101/MLV-App/commit/9f1377b802a0c5a72e0849392b645e2337749c3c).


@a1ex @g3gg0 can I call a new MLV reading/writing library "libMLV"?
Wow!
Title: The MLV format
Post by: DeafEyeJedi on October 12, 2019, 06:31:40 AM
Indeed that’s wicked @Ilia3101! 8)
Title: Re: The MLV format
Post by: a1ex on October 12, 2019, 08:11:35 AM
@a1ex @g3gg0 can I call a new MLV reading/writing library "libMLV"?

Green light from my side.
Title: Re: The MLV format
Post by: g3gg0 on October 12, 2019, 09:49:43 AM
@a1ex @g3gg0 can I call a new MLV reading/writing library "libMLV"?

absolutely.
thanks!
Title: Re: The MLV format
Post by: IDA_ML on October 12, 2019, 11:20:06 AM
Almost ready to release, a tool for converting raw photo sequences to MLV. Should be useful for timelapses, or converting BlackMagic CDNG footage to MLV.

Ilia3101,

What you are planning to do is revolutionary!  Many people have already sensed the taste of the RAW-video magic and will not go back, no matter what.  Many of them, tired of waiting for more capable and stable ML builds for their Canon cameras, bought the BMPCC4k to continue filming in RAW.  Others are waiting for new RAW-video capable cameras to come out on the market.  All these people look forward to using MLVApp for post processing their RAW videos.  This is where your converter would be extremely useful! In my opinion, MLVApp is better, more intuitive and easier to work with than all other NLE available and is also free.   Personally, when grading my most important videos, I prefer it to DaVinci Resolve despite MLVApp's low render speeds.

I keep my thumbs pressed for you to succeed!
Title: Re: The MLV format
Post by: Ilia3101 on October 12, 2019, 03:36:31 PM
@a1ex @g3gg0 Thanks! glad you guys don't mind me taking the name.

Can I ignore MLV_VIDEO_CLASS_H264/MLV_VIDEO_CLASS_JPEG/MLV_VIDEO_CLASS_YUV? I think they have never been implemented is that right?



What you are planning to do is revolutionary!  Many people have already sensed the taste of the RAW-video magic and will not go back, no matter what.  Many of them, tired of waiting for more capable and stable ML builds for their Canon cameras, bought the BMPCC4k to continue filming in RAW.

Yes I'd really like to create a solution for converting all camera raw formats to MLV, including red/arri/blackmagic if possible.

I have done some research on video raw formats:

So MLV is really the best option, it is truly open, actually free, simple enough to be understood just by looking at mlv.h, and has many original implementations (ML itself, mlv_dump, mlvproducer, mlv app, fastrawviewer, martin hering's app ...)

Might have something later today to release. The writing functionality of libMLV is coming first. Reading a little bit later.
Title: Re: The MLV format
Post by: g3gg0 on October 12, 2019, 03:54:45 PM
Can I ignore MLV_VIDEO_CLASS_H264/MLV_VIDEO_CLASS_JPEG/MLV_VIDEO_CLASS_YUV? I think they have never been implemented is that right?

you should honor the flags, i.e. returning smth like LIBMLV_ERR_UNSUPPORTED_ENCODING or similar
Title: Re: The MLV format
Post by: Luther on October 12, 2019, 03:55:23 PM
@Ilia3101 will you release under permissive license? If you release under GPL is think MLV will not be adopted by some companies. If we want global support for MLV, the library should be permissive (I suggest ISC license (https://en.wikipedia.org/wiki/ISC_license#OpenBSD_license)).
Title: Re: The MLV format
Post by: Ilia3101 on October 12, 2019, 03:57:49 PM
you should honor the flags, i.e. returning smth like LIBMLV_ERR_UNSUPPORTED_ENCODING or similar

Okay will do!

@Ilia3101 will you release under permissive license? If you release under GPL is think MLV will not be adopted by some companies. If we want global support for MLV, the library should be permissive (I suggest ISC license (https://en.wikipedia.org/wiki/ISC_license#OpenBSD_license)).

Yes, I think I will release all new libMLV library code with MIT. I am making it completely original.

Do you think MIT is good? If there is a better option that companies like more, I can use that, just let me know. Edit: just realised you suggested ISC. What is the advantage over MIT?

I thought about LGPL, as it's fully permissive in how it can be used, and mlv_structure.h file is LGPL anyway, but for some reason companies seem to be scared even of LGPL.
Title: Re: The MLV format
Post by: DeafEyeJedi on October 12, 2019, 07:41:26 PM
Do you think MIT is good? If there is a better option that companies like more, I can use that, just let me know. Edit: just realised you suggested ISC. What is the advantage over MIT?

I second this too re: advantage w ISC over MIT. Great call!

Regardless just like those mentioned... this is beyond revolutionary and can’t describe how much MLV_App has pried me away from major NLE’s (for good reasons) in despite of the rendering times and yet that’s the beauty of open source!
Title: Re: The MLV format
Post by: garry23 on October 12, 2019, 07:43:05 PM
At the risk of making a fool of myself I thought I would flag an interest here, ie as a non video guy.

The use case for me is taking a series of .cr2 images to simulate a long exposure, convert them onto a MLV file and process that file in MLVapp, ie to simulate a LE exposure longer than the ND or base image I can take.

Title: Re: The MLV format
Post by: Danne on October 12, 2019, 07:50:53 PM
Looking forward to this Ilia3101. Mlv app integration maybe?
Title: Re: The MLV format
Post by: Ilia3101 on October 13, 2019, 11:52:17 AM
We could add a feature to MLV App for converting other raw formats to MLV, but that doesn't really fit in to the design seamlessly, it would be like a whole separate app within MLV App, so I would like to create a separate GUI app for that (but command line converter coming first).

And as a reading/writing library, it won't be integrated in to MLV App. Partly because it's difficult, but I think it also adds some competition.

I didn't want to release anything yesterday because bit packing was not working, but I sorted it this morning.

Will release some stuff very soon, github.com/ilia3101/LibMLV soon to be made unprivate.

Also would like advice/help on compiling methods. Want it  to work on Windows too.
Title: Re: The MLV format
Post by: Ilia3101 on October 13, 2019, 11:53:15 AM
At the risk of making a fool of myself I thought I would flag an interest here, ie as a non video guy.

The use case for me is taking a series of .cr2 images to simulate a long exposure, convert them onto a MLV file and process that file in MLVapp, ie to simulate a LE exposure longer than the ND or base image I can take.

That should be possibe.
Title: Re: The MLV format
Post by: Danne on October 13, 2019, 11:56:29 AM
And as a reading/writing library, it won't be integrated in to MLV App. Partly because it's difficult, but I think it also adds some competition.

I didn't want to release anything yesterday because bit packing was not working, but I sorted it this morning.

Will release some stuff very soon, github.com/ilia3101/LibMLV soon to be made unprivate.

Also would like advice/help on compiling methods. Want it  to work on Windows too.
Awesome.
By the way. Is it linux or mac right now?
Title: Re: The MLV format
Post by: Ilia3101 on October 13, 2019, 12:50:11 PM
both working
Title: Re: The MLV format
Post by: masc on October 13, 2019, 03:11:23 PM
Also would like advice/help on compiling methods. Want it  to work on Windows too.
If you like to have it working on all platforms with mostly one code base: Qt is again our best friend ;)
You wrote it again in C/C++?
Title: Re: The MLV format
Post by: Ilia3101 on October 13, 2019, 03:33:47 PM
If you like to have it working on all platforms with mostly one code base: Qt is again our best friend ;)
You wrote it again in C/C++?

Qt for the converter GUI yes, I agree. But I want the library itself to depend on almost nothing. CMAKE seems like a good compiling solution just for the library.

Yes, it's written in C, But it will also have a nice C++ wrapper.

This time I'm not making the mistake of having millions of variables like mlvObject_t in MLV App has. LibMLV reading object contains only MLV blocks themselves and array of where frames are in the file (and which file).
Title: Re: The MLV format
Post by: masc on October 13, 2019, 03:39:47 PM
Qt for the converter GUI yes, I agree.
Qt is not just GUI. You can also build command line tools in Qt.
Title: Re: The MLV format
Post by: Ilia3101 on October 13, 2019, 03:47:41 PM
Qt is not just GUI. You can also build command line tools in Qt.

You mean use qt project (.pro) and let Qt do compiling work?
Title: Re: The MLV format
Post by: masc on October 13, 2019, 04:10:18 PM
Yapp, exactly. I am just not 100% sure if that works without Qt libs - this would be overkill. To be tested...

Edit: if you open QtCreator -> new project -> non-qt project -> Plain C/C++ Application..." This should do the job for Win/Linux/OSX.
(https://i.ibb.co/kxtYCKN/Bildschirmfoto-2019-10-13-um-20-53-37.png)
Title: Re: The MLV format
Post by: Ilia3101 on October 13, 2019, 09:52:46 PM
Ok here's a little bit of libMLV: https://github.com/ilia3101/LibMLV (raw2mlv included, compiles on mac and linux, writing functionality not released yet, generally unfinished)

Quite tired from all this, macOS compiling/linking decided to break today, so I'm also going to have a break.


Yapp, exactly. I am just not 100% sure if that works without Qt libs - this would be overkill. To be tested...

Edit: if you open QtCreator -> new project -> non-qt project -> Plain C/C++ Application..." This should do the job for Win/Linux/OSX.

Thanks. I will try using that for something. I have a command program thing that is simple C, won't use it for that, as it really would be overkill. But there will be more than one program with libMLV so it could come in useful. Definitely going to use Qt for the GUI app.
Title: Re: The MLV format
Post by: DeafEyeJedi on October 13, 2019, 11:24:56 PM
Awesome progress so far fellas!  8)
Title: Re: The MLV format
Post by: masc on October 14, 2019, 08:13:49 PM
Thanks. I will try using that for something. I have a command program thing that is simple C, won't use it for that, as it really would be overkill. But there will be more than one program with libMLV so it could come in useful. Definitely going to use Qt for the GUI app.
If you like to try with Qt, this would be your .pro file:
Code: [Select]
TEMPLATE = app
CONFIG += console
CONFIG -= app_bundle
CONFIG -= qt

SOURCES += read_raw.c \
    camera_matrices.c \
    raw2mlv.c \
    ../LibMLV/MLVFrameUtils.c \
    ../LibMLV/MLVWriter.c

HEADERS += \
    ../LibMLV/LibMLV.h \
    ../LibMLV/mlv_structs.h \
    ../LibMLV/MLVFrameUtils.h \
    ../LibMLV/MLVWriter.h
Shouldn't be much overkill, because it just uses the compiler and nothing else for building. But it is nice to handle, because you can use QtCreator's editor and buttons to compile - so much comfort.
Title: Re: The MLV format
Post by: Ilia3101 on October 14, 2019, 08:28:31 PM
Wow thank you! I will try that out. I'll try and add as many build methods as possible to all aspects of the library later.

But it is nice to handle, because you can use QtCreator's editor and buttons to compile - so much comfort.

Also allows you to use "qmake" command, comfort for me too :D
Title: Re: The MLV format
Post by: Luther on October 15, 2019, 02:09:44 AM
Edit: just realised you suggested ISC. What is the advantage over MIT?

Both say basically the same: use how you want, just mention authors name. Legally speaking it is also the same (AFAIK, I'm not a lawyer). The difference is: ISC is very short and has less ambiguity. It's also easier for bigger projects to use, because it doesn't increase the text size too much.

But I want the library itself to depend on almost nothing. CMAKE seems like a good compiling solution just for the library.

Nice. I think Qt is unnecessary. Even CMAKE is. Why not simply use make (https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html)?
Title: Re: The MLV format
Post by: Ilia3101 on October 16, 2019, 06:17:24 PM
Both say basically the same: use how you want, just mention authors name. Legally speaking it is also the same (AFAIK, I'm not a lawyer). The difference is: ISC is very short and has less ambiguity. It's also easier for bigger projects to use, because it doesn't increase the text size too much.

Ah ok. I may switch to this, still undecided on most aspects of libMLV. I will see what similar libraries are licensed with.

Nice. I think Qt is unnecessary. Even CMAKE is. Why not simply use make (https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html)?

LibRaw has many build files, makefile, CMAKE, Visual Studio project, even Qt project. I think it's worth trying to have many options in libMLV, so it can integrate with as many software projects as possible, as easy as possible. I like makefiles, but I also don't understand them and they are quite hard to make compatible with all systems.