Author Topic: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)  (Read 1069626 times)

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3155
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1875 on: June 07, 2014, 08:38:13 AM »
yet i didnt see what i always ask for in this case.
the output of "mlv_dump -v" for the whole file or the first blocks at least.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Kuky

  • New to the forum
  • *
  • Posts: 36
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1876 on: June 07, 2014, 12:55:19 PM »
yet i didnt see what i always ask for in this case.
the output of "mlv_dump -v" for the whole file or the first blocks at least.

http://pastebin.com/L5UwV9K2

Kuky

  • New to the forum
  • *
  • Posts: 36
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1877 on: June 07, 2014, 01:05:04 PM »
The sticky to end all stickies. Read this first.

Composition FPS?


baldand seems to think the problem is with the header.  IIRC, the correct FPS tag doesn't get written to files when using FPS override.  Until that is fixed, consider the quoted Q&A as correct, even though the wording doesn't exactly describe this problem.



edit:  Added a new Q&A.  Thanks for the heads up.




Yeah, thanks for googling it for me, but as I mentioned before it has nothing to do with the metadata tag. In fact it's just a workaround and the Q&A is relevant if the DOP and the editor are the same person. If not, my experience tells me, being a post supervisor since 1997, that many of the editors will ingest the footage, growling "garbage in, garbage out". Of course there are also editors who will take the initiative and make a phone call or will try to guess the correct fps.

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3605
  • Blunt and to the point
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1878 on: June 07, 2014, 03:30:29 PM »
but as I mentioned before it has nothing to do with the metadata tag.

My bold for emphasis.
If you shoot crop mode (x5 zoom) with a 5D Mark III and use FPS override (for example 24 exact FPS) MlRawViewer will report 29,776 fps. Maybe it's a metadata mistake.

http://www.catb.org/esr/faqs/smart-questions.html#beprecise


In fact it's just a workaround and the Q&A is relevant if the DOP and the editor are the same person. If not, my experience tells me

Hopefully from this day forth, experience will tell you, that if you are going to use experiential software, you will take care to inform the editor of any specific needs your footage has.

dpjpandone

  • Senior
  • ****
  • Posts: 284
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1879 on: June 18, 2014, 02:54:18 PM »
G3ggo,

I would like to help with the issue of hdmi overlay misalignment (issue#1582)

I can try different values and compile and test until they line up, can you give me a hint which lines are relevant?




 

chmee

  • Contributor
  • Hero Member
  • *****
  • Posts: 1031
  • raw2cdng
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1880 on: June 24, 2014, 04:32:02 PM »
small info: coders and all others interested in the mlv-format can analyze the data inside a mlv with this tool.
its far from being perfect, but it helps me to get minified files for debugging.
these files should work in all converters as well.
mlvdiag 1.0.0 - you can find it here

regards chmee

Audionut

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 3605
  • Blunt and to the point
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1881 on: June 30, 2014, 07:24:43 AM »
marekk' timelapse discussion moved here.

gravitatemediagroup

  • Member
  • ***
  • Posts: 241
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1882 on: July 10, 2014, 06:37:26 AM »
mk3 mlvraw images and raw video can be viewed in cam, but PC doesn't recognize any media on the card? sorry if this has been discussed already.

ansius

  • Member
  • ***
  • Posts: 177
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1883 on: July 10, 2014, 12:49:59 PM »
mk3 mlvraw images and raw video can be viewed in cam, but PC doesn't recognize any media on the card? sorry if this has been discussed already.

from card reader or trough USB to camera?
Canon EOS 7D & 40D, EF-S 17-85mm IS USM, EF 28-300mm IS USM, Mir-20, Mir-1, Helios 44-5, Zenitar ME1, Industar 50-2, Industar 61L/Z-MC, Jupiter 37A, TAIR-3
http://www.ansius.lv http://ansius.500px.com

AgentJJ

  • New to the forum
  • *
  • Posts: 38
  • Yup
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1884 on: July 10, 2014, 11:45:45 PM »
Can you wrap ML .RAW files back into an MLV file?

gravitatemediagroup

  • Member
  • ***
  • Posts: 241
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1885 on: July 11, 2014, 03:59:10 AM »
from card reader or trough USB to camera?

card reader, I have a different reader i'm about to try now.

update: different reader still no luck, going to try on a mac and a different PC.  maybe a windows update is causing me a glitch.

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1886 on: July 11, 2014, 05:30:07 PM »
Can you wrap ML .RAW files back into an MLV file?
Theoretically, but I see no point. The raw file is missing the metadata that mlv was designed to capture. Analogy: you can turn a plain text file into a rich text file, but when you do there's not magically going to appear a bunch of formatting. Might as well just leave it a plain text file.

AgentJJ

  • New to the forum
  • *
  • Posts: 38
  • Yup
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1887 on: July 11, 2014, 08:04:55 PM »
It would be purely retroactive.  I have about 3 TB of ML .RAW files that I would love to wrap up to MLV so that I could compress them and save a load of space!  I could write up a tool but I was hoping the protocol might already be implemented somewhere.

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1888 on: July 11, 2014, 11:30:44 PM »
I have about 3 TB of ML .RAW files that I would love to wrap up to MLV so that I could compress them and save a load of space!
Making them MLV would only make them larger. Why would you need them to be MLV to compress them?

The same analogy applies, why would you make a plain text file into a rich text file so that you could compress it? The rich text file is going to be larger. Just compress the plain text file.

AgentJJ

  • New to the forum
  • *
  • Posts: 38
  • Yup
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1889 on: July 14, 2014, 02:03:09 PM »
Only because mlv_dump already compresses the MLV files down via LMZA, while I don't see anything that compresses just the RAW files down.  The amount of compression via mlv_dump far exceeds the overhead in the wrapping.

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1890 on: July 14, 2014, 06:33:26 PM »
while I don't see anything that compresses just the RAW files down
There are thousands of programs that can losslessly compress data out there (for almost any OS you can think of)

how about gzip? (or if you specifically want LZMA, then lzip)

see also

AgentJJ

  • New to the forum
  • *
  • Posts: 38
  • Yup
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1891 on: July 16, 2014, 03:38:31 AM »
Quote
There are thousands of programs that can losslessly compress data out there (for almost any OS you can think of)

 :-[ Of course!  I'm a programmer and I forgot all about gzip ( and I use it pratically everyday with debian packages... ).

Thanks!  It's the simple things I forget...

gregmaustin

  • New to the forum
  • *
  • Posts: 3
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1892 on: July 16, 2014, 05:48:43 PM »
How do I check what the fPS is for a file? I'm on a mac.

./mlv_dump ????

Thank you very much

peoplemerge

  • Freshman
  • **
  • Posts: 50
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1893 on: July 17, 2014, 01:38:45 AM »
Compilation fails on OSX Maverick after Xcode was upgraded to version 5.1 on March 10, 2014.

[Edit:] I should have added that ML autoexec.bin and modules compile OK without any issue even after the Xcode upgrade.
compilation fails only for mlv_dump.


Code: [Select]
[ HOST_CC  ]   mlv_dump.host.o
clang: error: unknown argument: '-mno-ms-bitfields' [-Wunused-command-line-argument-hard-error-in-future]
clang: note: this will be a hard error (cannot be downgraded to a warning) in the future
make: *** [mlv_dump.host.o] Error 1

This did not help either:
export CFLAGS=-Qunused-arguments
export CPPFLAGS=-Qunused-arguments

Removed '-mno-ms-bitfields' from modules/mlv_rec/Makefile, but ld was not happy.

MLV_CFLAGS = -I$(SRC_DIR) -D MLV_USE_LZMA -m32 -Wpadded  -D _7ZIP_ST -D MLV2DNG

Code: [Select]
ld: warning: ld: warning: ignoring file ../../src/chdk-dng.host.o, file was built for i386 which is not the architecture being linked (x86_64): ../../src/chdk-dng.host.oignoring file mlv_dump.host.o, file was built for i386 which is not the architecture being linked (x86_64): mlv_dump.host.o

ld: warning: ignoring file ../lv_rec/raw2dng.host.o, file was built for i386 which is not the architecture being linked (x86_64): ../lv_rec/raw2dng.host.o
ld: warning: ignoring file lzma/lib7z.a, file was built for archive which is not the architecture being linked (x86_64): lzma/lib7z.a
Undefined symbols for architecture x86_64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [mlv_dump] Error 1

Removed -m32 from modules/mlv_rec/Makefile, no luck either.

MLV_CFLAGS = -I$(SRC_DIR) -D MLV_USE_LZMA -Wpadded -D _7ZIP_ST -D MLV2DNG

Code: [Select]
Macintosh:mlv_rec okumar$ make clean && make mlv_dump
[ RM ]  mlv_rec.o mlv.o mlv_rec.mo mlv_rec.sym mlv_rec.dep module_strings.h *.o *.d *.dep *.sym hgstamp
[ RM ]  mlv_dump mlv_dump.exe lzma/7zAlloc.host.o lzma/7zBuf.host.o lzma/7zBuf2.host.o lzma/7zCrc.host.o lzma/7zCrcOpt.host.o lzma/7zDec.host.o lzma/7zFile.host.o lzma/7zIn.host.o lzma/7zStream.host.o lzma/Alloc.host.o lzma/Bcj2.host.o lzma/Bra.host.o lzma/Bra86.host.o lzma/BraIA64.host.o lzma/CpuArch.host.o lzma/Delta.host.o lzma/LzFind.host.o lzma/Lzma2Dec.host.o lzma/Lzma2Enc.host.o lzma/Lzma86Dec.host.o lzma/Lzma86Enc.host.o lzma/LzmaDec.host.o lzma/LzmaEnc.host.o lzma/LzmaLib.host.o lzma/Ppmd7.host.o lzma/Ppmd7Dec.host.o lzma/Ppmd7Enc.host.o lzma/Sha256.host.o lzma/Xz.host.o lzma/XzCrc64.host.o lzma/lib7z.a lzma/Threads.w32.o lzma/LzFindMt.w32.o lzma/MtCoder.w32.o lzma/7zAlloc.w32.o lzma/7zBuf.w32.o lzma/7zBuf2.w32.o lzma/7zCrc.w32.o lzma/7zCrcOpt.w32.o lzma/7zDec.w32.o lzma/7zFile.w32.o lzma/7zIn.w32.o lzma/7zStream.w32.o lzma/Alloc.w32.o lzma/Bcj2.w32.o lzma/Bra.w32.o lzma/Bra86.w32.o lzma/BraIA64.w32.o lzma/CpuArch.w32.o lzma/Delta.w32.o lzma/LzFind.w32.o lzma/Lzma2Dec.w32.o lzma/Lzma2Enc.w32.o lzma/Lzma86Dec.w32.o lzma/Lzma86Enc.w32.o lzma/LzmaDec.w32.o lzma/LzmaEnc.w32.o lzma/LzmaLib.w32.o lzma/Ppmd7.w32.o lzma/Ppmd7Dec.w32.o lzma/Ppmd7Enc.w32.o lzma/Sha256.w32.o lzma/Xz.w32.o lzma/XzCrc64.w32.o lzma/lib7z.w32.a
[ HOST_CC  ]   mlv_dump.host.o
mlv_dump.c:2034:38: error: no member named 'buffer' in 'struct raw_info'
                            raw_info.buffer = frame_buffer;
                            ~~~~~~~~ ^
1 error generated.
make: *** [mlv_dump.host.o] Error 1

I'm having this error too, unable to build mlv_dump on new OSX OS.

Edit: the fix for OSX:

diff -r e12ad3d84e7b modules/mlv_rec/Makefile
--- a/modules/mlv_rec/Makefile   Thu Sep 11 23:50:29 2014 +0200
+++ b/modules/mlv_rec/Makefile   Sun Sep 14 23:59:14 2014 -0700
@@ -6,9 +6,9 @@
 # include modules environment
 include ../Makefile.modules

-MLV_CFLAGS = -I$(SRC_DIR) -D MLV_USE_LZMA -m32 -Wpadded -mno-ms-bitfields -D _7ZIP_ST -D MLV2DNG
+MLV_CFLAGS = -I$(SRC_DIR) -D MLV_USE_LZMA -m32 -Wpadded -D _7ZIP_ST -D MLV2DNG
 MLV_LFLAGS =
-MLV_LIBS = -lm
+MLV_LIBS = -lm -m32
 MLV_LIBS_MINGW = -lm

marekk

  • Senior
  • ****
  • Posts: 331
  • www.oddajmisie.com
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1894 on: July 17, 2014, 11:26:37 AM »
autoexec.bin and modules are compiled using a compiler for arm processor. mlv_dump is compiled using standard gcc compiler for Mac Osx. You should to install gcc using port.

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 562
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1895 on: July 20, 2014, 03:18:16 PM »
3 days ago, ffmpeg released a new major version 2.3. This release supports mlv-demuxing. I´m waiting for this version to find the way to the repos of my OpenSuse (packman).  I don´t know, what is possible with this new feature in ffmpeg, but I will find it out :)

Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

gravitatemediagroup

  • Member
  • ***
  • Posts: 241
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1896 on: July 22, 2014, 02:30:49 AM »
I get tons of dropped frames on MK3 in 5x crop.

jimmyD30

  • Senior
  • ****
  • Posts: 386
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1897 on: July 22, 2014, 03:00:11 AM »
Hmmm... I use 3x crop a bit myself, but haven't noticed any dropped frames. How are you determining this? Is there a log or are you noticing it when viewing the footage?

Also, are you allowing frame skipping when this happens and are you trying to record faster than the camera can write to the card?

gravitatemediagroup

  • Member
  • ***
  • Posts: 241
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1898 on: July 22, 2014, 05:27:05 AM »
In about 10 seconds of recording I have like 20+ dropped frames, more or less depending on the detial probably
.  It is telling me in the top right corner while recording, not sure if it is complately accurate or not.

barepixels

  • Member
  • ***
  • Posts: 134
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1899 on: July 24, 2014, 04:37:31 PM »
Hi g3gg0

few of us think mlv_dump for windows have a bug with large images such as those comes from full frame sensors.  can you investigate

http://www.magiclantern.fm/forum/index.php?topic=12733.msg123171;topicseen#msg123171


EDIT:  THANK YOU FOR THE NEW FIX
5D2 + nightly ML