Author Topic: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)  (Read 1072742 times)

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1975 on: January 12, 2016, 01:25:02 AM »
Sure thing :)

mothaibaphoto

  • Senior
  • ****
  • Posts: 392
  • pesky kid
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1976 on: January 12, 2016, 06:08:05 AM »
@dmilligan:
I can't understand why brilliant Advanced Intervalometer collecting dust buried deep in the forum, while less practice (IMHO) things like lua and compression gets into builds.

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1977 on: January 12, 2016, 02:41:39 PM »
dmilligan can you make a pull request that i can merge?
Sure thing :)
Well... the merge isn't clean since so much has changed, so it will take some more time than I thought. Also might want to wait to get dng.mo merged first since it has changes for mlv_dump too, and currently it's closer to being ready.

The only thing I really changed from your branch was to support both LZMA and LJ92, that way people who may have already used LZMA can still convert their files with the most recent version.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3391
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1978 on: January 12, 2016, 06:20:41 PM »

@dmilligan:
I can't understand why brilliant Advanced Intervalometer collecting dust buried deep in the forum, while less practice (IMHO) things like lua and compression gets into builds.

+1
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3155
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1979 on: January 12, 2016, 11:35:48 PM »
@dmilligan:
I can't understand why brilliant Advanced Intervalometer collecting dust buried deep in the forum, while less practice (IMHO) things like lua and compression gets into builds.

a) OT
b) where is the pull request?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6634
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1980 on: January 21, 2016, 03:09:58 PM »
@g3gg0
I wonder if it,s possible to create a HDR_average mode for mlv_dump? I recently started experimenting with an average filter in ffmpeg which blends every other picture thus delivering HDR processed files without the need for enfuse or other HDR applications.
http://www.magiclantern.fm/forum/index.php?topic=13512.msg160592#msg160592

Now how about a setting to call for in mlv_dump which would do the same? This would be very handy in conjunction with MLVFS or in ACR with slowmo footage(48, 50, 60 fps) creating averaged HDR dng files.
What do you think?

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3391
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1981 on: January 21, 2016, 06:53:54 PM »
I think this is an excellent idea if it's at all possible which I believe it is.

Nice work, Danne! [emoji2]
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3155
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1982 on: January 21, 2016, 09:13:35 PM »
so simply average two consecutive frames?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6634
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1983 on: January 21, 2016, 10:13:25 PM »
Yes, so it seems. Does wonders in higher fps  :)

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1984 on: January 21, 2016, 10:34:06 PM »
While it probably "works", simply averaging two frames seems like a suboptimal way to do HDR merging. The more noisy shadows of the darker exposure will be given more "weight" making the resulting image more noisy than it could be (hence reducing DR) and the highlights will have a strange "elbow" in their rolloff at the point where the brighter exposure starts clipping.

A better way would be to use the method employed by CeroNoise: http://www.magiclantern.fm/forum/index.php?topic=9581.0

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3155
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1985 on: January 21, 2016, 10:51:13 PM »
that would simplify things, wont it?
just export .dng from mlv_dump and process them in ceronoise?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6634
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1986 on: January 22, 2016, 08:32:39 AM »
Thanks for answers.

Quote
While it probably "works", simply averaging two frames seems like a suboptimal way to do HDR merging.
You might be right in theory but my tests shows very clean and professional results when using the tblend average filter with FFmpeg. Enfuse tends to flicker and getting the settings right for blacks and whites is a bit of a problem. Those problems are non existent with the tblend filter.
Question is if mlv_dump averaging does something different from the tblend filter?

Quote
that would simplify things, wont it?
just export .dng from mlv_dump and process them in ceronoise?
The biggest hassle with HDR is the post production marathon. A function in mlv_dump would simplify a lot. Is the averaging on the fly right now in mlv_dump? If so, HDR filming won,t take longer than working with regular files.
Could I test this somehow on a HDR file?


Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6634
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1987 on: January 22, 2016, 06:31:50 PM »
Tried a comparison by running HDR and camera set to 1fps and managed to record a mlv containing only 2 dng files. Seems to be a different way of averaging in mlv_dump and FFmpeg. Maybe overkill to implement HDR in mlv_dump right now. Higher priority would be other things like pixel_fix and dng modules. Anyway, who knows :)

Original file
https://drive.google.com/file/d/0B4tCJMlOYfirbFhoUzZHYnpMd0U/view?usp=sharing

FFmpeg


mlv_dump

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6634
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1988 on: March 23, 2016, 10:14:55 PM »
@g3gg0
Hi. I,m about to implement a nice automated workflow in MLP that will utilize the average darkframe function in mlv_dump but I encounter problems.
Code: [Select]
[ERROR] File '/Users/Daniel/Desktop/Myfiles/hej.MLV' ends in the middle of a block
[ERROR] Failed to load subtract frame (4)
The problem and the answer to why it,s not working seems to be here.
http://www.magiclantern.fm/forum/index.php?topic=7122.msg152952#msg152952

Is it possible to have this fixed also for the mac binary? Would be so nice to get this working.
Original mlv file
https://drive.google.com/file/d/0B4tCJMlOYfirTGR2S21IWFlLd1U/view?usp=sharing
The same file averaged
https://drive.google.com/file/d/0B4tCJMlOYfirTFJTUFNLRTlOblk/view?usp=sharing

*update
Got my hands on a version from 31 aug 2014 which seems to work without giving the error codes. Hopefully this can be fixed in latest version too.
https://drive.google.com/file/d/0B4tCJMlOYfirRXhZY045Z0V3eW8/view?usp=sharing

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6634
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1989 on: March 24, 2016, 11:17:18 AM »
Maybe not the right way to handle the problem but I bypassed the error message by changing the ERR_FILE to ERR_OK  in mlv_dump.c and now it seems to work.       

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3155
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6634
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1991 on: March 24, 2016, 01:43:29 PM »
Great! Working now  :D

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3155
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1992 on: March 24, 2016, 02:16:57 PM »
guess its a FRSP file?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6634
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1993 on: March 24, 2016, 02:37:30 PM »
Actually I was planning on using it also for regular movie files as well as with frsp. Is it even effective to add a darkframe together with let,s say mlv 24fps sequences?

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3155
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1994 on: March 24, 2016, 03:57:45 PM »
sure. just add as many you need/want to get a good enough dark frame.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3391
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1995 on: March 24, 2016, 05:03:46 PM »
Holy Darkness ... This is incredible news @g3gg0 and Thanks for steam rolling on this fix @Danne!

Perhaps should we call you 'Zero Dark MLP'?
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6634
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1996 on: March 24, 2016, 11:51:36 PM »
Hi g3gg0. Here,s my latest tribute to mlv_dump :). Tried a little footage and the averaging does wonders on higher isos. Banding and hot pixels gone.
http://www.magiclantern.fm/forum/index.php?topic=13512.msg164871#msg164871

alsey7

  • Freshman
  • **
  • Posts: 54
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1997 on: May 13, 2016, 04:25:52 AM »
hi guys long time that i used ML but where do i download MLV_snd for my Canon 7D thanks guys

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 6634
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1998 on: May 22, 2016, 09:37:19 AM »
Hi!
Want to report a problem when selecting chroma smooth in mlv_dump for mac os x. When selecting for instance
Code: [Select]
mlv_dump -f 1 --dng --cs5x5 -o hej M04-0025.MLV
It results in segmentation fault 11. Not sure if other platforms works ok?

Code: [Select]
MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: '/Users/Daniel/Desktop/Myfiles/M04-0025.MLV'
   - Convert to DNG frames
   - Output into 'hej'
File /Users/Daniel/Desktop/Myfiles/M04-0025.MLV opened
File /Users/Daniel/Desktop/Myfiles/M04-0025.M00 not existing.
Processing...

Vertical stripes correction:
  1.000  1.000  0.998  1.008  0.996  0.996  0.995  0.995
Segmentation fault: 11

There,s a whole bunch of warnings when compiling mlv_dump. An excerpt.

Code: [Select]
Last login: Sat May 21 13:42:53 on ttys001
Daniels-macbook-pro:mlv_rec Daniel$ make mlv_dump
[ HOST_CC  ]   mlv_dump.host.o
mlv_dump.c: In function 'chroma_smooth':
mlv_dump.c:946:31: warning: passing argument 1 of 'chroma_smooth_2x2' from incompatible pointer type [-Wincompatible-pointer-types]
             chroma_smooth_2x2(aux, aux2, raw2ev, ev2raw);
                               ^
In file included from mlv_dump.c:894:0:
../dual_iso/chroma_smooth.c:2:28: note: expected 'uint32_t * {aka unsigned int *}' but argument is of type 'short unsigned int *'
 #define CHROMA_SMOOTH_FUNC chroma_smooth_2x2
                            ^
../dual_iso/chroma_smooth.c:18:13: note: in expansion of macro 'CHROMA_SMOOTH_FUNC'
 static void CHROMA_SMOOTH_FUNC(uint32_t * inp, uint32_t * out, int* raw2ev, int* ev2raw)
             ^
mlv_dump.c:946:36: warning: passing argument 2 of 'chroma_smooth_2x2' from incompatible pointer type [-Wincompatible-pointer-types]
             chroma_smooth_2x2(aux, aux2, raw2ev, ev2raw);
                                    ^
In file included from mlv_dump.c:894:0:
../dual_iso/chroma_smooth.c:2:28: note: expected 'uint32_t * {aka unsigned int *}' but argument is of type 'short unsigned int *'
 #define CHROMA_SMOOTH_FUNC chroma_smooth_2x2
                            ^
../dual_iso/chroma_smooth.c:18:13: note: in expansion of macro 'CHROMA_SMOOTH_FUNC'
 static void CHROMA_SMOOTH_FUNC(uint32_t * inp, uint32_t * out, int* raw2ev, int* ev2raw)
             ^
mlv_dump.c:949:31: warning: passing argument 1 of 'chroma_smooth_3x3' from incompatible pointer type [-Wincompatible-pointer-types]
             chroma_smooth_3x3(aux, aux2, raw2ev, ev2raw);
                               ^
In file included from mlv_dump.c:898:0:
../dual_iso/chroma_smooth.c:7:28: note: expected 'uint32_t * {aka unsigned int *}' but argument is of type 'short unsigned int *'
 #define CHROMA_SMOOTH_FUNC chroma_smooth_3x3
                            ^
../dual_iso/chroma_smooth.c:18:13: note: in expansion of macro 'CHROMA_SMOOTH_FUNC'
 static void CHROMA_SMOOTH_FUNC(uint32_t * inp, uint32_t * out, int* raw2ev, int* ev2raw)
             ^
mlv_dump.c:949:36: warning: passing argument 2 of 'chroma_smooth_3x3' from incompatible pointer type [-Wincompatible-pointer-types]
             chroma_smooth_3x3(aux, aux2, raw2ev, ev2raw);
                                    ^
In file included from mlv_dump.c:898:0:
../dual_iso/chroma_smooth.c:7:28: note: expected 'uint32_t * {aka unsigned int *}' but argument is of type 'short unsigned int *'
 #define CHROMA_SMOOTH_FUNC chroma_smooth_3x3
                            ^
../dual_iso/chroma_smooth.c:18:13: note: in expansion of macro 'CHROMA_SMOOTH_FUNC'
 static void CHROMA_SMOOTH_FUNC(uint32_t * inp, uint32_t * out, int* raw2ev, int* ev2raw)
             ^
mlv_dump.c:952:31: warning: passing argument 1 of 'chroma_smooth_5x5' from incompatible pointer type [-Wincompatible-pointer-types]
             chroma_smooth_5x5(aux, aux2, raw2ev, ev2raw);
                               ^
In file included from mlv_dump.c:902:0:
../dual_iso/chroma_smooth.c:12:28: note: expected 'uint32_t * {aka unsigned int *}' but argument is of type 'short unsigned int *'
 #define CHROMA_SMOOTH_FUNC chroma_smooth_5x5
                            ^
../dual_iso/chroma_smooth.c:18:13: note: in expansion of macro 'CHROMA_SMOOTH_FUNC'
 static void CHROMA_SMOOTH_FUNC(uint32_t * inp, uint32_t * out, int* raw2ev, int* ev2raw)
             ^
mlv_dump.c:952:36: warning: passing argument 2 of 'chroma_smooth_5x5' from incompatible pointer type [-Wincompatible-pointer-types]
             chroma_smooth_5x5(aux, aux2, raw2ev, ev2raw);
                                    ^
In file included from mlv_dump.c:902:0:
../dual_iso/chroma_smooth.c:12:28: note: expected 'uint32_t * {aka unsigned int *}' but argument is of type 'short unsigned int *'
 #define CHROMA_SMOOTH_FUNC chroma_smooth_5x5
                            ^
../dual_iso/chroma_smooth.c:18:13: note: in expansion of macro 'CHROMA_SMOOTH_FUNC'
 static void CHROMA_SMOOTH_FUNC(uint32_t * inp, uint32_t * out, int* raw2ev, int* ev2raw)
             ^
[ HOST_CC  ]   ../../src/chdk-dng.host.o
../../src/chdk-dng.c:219:9: warning: padding struct to align 'exp_program' [-Wpadded]
     int exp_program;
         ^
../../src/chdk-dng.c: In function 'write_dng':
../../src/chdk-dng.c:727:60: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         if (write(fd, dng_header_buf, dng_header_buf_size) != dng_header_buf_size) return 0;
                                                            ^
../../src/chdk-dng.c:728:68: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         if (write(fd, thumbnail_buf, dng_th_width*dng_th_height*3) != dng_th_width*dng_th_height*3) return 0;
                                                                    ^
../../src/chdk-dng.c:731:68: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         if (write(fd, UNCACHEABLE(rawadr), camera_sensor.raw

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3155
Re: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)
« Reply #1999 on: May 22, 2016, 08:18:35 PM »
what kind of file is it?
can you dump it using -v ?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!