Author Topic: Magic Lantern (RAW) Video format v2.0 (mlv_rec.mo)  (Read 1420905 times)

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Magic Lantern (RAW) Video format v2.0
« Reply #625 on: October 06, 2013, 05:51:21 PM »
to raw.c or to chdk-dng.c ?

Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Magic Lantern (RAW) Video format v2.0
« Reply #626 on: October 06, 2013, 06:07:51 PM »
to raw.c or to chdk-dng.c ?

Edgar

OK, to raw.h  :)

I compiled the whole stuff new with this changes and now the sigsegfault has gone. mlv_dump runs and I can decode the file with raw2dng

ThankYou


Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3190
Re: Magic Lantern (RAW) Video format v2.0
« Reply #627 on: October 06, 2013, 06:14:26 PM »
ok thats the reason why i always use stdint types "uint32_t" etc instead of the random width types "int, long, short".
this saves you from architecture trouble.

(ok i rarely care for endianess conversion in my code and always assume we work on little endian, but thats an other issue)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Magic Lantern (RAW) Video format v2.0
« Reply #628 on: October 06, 2013, 06:22:23 PM »
i guess it is not your fault, but mine :)
you can keep the -m32 removed maybe.

can you check if compression still works?

Compression and decompression seems to work

Code: [Select]
edgar@linux-lyl7:~/test> ./mlv_dump -c -l 9 -o compress.mlv 1.MLV

 MLV Dumper v1.0
-----------------

[i] Mode of operation:
   - Input MLV file: '1.MLV'
   - Rewrite MLV
   - Compress frame data
   - Output into 'compress.mlv'
[i] File 1.MLV opened
[i] Processing...
[i] Unknown Block: NULL, skipping
  saving: 790272 -> 740650  (93.72%)
[i] Unknown Block: NULL, skipping
  saving: 790272 -> 748475  (94.71%)
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Magic Lantern (RAW) Video format v2.0
« Reply #629 on: October 06, 2013, 06:39:00 PM »
The camera-name looks a bit strange  :)

edgar@linux-lyl7:~/test> ./mlv_dump -v 1.MLV >test
Code: [Select]
...
Block: IDNT
  Offset: 0x000001ac
    Size: 84
    Time: 28.726000 ms
     Camera Name:   'ERR:1 md:0x       0 ml:0'
     Camera Serial: ''
     Camera Model:  0x00000000
...

Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3190
Re: Magic Lantern (RAW) Video format v2.0
« Reply #630 on: October 06, 2013, 06:57:34 PM »
thats because of:
"/* 00:00:36.154540  */ WARNING: This model doesn't have 'PROPAD_GetPropertyData' defined. Reading properties not possible."

they usually can be found by searching for the string and checking who is being called.
PROP_GetMulticastProperty is usually named prop_get_value, thats the reason why it has the same address.

e.g. 7D:
/** corectly named canon property reading functions **/
NSTUB(0xFF0687C8, PROP_GetMulticastProperty)
NSTUB(0xFF06934C, PROPAD_GetPropertyData)

5D3:
/** corectly named canon property reading functions **/
NSTUB(0xFF123DE4, PROP_GetMulticastProperty)
NSTUB(0xFF12523C, PROPAD_GetPropertyData)
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3190
Re: Magic Lantern (RAW) Video format v2.0
« Reply #631 on: October 06, 2013, 07:18:08 PM »
can you hg pull and recompile magic lantern again?
no need to recompile the modules.
it may be necessary to undo the stdint.h and raw.h changes you made

you then should see your camera information in the recorded movie.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

oddname

  • New to the forum
  • *
  • Posts: 45
Re: Magic Lantern (RAW) Video format v2.0
« Reply #632 on: October 06, 2013, 07:43:19 PM »
you wanted to try it again, you said a few weeks ago ;)

the module is not camera-dependent, you can use it for any model.

it should work with any camera on which the original raw_rec works too.
but it may require some symbols in the ML/MODULES/<model>.SYM, so you may get an error.

which error do you get? which model?

Which I did, but i noticed ppl with 5d2 posted logs and videos showing the same results.
Tried again  with the same results, lock-up.

Thought there was another recent build to try :)

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Magic Lantern (RAW) Video format v2.0
« Reply #633 on: October 06, 2013, 07:57:25 PM »
can you hg pull and recompile magic lantern again?
...
you then should see your camera information in the recorded movie.

I added the prop... per hand to stubs.S, because your repos are a little bit amazed at the moment  ;) . Now the IDNT-Block with camera-name is ok.

ThankYou

Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Magic Lantern (RAW) Video format v2.0
« Reply #634 on: October 06, 2013, 08:38:11 PM »
I'm still getting hsuite error on 7D, second record

Replaced the raw struct too.

Quote
ML ASSERT:
(int)size == hSuite->size
at ../../src/exmem.c:122 (shoot_malloc_suite), task raw_rec_task
lv:1 mode:3

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3190
Re: Magic Lantern (RAW) Video format v2.0
« Reply #635 on: October 06, 2013, 08:48:32 PM »
hmm so on 600D and 7D there are such errors?
i tried a bunch of videos on 7D with some older version and had no such errors. damn.

which settings?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Magic Lantern (RAW) Video format v2.0
« Reply #636 on: October 06, 2013, 09:15:30 PM »
Just normal stuff/ gd off + hacks.

I'm trying the binary and it says no info structure found...

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: Magic Lantern (RAW) Video format v2.0
« Reply #637 on: October 06, 2013, 09:17:05 PM »
Cool, I see MLV works on 550D, just made my first MLV clip.
500D/T1i  550D/T2i  600D/T3i  700D/T5i

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Magic Lantern (RAW) Video format v2.0
« Reply #638 on: October 06, 2013, 09:37:00 PM »
Code: [Select]
6D freezes when stopping 2nd record

ML ASSERT:
(int)size == hSuite->size
at ../../src/exmem.c:122 (shoot_malloc_suite), task raw_rec_task
lv:1 mode:3

Have i f'ed something up?
https://bitbucket.org/OtherOnePercent/tragic-lantern-6d/src/0ae870f7dbd161440955320583c5eff381fb8b60/modules/mlv_rec/mlv_rec.c

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Magic Lantern (RAW) Video format v2.0
« Reply #639 on: October 06, 2013, 09:44:09 PM »
Some other hints for this camera-crash on my 600d, maybe

I can record as many videos I want, if I switch off the camera after every video.

I record a video and stop it and than switch away from videomode. Now, if i switch back to videomode, the camera tells me: busy, please wait (beschäftigt, bitte warten)

Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

mk11174

  • Contributor
  • Hero Member
  • *****
  • Posts: 809
Re: Magic Lantern (RAW) Video format v2.0
« Reply #640 on: October 06, 2013, 09:54:07 PM »
6D freezes when stopping 2nd record


Same freeze on 550D
500D/T1i  550D/T2i  600D/T3i  700D/T5i

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3190
Re: Magic Lantern (RAW) Video format v2.0
« Reply #641 on: October 06, 2013, 10:23:37 PM »
ok there seems to be some weird shoot mem issue... hmm
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: Magic Lantern (RAW) Video format v2.0
« Reply #642 on: October 12, 2013, 05:19:56 PM »
Tested todays build on 5D3.

At the moment the spanning function records both files on CF card and SD is not being used.
Recorded files are ok.
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: Magic Lantern (RAW) Video format v2.0
« Reply #643 on: October 12, 2013, 06:58:24 PM »
Ok, I found the commits, thank you for fixing it.. I see 83 -85MB/s instant writes but seems a bit inconsistent.

When I do the buffer test:

7D FULL 1x

M 0 B 3 79.1 5GB
M 0 B 3 80.1 6GB
M 0 B 3 81.9 14GB
M 0 B 3 82.1 14GB

As card becomes more full...

M1 81.2 8GB
M1  80.1 7GB
M1  72.1 1.2GB
M1  77.4 3.2GB

M2 78.1 2.5GB
M2 79.2 4.3GB
M2 70.4 1.5GB

M3 60.5 800MB
M3 73 1.5GB
M3 74.4 1.7GB

M4 71.1 1.3GB
M4 74.5 1.9GB
M4 72.2 1.4GB
M4 75.2 2GB

Raw_rec writes  a full card 2nd time recording.

Maybe this test needs to be done on 2.5K in zoom mode?




escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Magic Lantern (RAW) Video format v2.0
« Reply #644 on: October 12, 2013, 08:16:17 PM »
raw_rec doesn´t compile with latest sources (magic-lantern-mlv) on my 600d. I have to change raw_rec.c like this:

Code: [Select]
edgar@linux-lyl7:~/MagicLantern/magic-lantern-mlv/modules/raw_rec> diff raw_rec_original.c raw_rec.c
68,69c68,69
< #include <math.h
< #include <cropmarks.h>>
---
> #include <math.h>
> #include <cropmarks.h>
342c342
< static unsigned int raw_rec_should_preview(unsigned int ctx);
---
> static uint32_t raw_rec_should_preview(uint32_t ctx);

Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed


escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Magic Lantern (RAW) Video format v2.0
« Reply #646 on: October 12, 2013, 09:06:20 PM »
Didn´t see the pull request.  Great

Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3190
Re: Magic Lantern (RAW) Video format v2.0
« Reply #647 on: October 12, 2013, 09:57:13 PM »
thanks for the support!
merged in the necessary stuff for 5D2.

i still have the write performance drops every few seconds and no idea where they come from.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

escho

  • Contributor
  • Hero Member
  • *****
  • Posts: 563
Re: Magic Lantern (RAW) Video format v2.0
« Reply #648 on: October 12, 2013, 10:12:34 PM »
600D:

Any reasons not to append these changes to the mlv-sources?  mlv_dump doesnt run wifhout for me

can you open raw.h in src/ directory and replace the raw_info struct with this one?

Code: [Select]
/* raw image info (geometry, calibration levels, color, DR etc); parts of this were copied from CHDK */
struct raw_info {
    int32_t api_version;            // increase this when changing the structure
    uint32_t buffer;               // points to image data
   
    int32_t height, width, pitch;
    int32_t frame_size;
    int32_t bits_per_pixel;         // 14

    int32_t black_level;            // autodetected
    int32_t white_level;            // somewhere around 13000 - 16000, varies with camera, settings etc
                                // would be best to autodetect it, but we can't do this reliably yet
    union                       // DNG JPEG info
    {
        struct
        {
            int32_t x, y;           // DNG JPEG top left corner
            int32_t width, height;  // DNG JPEG size
        } jpeg;
        struct
        {
            int32_t origin[2];
            int32_t size[2];
        } crop;
    };
    union                       // DNG active sensor area (Y1, X1, Y2, X2)
    {
        struct
        {
            int32_t y1, x1, y2, x2;
        } active_area;
        int32_t dng_active_area[4];
    };
    int32_t exposure_bias[2];       // DNG Exposure Bias (idk what's that)
    int32_t cfa_pattern;            // stick to 0x02010100 (RGBG) if you can
    int32_t calibration_illuminant1;
    int32_t color_matrix1[18];      // DNG Color Matrix
   
    int32_t dynamic_range;          // EV x100, from analyzing black level and noise (very close to DxO)
};



you can keep the -m32 removed maybe.


add a
#include <stdint.h>
to the c file as first line

Edgar
https://sternenkarten.com/
600D, 6D, openSUSE Tumbleweed

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3190
Re: Magic Lantern (RAW) Video format v2.0
« Reply #649 on: October 12, 2013, 10:26:46 PM »
can you enable the buffer graphs and check if you also get some write rate drop every now and then?

green line = buffer fill level
red line = write rate
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!