Messages

Merged cleanly and tested out fine so I went ahead and committed it. The lua_fix branch has been merged into crop_rec_4k a few time and then crop_rec_4k merged into crop_rec_4k_mlv_snd but it looks like these branches are a bit behind the latest lua_fix changes. Since there is some work in progress on the crop_rec_4k branch that affects the EOSM and 700D I didn't want to mess around with merging lua_fix into it.

Thank you dfort, It's looking much better and now i have only to figure out a conflict with raw.c (It touching 70D) apart the ones on the modules (easy to solve). Will look into it later or tomorrow.

Are these the commits name to follow?

Code Select Expand


"advanced lens information in ML core"
"added LUA interface and functions to set manual lens data"
"mlv_rec: add ELNS support"
"mlv_lite: add ELNS support"
"silent: handle advanced lens info"


I'm trying to merge manual_lens_info to crop_rec_4k_mlv_snd but I'm a bit confused on which changes use to solve conflicts from commits made in lua_fix and the experimental stuffs from the other branch.

Is it possible to merge lua_fix into crop_rec_4k_mlv_snd? This should make managing conflicts from manual_lens_info much easier, and should also make changeset much lighter... Also should leaves diffs from manual_lens_info code when pushing the PR, making room for commits names suggested by g3gg0

Here are logs from 50D: Cr2 capture and Cr2 Review

They were taken without entering LiveView.
Hope they are good, let me know if you need others.

Found a bug after taking a silent picture in a commit from lua_fix.

Doens't happens with mlv_rec, mlv_lite and if I comment "force_liveview()" line.
Maybe it get queued?

Do you want to do the pull request or do you want me to do it?

If everything is working fine I would like to commit the "dynamic" version first on manual_lens_info (to have the original branch up to date) and then make a new PR to crop_rec_4k_mlv_snd (is this the currently bleeding edge branch? I'll need your help to review this PR). What do you think about?

Note that we'll need to break it down into smaller commits.

Regarding breaking down into smaller commits, I can do this in manual_lens_inf PR; do it needs to be done also when merging to crop_rec?

our repositories seem to be out of sync because your patch didn't apply cleanly.

My repo was based on top of crop_rec_4k but then we moved to the one with sound support.
I had to solve more conflicts when merged mlv_snd but due to some problems determining which diff were the most up to data, I screw it up and moved working on top of manual_lens_info adapting changes to works with you repo.

https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/22/elns-for-crop_rec_4k/diff

Having your repo as a reference could be useful to avoid missing something when merging to crop_rec_4k*, because differences from crop_rec_4k and crop_rec_4k_mlv_snd are not exactly clear to me.

LENS block processing is still untouched from some times (both mlv.c and mlv_dump.c); the only changes was about serial number fix made by g3gg0.

I think it was unoticed  until now or maybe it showed up after increasing size of lens_info.name to 64 byte.

In mlv 2.0 specifications, lens name is a optional null terminated string, so maybe option number 2 is to be preferred to avoid interference with other processing tools?

I think because it's printing serial number at the end of lens name, due to a missing termination character of the string... (present after first 32 char)

Two option here:
1) hardening string with a termination character in mlv.c:

Code Select Expand


diff --git a/modules/mlv_rec/mlv.c b/modules/mlv_rec/mlv.c
--- a/modules/mlv_rec/mlv.c
+++ b/modules/mlv_rec/mlv.c
@@ -58,7 +58,7 @@
     char buf[33];
     snprintf(buf, sizeof(buf), "%X%08X", (uint32_t) (lens_info.lens_serial >> 32), (uint32_t)(lens_info.lens_serial & 0xFFFFFFFF));
     
-    strncpy((char *)hdr->lensName, lens_info.name, 32);
+    snprintf((char *)hdr->lensName, 32, "%s", lens_info.name);
     strncpy((char *)hdr->lensSerial, buf, 32);
}

As a drawback we will loose a char of the string, making space for a 31 char string without using ELNS block

2) modify mlv_dump to copy first 32 character of the string in a "buffer" and then print it instead of lens_info.lensName

Code Select Expand


diff --git a/modules/mlv_rec/mlv_dump.c b/modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c
+++ b/modules/mlv_rec/mlv_dump.c
@@ -3670,18 +3670,22 @@

                 if(verbose)
                 {
+                    /* Ensure lens name is null trminated to avoid printing serial number */
+                    char name[33];
+                    snprintf(name, sizeof(name), "%s", lens_info.lensName);
+
                     uint64_t serial = 0;
                     char serial_str[64];
                     char *end;
-                   
+
                     strcpy(serial_str, "no valid S/N");
                     serial = strtoull((char *)lens_info.lensSerial, &end, 16);
                     if (serial && !*end)
                     {
                         sprintf(serial_str, "%"PRIu64, serial);
                     }
-                   
-                    print_msg(MSG_INFO, "     Name:        '%s'\n", lens_info.lensName);
+
+                    print_msg(MSG_INFO, "     Name:        '%s'\n", name);
                     print_msg(MSG_INFO, "     Serial:      '%s' (%s)\n", lens_info.lensSerial, serial_str);
                     print_msg(MSG_INFO, "     Focal Len:   %d mm\n", lens_info.focalLength);
                     print_msg(MSG_INFO, "     Focus Dist:  %d mm\n", lens_info.focalDist);


Can cause similar issue when processing with other tools if they don't take care of it

I post the patch also here just in case someone want to try this on top of dfort's repository:

Code Select Expand


diff --git a/modules/mlv_lite/mlv_lite.c b/modules/mlv_lite/mlv_lite.c
--- a/modules/mlv_lite/mlv_lite.c
+++ b/modules/mlv_lite/mlv_lite.c
@@ -359,10 +359,10 @@
static GUARDED_BY(RawRecTask)   mlv_idnt_hdr_t idnt_hdr;
static GUARDED_BY(RawRecTask)   mlv_expo_hdr_t expo_hdr;
static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
-static GUARDED_BY(RawRecTask)   mlv_elns_hdr_t elns_hdr;
static GUARDED_BY(RawRecTask)   mlv_rtci_hdr_t rtci_hdr;
static GUARDED_BY(RawRecTask)   mlv_wbal_hdr_t wbal_hdr;
-static GUARDED_BY(LiveViewTask) mlv_vidf_hdr_t vidf_hdr;
+static GUARDED_BY(LiveViewTask) mlv_vidf_hdr_t vidf_hdr;
+static GUARDED_BY(RawRecTask)   mlv_elns_hdr_t *elns_hdr;
static GUARDED_BY(RawRecTask)   uint64_t mlv_start_timestamp = 0;
        GUARDED_BY(RawRecTask)   uint32_t raw_rec_trace_ctx = TRACE_ERROR;

@@ -3048,7 +3048,7 @@
         fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&idnt_hdr);
         fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&expo_hdr);
         fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&lens_hdr);
-        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&elns_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)elns_hdr);
         fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&rtci_hdr);
         fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&wbal_hdr);
         fail |= mlv_write_vers_blocks(f, mlv_start_timestamp);
diff --git a/modules/mlv_rec/dng/dng.c b/modules/mlv_rec/dng/dng.c
--- a/modules/mlv_rec/dng/dng.c
+++ b/modules/mlv_rec/dng/dng.c
@@ -669,7 +669,12 @@
             {tcBaselineExposureOffset,      ttSRational,RATIONAL_ENTRY2(0, 1, header, &data_offset)},
             {tcImageDescription,            ttAscii,    STRING_ENTRY(frame_info->info_str, header, &data_offset)},
         };
-       
+
+        /* Use Lens Model from ELNS Block if present */
+        char* lens = (frame_info->elns_str != NULL)
+              && (strlen(frame_info->elns_str) > strlen(frame_info->lens_hdr.lensName))
+              ? frame_info->elns_str : frame_info->lens_hdr.lensName;
+
         struct directory_entry EXIF_IFD[EXIF_IFD_COUNT] =
         {
             {tcExposureTime,                ttRational, RATIONAL_ENTRY2((int32_t)frame_info->expo_hdr.shutterValue/1000, 1000, header, &data_offset)},
@@ -682,7 +687,7 @@
             {tcFocalPlaneXResolutionExif,   ttRational, RATIONAL_ENTRY(focal_resolution_x, header, &data_offset, 2)},
             {tcFocalPlaneYResolutionExif,   ttRational, RATIONAL_ENTRY(focal_resolution_y, header, &data_offset, 2)},
             {tcFocalPlaneResolutionUnitExif,ttShort,    1,      camera_id[current_cam].focal_unit}, //inches
-            {tcLensModelExif,               ttAscii,    STRING_ENTRY((char*)frame_info->lens_hdr.lensName, header, &data_offset)},
+            {tcLensModelExif,               ttAscii,    STRING_ENTRY(lens, header, &data_offset)},
         };
         
         /* do not put image description, if the length is zero */
diff --git a/modules/mlv_rec/dng/dng.h b/modules/mlv_rec/dng/dng.h
--- a/modules/mlv_rec/dng/dng.h
+++ b/modules/mlv_rec/dng/dng.h
@@ -76,8 +76,8 @@
     mlv_rawc_hdr_t rawc_hdr;
     mlv_expo_hdr_t expo_hdr;
     mlv_lens_hdr_t lens_hdr;
-    mlv_elns_hdr_t elns_hdr;
     mlv_wbal_hdr_t wbal_hdr;
+    char *elns_str;
     char *info_str;
};

diff --git a/modules/mlv_rec/mlv.c b/modules/mlv_rec/mlv.c
--- a/modules/mlv_rec/mlv.c
+++ b/modules/mlv_rec/mlv.c
@@ -62,7 +62,7 @@
     strncpy((char *)hdr->lensSerial, buf, 32);
}

-void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp)
+void mlv_fill_elns(mlv_elns_hdr_t **hdr, uint64_t start_timestamp)
{
     /* Calculate total block length: header + fixed data + variable lensName string */
     int string_length = strlen(lens_info.name);
@@ -73,6 +73,7 @@
     mlv_set_type((mlv_hdr_t *)header, "ELNS");
     mlv_set_timestamp((mlv_hdr_t *)header, start_timestamp);
     header->blockSize = block_length;
+    /* Fill ELNS data */
     header->focalLengthMin = lens_info.lens_focal_min;
     header->focalLengthMax = lens_info.lens_focal_max;
     header->apertureMin = RAW2VALUE(aperture, lens_info.raw_aperture_min) / 10.0;
@@ -84,10 +85,10 @@

     /* Store lensName string at the end of mlv_elns_hdr_t */
     char *lens_hdr_payload = (char *)&header[1];
-    strcpy(lens_hdr_payload, lens_info.name);
+    snprintf(lens_hdr_payload, string_length + 1, "%s", lens_info.name);

     /* update block with new values */
-    hdr = header;
+    *hdr = header;
}

void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp)
diff --git a/modules/mlv_rec/mlv.h b/modules/mlv_rec/mlv.h
--- a/modules/mlv_rec/mlv.h
+++ b/modules/mlv_rec/mlv.h
@@ -176,9 +176,7 @@
     uint8_t     extenderInfo;       /* extender information, if provided by camera       */
     uint8_t     capabilities;       /* capability information, if provided by camera     */
     uint8_t     chipped;            /* when not zero, lens is communicating with camera  */
-    uint8_t     length;             /* to allow lens with name longer than 32byte        */
  /* uint8_t     lensName[variable];    full lens string, null terminated                 */
-    // TODO: Review Specs
}  mlv_elns_hdr_t;

typedef struct {
@@ -304,7 +302,7 @@
void mlv_fill_rtci(mlv_rtci_hdr_t *hdr, uint64_t start_timestamp);
void mlv_fill_expo(mlv_expo_hdr_t *hdr, uint64_t start_timestamp);
void mlv_fill_lens(mlv_lens_hdr_t *hdr, uint64_t start_timestamp);
-void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp);
+void mlv_fill_elns(mlv_elns_hdr_t **hdr, uint64_t start_timestamp);
void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
diff --git a/modules/mlv_rec/mlv_dump.c b/modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c
+++ b/modules/mlv_rec/mlv_dump.c
@@ -1941,7 +1941,8 @@
     memset(&rtci_info, 0x00, sizeof(mlv_rtci_hdr_t));
     memset(&rawc_info, 0x00, sizeof(mlv_rawc_hdr_t));
     memset(&main_header, 0x00, sizeof(mlv_file_hdr_t));
-
+
+    char elns_string[1024] = "";
     char info_string[1024] = "";

     /* this table contains the XREF chunk read from idx file, if existing */
@@ -3468,7 +3469,8 @@
                             frame_info.expo_hdr             = expo_info;
                             frame_info.lens_hdr             = lens_info;
                             frame_info.wbal_hdr             = wbal_info;
-                            frame_info.rawc_hdr             = rawc_info;
+                            frame_info.rawc_hdr             = rawc_info;
+                            frame_info.elns_str             = elns_string;
                             frame_info.info_str             = info_string;
                             frame_info.rawi_hdr.xRes        = lv_rec_footer.xRes;
                             frame_info.rawi_hdr.yRes        = lv_rec_footer.yRes;
@@ -3780,30 +3782,24 @@
             {
                 mlv_elns_hdr_t block_hdr = *(mlv_elns_hdr_t *)mlv_block;

-                /* get the string length and malloc a buffer for that string */
-                int str_length = block_hdr.length;
-
-                if(str_length)
+                void *payload = BYTE_OFFSET(mlv_block, sizeof(mlv_elns_hdr_t));
+                int str_length = MIN(block_hdr.blockSize - sizeof(block_hdr), sizeof(elns_string) - 1);
+
+                /* Fill lens model data for DNG processing */
+                strncpy(elns_string, payload, str_length);
+                elns_string[str_length] = '\000';
+
+                if(verbose)
                 {
-                    void *payload = BYTE_OFFSET(mlv_block, sizeof(mlv_elns_hdr_t) - 1);
-                    char *buf = malloc(str_length + 1);
-
-                    strncpy(buf, payload, str_length);
-                    buf[str_length] = '\000';
-
-                    if(verbose)
-                    {
-                        print_msg(MSG_INFO, "     Name:                '%s'\n", buf);
-                        print_msg(MSG_INFO, "     Focal Length Min:    %d mm\n", elns_info.focalLengthMin);
-                        print_msg(MSG_INFO, "     Focal Length Max:    %d mm\n", elns_info.focalLengthMax);
-                        print_msg(MSG_INFO, "     Aperture Min:        f/%.2f\n", (double)elns_info.apertureMin);
-                        print_msg(MSG_INFO, "     Aperture Max:        f/%.2f\n", (double)elns_info.apertureMax);
-                        print_msg(MSG_INFO, "     Version:             %d\n", elns_info.version);
-                        print_msg(MSG_INFO, "     Extender Info:       0x%02X\n", elns_info.extenderInfo);
-                        print_msg(MSG_INFO, "     Capabilities:        0x%02X\n", elns_info.capabilities);
-                        print_msg(MSG_INFO, "     Chipped:             0x%02X\n", elns_info.chipped);
-                    }
-                    free(buf);
+                    print_msg(MSG_INFO, "     Name:                '%s'\n", payload);
+                    print_msg(MSG_INFO, "     Focal Length Min:    %d mm\n", block_hdr.focalLengthMin);
+                    print_msg(MSG_INFO, "     Focal Length Max:    %d mm\n", block_hdr.focalLengthMax);
+                    print_msg(MSG_INFO, "     Aperture Min:        f/%.2f\n", (double)block_hdr.apertureMin);
+                    print_msg(MSG_INFO, "     Aperture Max:        f/%.2f\n", (double)block_hdr.apertureMax);
+                    print_msg(MSG_INFO, "     Version:             %d\n", block_hdr.version);
+                    print_msg(MSG_INFO, "     Extender Info:       0x%02X\n", block_hdr.extenderInfo);
+                    print_msg(MSG_INFO, "     Capabilities:        0x%02X\n", block_hdr.capabilities);
+                    print_msg(MSG_INFO, "     Chipped:             0x%02X\n", block_hdr.chipped);
                 }
             }
             else if(!memcmp(mlv_block->blockType, "ELVL", 4))
diff --git a/modules/mlv_rec/mlv_rec.c b/modules/mlv_rec/mlv_rec.c
--- a/modules/mlv_rec/mlv_rec.c
+++ b/modules/mlv_rec/mlv_rec.c
@@ -200,9 +200,9 @@

static mlv_expo_hdr_t last_expo_hdr;
static mlv_lens_hdr_t last_lens_hdr;
-static mlv_elns_hdr_t last_elns_hdr;
static mlv_wbal_hdr_t last_wbal_hdr;
-static mlv_styl_hdr_t last_styl_hdr;
+static mlv_styl_hdr_t last_styl_hdr;
+static mlv_elns_hdr_t *last_elns_hdr;


/* for debugging */
@@ -1535,13 +1535,13 @@
             trace_write(raw_rec_trace_ctx, "[polling_cbr] queueing INFO blocks");
             mlv_expo_hdr_t *expo_hdr = malloc(sizeof(mlv_expo_hdr_t));
             mlv_lens_hdr_t *lens_hdr = malloc(sizeof(mlv_lens_hdr_t));
+            mlv_wbal_hdr_t *wbal_hdr = malloc(sizeof(mlv_wbal_hdr_t));
             mlv_elns_hdr_t *elns_hdr = malloc(sizeof(mlv_elns_hdr_t));
-            mlv_wbal_hdr_t *wbal_hdr = malloc(sizeof(mlv_wbal_hdr_t));

             mlv_fill_expo(expo_hdr, mlv_start_timestamp);
             mlv_fill_lens(lens_hdr, mlv_start_timestamp);
-            mlv_fill_elns(elns_hdr, mlv_start_timestamp);
-            mlv_fill_wbal(wbal_hdr, mlv_start_timestamp);
+            mlv_fill_wbal(wbal_hdr, mlv_start_timestamp);
+            mlv_fill_elns(&elns_hdr, mlv_start_timestamp);

             msg_queue_post(mlv_block_queue, (uint32_t) expo_hdr);
             msg_queue_post(mlv_block_queue, (uint32_t) lens_hdr);
@@ -2674,10 +2674,10 @@
         mlv_rtci_hdr_t rtci_hdr;
         mlv_expo_hdr_t expo_hdr;
         mlv_lens_hdr_t lens_hdr;
-        mlv_elns_hdr_t elns_hdr;
         mlv_idnt_hdr_t idnt_hdr;
         mlv_wbal_hdr_t wbal_hdr;
         mlv_styl_hdr_t styl_hdr;
+        mlv_elns_hdr_t *elns_hdr;

         mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
         mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
@@ -2694,12 +2694,12 @@
         idnt_hdr.timestamp = 4;
         wbal_hdr.timestamp = 5;
         styl_hdr.timestamp = 6;
-        elns_hdr.timestamp = 7;
+        elns_hdr->timestamp = 7;

         mlv_write_hdr(f, (mlv_hdr_t *)&rtci_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&expo_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&lens_hdr);
-        mlv_write_hdr(f, (mlv_hdr_t *)&elns_hdr);
+        mlv_write_hdr(f, (mlv_hdr_t *)elns_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&idnt_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&wbal_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&styl_hdr);
@@ -3211,7 +3211,7 @@

         mlv_expo_hdr_t old_expo = last_expo_hdr;
         mlv_lens_hdr_t old_lens = last_lens_hdr;
-        mlv_elns_hdr_t old_elns = last_elns_hdr;
+        mlv_elns_hdr_t *old_elns = last_elns_hdr;

         mlv_fill_expo(&last_expo_hdr, mlv_start_timestamp);
         mlv_fill_lens(&last_lens_hdr, mlv_start_timestamp);
@@ -3220,7 +3220,7 @@
         /* update timestamp for comparing content changes */
         old_expo.timestamp = last_expo_hdr.timestamp;
         old_lens.timestamp = last_lens_hdr.timestamp;
-        old_elns.timestamp = last_elns_hdr.timestamp;
+        old_elns->timestamp = last_elns_hdr->timestamp;

         /* write new state if something changed */
         if(memcmp(&last_expo_hdr, &old_expo, sizeof(mlv_expo_hdr_t)))
@@ -3240,8 +3240,8 @@
         /* write new state if something changed */
         if(memcmp(&last_elns_hdr, &old_elns, sizeof(mlv_elns_hdr_t)))
         {
-            mlv_hdr_t *hdr = malloc(sizeof(mlv_elns_hdr_t));
-            memcpy(hdr, &last_elns_hdr, sizeof(mlv_elns_hdr_t));
+            mlv_hdr_t *hdr = malloc(last_elns_hdr->blockSize);
+            memcpy(hdr, last_elns_hdr, last_elns_hdr->blockSize);
             msg_queue_post(mlv_block_queue, (uint32_t) hdr);
         }
     }
diff --git a/modules/silent/silent.c b/modules/silent/silent.c
--- a/modules/silent/silent.c
+++ b/modules/silent/silent.c
@@ -28,10 +28,10 @@
extern WEAK_FUNC(ret_0) void mlv_fill_rtci(mlv_rtci_hdr_t *hdr, uint64_t start_timestamp);
extern WEAK_FUNC(ret_0) void mlv_fill_expo(mlv_expo_hdr_t *hdr, uint64_t start_timestamp);
extern WEAK_FUNC(ret_0) void mlv_fill_lens(mlv_lens_hdr_t *hdr, uint64_t start_timestamp);
-extern WEAK_FUNC(ret_0) void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp);
extern WEAK_FUNC(ret_0) void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
extern WEAK_FUNC(ret_0) void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
extern WEAK_FUNC(ret_0) void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
+extern WEAK_FUNC(ret_0) void mlv_fill_elns(mlv_elns_hdr_t **hdr, uint64_t start_timestamp);
extern WEAK_FUNC(ret_0_long) uint64_t mlv_generate_guid();
extern WEAK_FUNC(ret_0) void mlv_init_fileheader(mlv_file_hdr_t *hdr);
extern WEAK_FUNC(ret_0) void mlv_set_type(mlv_hdr_t *hdr, char *type);
@@ -366,11 +366,11 @@
     mlv_rtci_hdr_t rtci_hdr;
     mlv_expo_hdr_t expo_hdr;
     mlv_lens_hdr_t lens_hdr;
-    mlv_elns_hdr_t elns_hdr;
     mlv_idnt_hdr_t idnt_hdr;
     mlv_wbal_hdr_t wbal_hdr;
     mlv_styl_hdr_t styl_hdr;
     mlv_vidf_hdr_t vidf_hdr;
+    mlv_elns_hdr_t *elns_hdr;
     FILE* save_file = NULL;   
     
     /* default case: use last filename */
@@ -489,7 +489,7 @@
     if (FIO_WriteFile(save_file, &rtci_hdr, rtci_hdr.blockSize) != (int)rtci_hdr.blockSize) goto write_error;
     if (FIO_WriteFile(save_file, &expo_hdr, expo_hdr.blockSize) != (int)expo_hdr.blockSize) goto write_error;
     if (FIO_WriteFile(save_file, &lens_hdr, lens_hdr.blockSize) != (int)lens_hdr.blockSize) goto write_error;
-    if (FIO_WriteFile(save_file, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) goto write_error;
+    if (FIO_WriteFile(save_file, elns_hdr, elns_hdr->blockSize) != (int)elns_hdr->blockSize) goto write_error;

     memset(&vidf_hdr, 0, sizeof(mlv_vidf_hdr_t));
     mlv_set_type((mlv_hdr_t *)&vidf_hdr, "VIDF");
diff --git a/scripts/lib/config.lua b/scripts/lib/config.lua
--- a/scripts/lib/config.lua
+++ b/scripts/lib/config.lua
@@ -79,8 +79,26 @@

end

+-- Load config from file if exists, otherwise create a new table in memory
local create_internal = function(default,thisfile)
+  local filename = string.format("%s%s.lcf", dryos.config_dir.path,thisfile)
+  local cfg = config.findConfig(filename)

+  if cfg == nil then
+    -- Create a config from scratch
+    cfg = {}
+    cfg.filename = filename
+    cfg.default = default -- TODO: Replicate .data's structure
+    cfg.data = {}
+    -- check for existing .cfg to load
+    setmetatable(cfg,config)
+    -- load previus config from file if available
+    cfg.data = cfg:load()
+    -- add to data structure
+    table.insert(config.configs,cfg)
+  end
+
+  return cfg
end

local function recursiveLoad(m,cfg)
@@ -108,33 +126,17 @@
]]
function config.create(default)
   local short_name = string.match(debug.getinfo(2,"S").short_src,"/([^/%.]+)%.[^/%.]+$")
-  local filename = string.format("%s%s.lcf", dryos.config_dir.path,short_name)
+  local cfg = create_internal(default,short_name)

-  local cfg = config.findConfig(filename)
-  if cfg ~= nil then
-    -- Append config data
-    for k,v in pairs(default) do
-      cfg.data[k] = v
-    end
-  else
-    -- Create a config from scratch
-    cfg = {}
-    cfg.filename = filename
-    cfg.default = default -- TODO: Replicate .data's structure
-    cfg.data = {}
-    -- check for existing .cfg to load
-    setmetatable(cfg,config)
-    cfg.data = cfg:load()
-    if cfg.data == nil then
-      -- Create a config from scratch
-      for k,v in pairs(default) do
-        cfg.data[k] = v
-      end
-    end
-    table.insert(config.configs,cfg)
-   end
+  -- Append config data
+  for k,v in pairs(default) do
+    cfg.default[k] = v
+    cfg.data[k] = v
+  end

-   return cfg
+  --  cfg.default = default -- TODO: Replicate .data's structure
+
+  return cfg
end

--[[---------------------------------------------------------------------------
@@ -158,37 +160,18 @@
     insertMenu(default,m)

     local short_name = string.match(debug.getinfo(2,"S").short_src,"/([^/%.]+)%.[^/%.]+$")
-    local filename = string.format("%s%s.lcf", dryos.config_dir.path,short_name)
+    local cfg = create_internal(default,short_name)

-    local cfg = config.findConfig(filename)
-    if cfg ~= nil then
-      -- Already present in config.configs, append menu
-      if cfg.data[m.name] ~= nil then
-        -- Avoid overwriting values when loading config form .cfg
-        cfg.data[m.name].menu = m
-        recursiveLoad(m,cfg.data[m.name])
-      else
-        insertMenu(cfg.data,m)
-      end
+    if cfg.data[m.name] == nil then
+      -- Create a config for menu from scratch
+      insertMenu(cfg.data,m)
     else
-      -- Create a config from scratch
-      cfg = {}
-      cfg.filename = filename
-      cfg.default = default -- TODO: Replicate .data's structure
-      cfg.data = {}
-      -- check for existing .cfg to load
-      setmetatable(cfg,config)
-      cfg.data = cfg:load()
-      if cfg.data == nil then
-        -- Create a config from scratch
-        insertMenu(cfg.data,m)
-      else
-        -- load values to menu
-        cfg.data[m.name].menu = m
-        recursiveLoad(m,cfg.data[m.name])
-      end
-      table.insert(config.configs,cfg)
-     end
+      -- Already present in config.configs, load values to menu from config
+      cfg.data[m.name].menu = m
+      recursiveLoad(m,cfg.data[m.name])
+    end
+
+    --    cfg.default = default -- TODO: Replicate .data's structure

     return cfg.data[m.name]
end
@@ -213,9 +196,6 @@
@function saving
]]
function config:saving()
-local short_name = string.match(debug.getinfo(2,"S").short_src,"/([^/%.]+)%.[^/%.]+$")
-local filename = string.format("%s%s.lcf", dryos.config_dir.path,short_name)
-
   -- Copy values of each menu
   for k,v in pairs(self.data) do
     -- k -> A table representing a menu entry or a single entry of a simple config
@@ -238,14 +218,15 @@
]]
function config:save()
     local f = io.open(self.filename,"w")
+    assert(f ~= nil, "Could not save config: "..self.filename)
+    -- Serialize data into a loadable format
     f:write("return ")
-    assert(f ~= nil, "Could not save config: "..self.filename)
-    config.serialize(f,self.data)
+    config.serialize(f,self.data,1)
     f:close()
end

--private
-function config.serialize(f,o)
+function config.serialize(f,o,lvl)
     if type(o) == "number" or type(o) == "boolean" then
         f:write(tostring(o))
     elseif type(o) == "string" then
@@ -254,13 +235,17 @@
         f:write("{\n")
         for k,v in pairs(o) do
           if k ~= "menu" then
-            f:write("\t[")
-            config.serialize(f,k)
+            -- Indent starting line
+            f:write(string.rep("\t", lvl))
+            f:write("[")
+            config.serialize(f,k,lvl+1)
             f:write("] = ")
-            config.serialize(f,v)
+            config.serialize(f,v,lvl+1)
             f:write(",\n")
           end
         end
+        -- Indent closing bracket
+        f:write(string.rep("\t", lvl-1))
         f:write("}")
     else
         --something we don't know how to serialize, just skip it


Tested those changes in manual_lens_info and adapted for crop_rec_4k.
Any Feedback is welcome

Good news Guys! Managed to discover what was preventing to save .mlv files some days ago and found a fix!

Currently elns struct is passed by value, so mlv_fill_elns will leaves struct initialised in memory, preventing FIOWriteFile to write correct datas...

Passing elns's struct argument as a double pointer and adapting related code will makes thing works:

Code Select Expand


void mlv_fill_elns(mlv_elns_hdr_t **hdr, uint64_t start_timestamp);


Tested on silent and mlv_lite; I need to see how to adapt mlv_rec and mlv_dump in next days.
@dfort Ill'try to send you a patch to test on crop_rec_4k

Uh oh, just discovered a problem. Extracting DNG files using "mlv_dump --dng" and looking at the DNG metadata with exiftool:

Code Select Expand

Lens Model                      : Zeiss Makro-Planar T* 2/100 ZF.


If there is an ELNS block mlv_dump should be using it instead of the LENS block. Also tried it with MLVFS and found the same issue.

Found where this is handled in dng.c: added in dng_fill_header() function
It's easy to adapt it to use elns instead if present, but it needs to be able to see datas by adding in dng.h:

Code Select Expand


struct frame_info
{
     ....

    /* block headers */
    ....

    mlv_elns_hdr_t elns_hdr;

    ....
};


ive just seen that there is some again fixed-length implementaition in manual_lens_info_64byte branch and manual_lens_info

We are experimenting dynamic length version on top of fake PR.
manual_lens_info_64byte was merged into manual_lens_info

is there any tool yet that processes the ELNS block?
i still want the name to be dynamic - just like some other blocks are.

Yes, mlv_dump from fake PR was modified to manage dynamic length with help from "length" field (which need to be removed as you said) to know how many chars to read and print.

Still need some help with the pointer of extended lens name (in mlv.c)... I think it is causing troubles like writing over wrong memory location when string copy is used

if I comment this line of codee:

Code Select Expand


if (FIO_WriteFile(save_file, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) goto write_error;     //silent.c
if (FIO_WriteFile(f, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) return 0;                     //mlv_lite.c

I get silent and mlv_lite to works, I can review mlv from pc and no ELNS block inside

Don't know why mlv_rec is not working anymore...

There should be something wrong with this code block:

Code Select Expand


void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp){
    /* Calculate total block length: header + fixed data + variable lensName string */
    int string_length = strlen(lens_info.name);
    int block_length = (string_length + sizeof(mlv_elns_hdr_t) + 3) & ~3;                    //Maybe this is causing troubles

    mlv_elns_hdr_t *header = malloc(block_length);

    /* prepare header */
    mlv_set_type((mlv_hdr_t *)header, "ELNS");
    mlv_set_timestamp((mlv_hdr_t *)header, start_timestamp);
    header->blockSize = block_length;
    header->length = string_length;

    header->focalLengthMin = lens_info.lens_focal_min;
    header->focalLengthMax = lens_info.lens_focal_max;
    header->apertureMin = RAW2VALUE(aperture, lens_info.raw_aperture_min) / 10.0;
    header->apertureMax = RAW2VALUE(aperture, lens_info.raw_aperture_max) / 10.0;
    header->version = lens_info.lens_version;
    header->extenderInfo = lens_info.lens_extender;
    header->capabilities = lens_info.lens_capabilities;
    header->chipped = lens_info.lens_exists;

    /* Store lensName string at the end of mlv_elns_hdr_t */
    char *lens_hdr_payload = (char *)&header[1];                  //Is this ok? Doesn't return a pointer right after header ending?
    strcpy(lens_hdr_payload, lens_info.name);                     //Will overwrite wrong memory region if above pointer is wrong

    /* update block with new values */
    hdr = header;
}

Resolving this issue will probably get us a step closer to getting ELNS (extended lens metadata?) working on the crop_rec_4k branch.

Using the revised version of ELNS block, I can't get silent pictures works, not simple neither FRSP.
Mlv video files are created and can be viewed from mlv_play when produced with mlv_rec.
No ELNS metadata is written to file....

Silent pictures code prints that error when block size doesn't match or can't write MLVI header. As example It get called from those line:

Code Select Expand


if (FIO_WriteFile(save_file, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) goto write_error;
or
if (!silent_write_mlv_chunk_headers(save_file, raw_info, 0)) goto write_error;


Yesterday I got an assert triggered with mlv_rec:

Code Select Expand


ASSERT((current_hdr->blockSize > 0) && (current_hdr->blockSize < 0x20000000));


I suspect that is an issues when calculating or checking block/header size... I remember that some time block size was printed as a negative number for some unknown reason...

@aprofiti - You have a good understanding of this. I only got the crop_rec_4k merge barely limping along. It would be great if you could create a branch on your fork and point us to it. Of course eventually make a pull request on one of the experimental branches in the main repository.

https://bitbucket.org/aprofiti/magic-lantern/pull-requests/1/crop-rec-4k-mlv-snd-elns/diff

Initially I tried to merge lua_fix into manual_lens_info to get fix for memory and latest updates, then merged into crop_rec_4k_mlv_snd but I was getting a lot of conflicts due to lua_fix not being merged into crop_rec_4k from some times...

So I started again from scratch and merged manual_lens_info directly into crop_rec and started to solve the conflicts we already managed in Dfort branch; also I have to check manually a lot diff introduces by merge tool, as it was considering manual_lens_info as most updated between two and picking automatically a lot of diff from it...

I noticed new conflicts from those files:
src/raw.c
src/property.c
src/menu.c
src/focus.c
and probably others I don't remember

I need someone who understand better than me (never worked on it) what was introduced into crop_rec_4k branch and review the PR.

Hoping g3ggo have time to dig into it and report.
@a1ex can you have a look?

Waiting for you guys, as I can't test this PR because my camera is not supported

And finally if you can try to merge this branch with the crop_rec_4k_mlv_snd?, I did that and only mlv.lite has some merge conflict.

My goal is to have sound with mlv compressed. Thank you for this monster merge

This seems to be easy to fix. It's because of a refactor from the branch you was merging

You can merge crop_rec_4k_mlv_snd, resolve merge conflicts using the new code and place this line in-between the block that was causing the conflict (for example at line number 3848):

Code Select Expand


fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&elns_hdr);

But this method it's not recommended at this stage, because merging will also remove some lines used to check and process "elns" block...

Code Select Expand


static GUARDED_BY(RawRecTask)   mlv_elns_hdr_t elns_hdr;
mlv_fill_elns(&elns_hdr, mlv_start_timestamp);


Use instead this patch I made from current status of dfort's branch:

Code Select Expand

diff --git a/modules/dual_iso/dual_iso.c b/modules/dual_iso/dual_iso.c
--- a/modules/dual_iso/dual_iso.c
+++ b/modules/dual_iso/dual_iso.c
@@ -69,6 +69,8 @@
#include <fileprefix.h>
#include <raw.h>
#include <patch.h>
+#include "../mlv_rec/mlv.h"
+#include "../mlv_rec/mlv_rec_interface.h"

static CONFIG_INT("isoless.hdr", isoless_hdr, 0);
static CONFIG_INT("isoless.iso", isoless_recovery_iso, 3);
@@ -667,6 +669,24 @@
     },
};

+/* callback routine for mlv_rec to add a custom DISO block after recording started (which already was specified in mlv.h in definition phase) */
+static void isoless_mlv_rec_cbr (uint32_t event, void *ctx, mlv_hdr_t *hdr)
+{
+    /* construct a free-able pointer to later pass it to mlv_rec_queue_block */
+    mlv_diso_hdr_t *dual_iso_block = malloc(sizeof(mlv_diso_hdr_t));
+   
+    /* set the correct type and size */
+    mlv_set_type((mlv_hdr_t *)dual_iso_block, "DISO");
+    dual_iso_block->blockSize = sizeof(mlv_diso_hdr_t);
+   
+    /* and fill with data */
+    dual_iso_block->dualMode = dual_iso_is_active();
+    dual_iso_block->isoValue = isoless_recovery_iso;
+   
+    /* finally pass it to mlv_rec which will free the block when it has been processed */
+    mlv_rec_queue_block((mlv_hdr_t *)dual_iso_block);
+}
+
static unsigned int isoless_init()
{
     if (is_camera("5D3", "1.1.3") || is_camera("5D3", "1.2.3"))
@@ -985,6 +1005,9 @@
         isoless_hdr = 0;
         return 1;
     }
+   
+    mlv_rec_register_cbr(MLV_REC_EVENT_PREPARING, &isoless_mlv_rec_cbr, NULL);
+   
     return 0;
}

diff --git a/modules/mlv_lite/mlv_lite.c b/modules/mlv_lite/mlv_lite.c
--- a/modules/mlv_lite/mlv_lite.c
+++ b/modules/mlv_lite/mlv_lite.c
@@ -42,6 +42,8 @@
  * Boston, MA  02110-1301, USA.
  */

+#define __MLV_LITE_C__
+
#include <module.h>
#include <dryos.h>
#include <property.h>
@@ -64,11 +66,13 @@
#include "focus.h"
#include "fps.h"
#include "../mlv_rec/mlv.h"
+#include "../mlv_rec/mlv_rec_interface.h"
#include "../trace/trace.h"
#include "powersave.h"
#include "shoot.h"
#include "fileprefix.h"
#include "timer.h"
+#include "ml-cbr.h"
#include "../silent/lossless.h"
#include "ml-cbr.h"

@@ -100,7 +104,6 @@
static int cam_5d3 = 0;
static int cam_5d3_113 = 0;
static int cam_5d3_123 = 0;
-
/**
  * resolution (in pixels) should be multiple of 16 horizontally (see http://www.magiclantern.fm/forum/index.php?topic=5839.0)
  * furthermore, resolution (in bytes) should be multiple of 8 in order to use the fastest EDMAC flags ( http://magiclantern.wikia.com/wiki/Register_Map#EDMAC ),
@@ -152,6 +155,7 @@
static CONFIG_INT("raw.small.hacks", small_hacks, 1);

static CONFIG_INT("raw.h264.proxy", h264_proxy_menu, 0);
+static CONFIG_INT("raw.sync_beep", sync_beep, 1);

static CONFIG_INT("raw.output_format", output_format, 3);
#define OUTPUT_14BIT_NATIVE 0
@@ -305,10 +309,12 @@
                            max_frame_size for uncompressed data, lower for compressed */
     int payload_size;   /* size effectively used by image data */
     int frame_number;   /* from 0 to n */
+    int is_meta;        /* when used by some other module and does not contain VIDF, disables consistency checks used for video frame slots */
     enum {
         SLOT_FREE,          /* available for image capture */
         SLOT_RESERVED,      /* it may become available when resizing the previous slots */
         SLOT_CAPTURING,     /* in progress */
+        SLOT_LOCKED,        /* locked by some other module */
         SLOT_FULL,          /* contains fully captured image data */
         SLOT_WRITING        /* it's being saved to card */
     } status;
@@ -326,6 +332,7 @@
static GUARDED_BY(LiveViewTask) int capture_slot = -1;              /* in what slot are we capturing now (index) */
static volatile                 int force_new_buffer = 0;           /* if some other task decides it's better to search for a new buffer */

+
static GUARDED_BY(LiveViewTask) int writing_queue[COUNT(slots)+1];  /* queue of completed frames (slot indices) waiting to be saved */
static GUARDED_BY(LiveViewTask) int writing_queue_tail = 0;         /* place captured frames here */
static GUARDED_BY(RawRecTask)   int writing_queue_head = 0;         /* extract frames to be written from here */
@@ -341,6 +348,7 @@
static GUARDED_BY(RawRecTask)   int writing_time = 0;               /* time spent by raw_video_rec_task in FIO_WriteFile calls */
static GUARDED_BY(RawRecTask)   int idle_time = 0;                  /* time spent by raw_video_rec_task doing something else */
static volatile                 uint32_t edmac_active = 0;
+static volatile                 uint32_t skip_frames = 0;

/* for compress_task */
static struct msg_queue * compress_mq = 0;
@@ -350,7 +358,7 @@
static GUARDED_BY(RawRecTask)   mlv_rawc_hdr_t rawc_hdr;
static GUARDED_BY(RawRecTask)   mlv_idnt_hdr_t idnt_hdr;
static GUARDED_BY(RawRecTask)   mlv_expo_hdr_t expo_hdr;
-static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
+static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
static GUARDED_BY(RawRecTask)   mlv_elns_hdr_t elns_hdr;
static GUARDED_BY(RawRecTask)   mlv_rtci_hdr_t rtci_hdr;
static GUARDED_BY(RawRecTask)   mlv_wbal_hdr_t wbal_hdr;
@@ -358,12 +366,180 @@
static GUARDED_BY(RawRecTask)   uint64_t mlv_start_timestamp = 0;
        GUARDED_BY(RawRecTask)   uint32_t raw_rec_trace_ctx = TRACE_ERROR;

-/* interface to other modules: these are called when recording starts or stops  */
-extern WEAK_FUNC(ret_0) unsigned int raw_rec_cbr_starting();
-extern WEAK_FUNC(ret_0) unsigned int raw_rec_cbr_stopping();
-
static int raw_rec_should_preview(void);

+/* old mlv_rec interface stuff here */
+struct msg_queue *mlv_block_queue = NULL;
+/* registry of all other modules CBRs */
+static cbr_entry_t registered_cbrs[32];
+
+
+/* allow modules to set how many frames should be skipped */
+void mlv_rec_skip_frames(uint32_t count)
+{
+    skip_frames = count;
+}
+
+/* register a callback function that is called when one of the events specified happens.
+   event can be a OR'ed list of the events specified in mlv_rec_interface.h
+ */
+uint32_t mlv_rec_register_cbr(uint32_t event, event_cbr_t cbr, void *ctx)
+{
+    if(RAW_IS_RECORDING)
+    {
+        return 0;
+    }
+   
+    uint32_t ret = 0;
+    uint32_t old_int = cli();
+    for(int pos = 0; pos < COUNT(registered_cbrs); pos++)
+    {
+        if(registered_cbrs[pos].cbr == NULL)
+        {
+            registered_cbrs[pos].event = event;
+            registered_cbrs[pos].cbr = cbr;
+            registered_cbrs[pos].ctx = ctx;
+            ret = 1;
+            break;
+        }
+    }
+    sei(old_int);
+   
+    return ret;
+}
+
+/* unregister the specified CBR from all registered events */
+uint32_t mlv_rec_unregister_cbr(event_cbr_t cbr)
+{
+    if(RAW_IS_RECORDING)
+    {
+        return 0;
+    }
+   
+    uint32_t ret = 0;
+    uint32_t old_int = cli();
+    for(int pos = 0; (registered_cbrs[pos].cbr != NULL) && (pos < COUNT(registered_cbrs)); pos++)
+    {
+        /* is this the callback routine to be unregistered? */
+        if(registered_cbrs[pos].cbr == cbr)
+        {
+            /* if so, just shift all entries below one entry up. this keeps a void-less list with all CBRs to improve performance */
+            int32_t remaining = COUNT(registered_cbrs) - pos - 1;
+           
+            registered_cbrs[pos].cbr = NULL;
+           
+            if(remaining > 0)
+            {
+                memcpy(&registered_cbrs[pos], &registered_cbrs[pos + 1], remaining * sizeof(cbr_entry_t));
+                registered_cbrs[COUNT(registered_cbrs)-1].ctx = NULL;
+                ret = 1;
+                break;
+            }
+        }
+    }
+    sei(old_int);
+   
+    return ret;
+}
+
+/* call registered callbacks for the events specified */
+static void mlv_rec_call_cbr(uint32_t event, mlv_hdr_t *hdr)
+{
+    for(int pos = 0; (registered_cbrs[pos].cbr != NULL) && (pos < COUNT(registered_cbrs)); pos++)
+    {
+        /* there is still a possible race condition - if a module unregisters it's CBR during this function being called.
+           while this is unlikely to ever happen (all current modules register their CBRs upon init and not within a CBR,
+           this is still something that should be hardened. locks might be a bit too expensive.
+           copying every entry to stack also a bit costly, but will cause other side effects.
+        */
+        if(registered_cbrs[pos].event & event)
+        {
+            registered_cbrs[pos].cbr(event, registered_cbrs[pos].ctx, hdr);
+        }
+    }
+}
+
+/* helper to write a MLV block into a FILE* and return if successful */
+static int32_t mlv_write_hdr(FILE* f, mlv_hdr_t *hdr)
+{
+    mlv_rec_call_cbr(MLV_REC_EVENT_BLOCK, hdr);
+
+    uint32_t written = FIO_WriteFile(f, hdr, hdr->blockSize);
+
+    return written == hdr->blockSize;
+}
+
+/* return details about allocated slot */
+void mlv_rec_get_slot_info(int32_t slot, uint32_t *size, void **address)
+{
+    if(slot < 0 || slot >= total_slot_count)
+    {
+        *address = NULL;
+        *size = 0;
+        return;
+    }
+   
+    *address = slots[slot].ptr;
+    *size = slots[slot].size;
+}
+
+/* this can be called from anywhere to get a free memory slot. must be submitted using mlv_rec_release_slot() */
+int32_t mlv_rec_get_free_slot()
+{
+    int32_t ret = -1;
+   
+    for (int i = 0; (i < total_slot_count) && (ret == -1); i++)
+    {
+        uint32_t old_int = cli();
+        if (slots[i].status == SLOT_FREE)
+        {
+            slots[i].status = SLOT_LOCKED;
+            ret = i;
+        }
+        sei(old_int);
+    }
+   
+    return ret;
+}
+
+/* mark a previously with mlv_rec_get_free_slot() allocated slot for being reused or written into the file */
+void mlv_rec_release_slot(int32_t slot, uint32_t write)
+{
+    if(slot < 0 || slot >= total_slot_count)
+    {
+        return;
+    }
+
+    if(write)
+    {
+        uint32_t old_int = cli();
+        slots[slot].status = SLOT_FULL;
+        slots[slot].is_meta = 1;
+        writing_queue[writing_queue_tail] = slot;
+        INC_MOD(writing_queue_tail, COUNT(writing_queue));
+        sei(old_int);
+    }
+    else
+    {
+        slots[slot].status = SLOT_FREE;
+    }
+}
+
+/* set the timestamp relative to recording start */
+void mlv_rec_set_rel_timestamp(mlv_hdr_t *hdr, uint64_t timestamp)
+{
+    hdr->timestamp = timestamp - mlv_start_timestamp;
+}
+
+/* queuing of blocks from other modules */
+uint32_t mlv_rec_queue_block(mlv_hdr_t *hdr)
+{
+    mlv_set_timestamp(hdr, mlv_start_timestamp);
+    msg_queue_post(mlv_block_queue, (uint32_t) hdr);
+   
+    return 1;
+}
+
static inline int pre_recording_buffer_full()
{
     /* fixme: not very accurate with variable frame sizes */
@@ -1460,9 +1636,11 @@
             }

             int color = slots[i].status == SLOT_FREE      ? COLOR_GRAY(10) :
+                        slots[i].is_meta                  ? COLOR_BLUE :
                         slots[i].status == SLOT_WRITING   ? COLOR_GREEN1 :
                         slots[i].status == SLOT_FULL      ? COLOR_LIGHT_BLUE :
                         slots[i].status == SLOT_RESERVED  ? COLOR_GRAY(50) :
+                        slots[i].status == SLOT_LOCKED    ? COLOR_YELLOW :
                                                             COLOR_RED ;

             uint32_t x1 = (uint32_t) slots[i].ptr - chunk_start;
@@ -2578,6 +2756,19 @@
         frame_count++;
         return;
     }
+
+    /* some modules may do some specific stuff right when we started recording */
+    if (frame_count == 1)
+    {
+        mlv_rec_call_cbr(MLV_REC_EVENT_STARTED, NULL);
+    }
+
+    /* other modules can ask for some frames to skip, e.g. for syncing audio */
+    if (skip_frames > 0)
+    {
+        skip_frames--;
+        return;
+    }
     
     if (edmac_active)
     {
@@ -2670,7 +2861,7 @@
     if (!RAW_IS_RECORDING) return 0;
     if (!raw_lv_settings_still_valid()) { raw_recording_state = RAW_FINISHING; return 0; }
     if (buffer_full) return 0;
-
+   
     /* double-buffering */
     raw_lv_redirect_edmac(fullsize_buffers[fullsize_buffer_pos % 2]);

@@ -2828,7 +3019,7 @@

     mlv_fill_idnt(&idnt_hdr, mlv_start_timestamp);
     mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
-    mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+    mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
     mlv_fill_elns(&elns_hdr, mlv_start_timestamp);
     mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
     mlv_fill_wbal(&wbal_hdr, mlv_start_timestamp);
@@ -2841,18 +3032,49 @@
}

static REQUIRES(RawRecTask)
-int write_mlv_chunk_headers(FILE* f)
+int write_mlv_chunk_headers(FILE* f, int chunk)
{
-    if (FIO_WriteFile(f, &file_hdr, file_hdr.blockSize) != (int)file_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &rawi_hdr, rawi_hdr.blockSize) != (int)rawi_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &rawc_hdr, rawc_hdr.blockSize) != (int)rawc_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &idnt_hdr, idnt_hdr.blockSize) != (int)idnt_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &expo_hdr, expo_hdr.blockSize) != (int)expo_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &lens_hdr, lens_hdr.blockSize) != (int)lens_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &rtci_hdr, rtci_hdr.blockSize) != (int)rtci_hdr.blockSize) return 0;
-    if (FIO_WriteFile(f, &wbal_hdr, wbal_hdr.blockSize) != (int)wbal_hdr.blockSize) return 0;
-    if (mlv_write_vers_blocks(f, mlv_start_timestamp)) return 0;
+    /* looks a bit cleaner not to have several return points */
+    int fail = 0;
+   
+    /* all chunks contain the MLVI header */
+    fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&file_hdr);
+   
+    /* only the first chunk contains this information if nothing changes */
+    if(chunk == 0)
+    {
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&rawi_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&rawc_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&idnt_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&expo_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&lens_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&elns_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&rtci_hdr);
+        fail |= !mlv_write_hdr(f, (mlv_hdr_t *)&wbal_hdr);
+        fail |= mlv_write_vers_blocks(f, mlv_start_timestamp);
+    }
+   
+    /* write all queued blocks, if any */
+    uint32_t msg_count = 0;
+    msg_queue_count(mlv_block_queue, &msg_count);
+   
+    for(uint32_t msg = 0; msg < msg_count; msg++)
+    {
+        mlv_hdr_t *block = NULL;
+
+        /* there is a block in the queue, try to get that block */
+        if(!msg_queue_receive(mlv_block_queue, &block, 0))
+        {
+            fail |= !mlv_write_hdr(f, block);
+            free(block);
+        }
+    }
+   
+    /* any of the above writes failed, exit */
+    if(fail)
+    {
+        return 0;
+    }
     
     int hdr_size = FIO_SeekSkipFile(f, 0, SEEK_CUR);
     
@@ -2874,13 +3096,19 @@
void finish_chunk(FILE* f)
{
     file_hdr.videoFrameCount = chunk_frame_count;
+   
+    /* call the CBRs which may update fields */
+    mlv_rec_call_cbr(MLV_REC_EVENT_BLOCK, (mlv_hdr_t *)&file_hdr);
+   
     FIO_SeekSkipFile(f, 0, SEEK_SET);
     FIO_WriteFile(f, &file_hdr, file_hdr.blockSize);
     FIO_CloseFile(f);
     chunk_frame_count = 0;
}

-/* This saves a group of frames, also taking care of file splitting if required */
+/* This saves a group of frames, also taking care of file splitting if required.
+   Parameter num_frames is meant for counting the VIDF blocks for updating MLVI header.
+ */
static REQUIRES(RawRecTask)
int write_frames(FILE** pf, void* ptr, int group_size, int num_frames)
{
@@ -2897,7 +3125,7 @@
         if (!g) return 0;
         
         file_hdr.fileNum = mlv_chunk;
-        written_chunk = write_mlv_chunk_headers(g);
+        written_chunk = write_mlv_chunk_headers(g, mlv_chunk);
         written_total += written_chunk;
         
         if (written_chunk)
@@ -2954,7 +3182,7 @@
         if (!g) return 0;
         
         file_hdr.fileNum = mlv_chunk;
-        written_chunk = write_mlv_chunk_headers(g);
+        written_chunk = write_mlv_chunk_headers(g, mlv_chunk);
         written_total += written_chunk;
         
         int r2 = written_chunk ? FIO_WriteFile(g, ptr, group_size) : 0;
@@ -3012,6 +3240,7 @@
     raw_recording_state = RAW_PREPARING;
     give_semaphore(settings_sem);

+    mlv_rec_call_cbr(MLV_REC_EVENT_PREPARING, NULL);
     /* locals */
     FILE* f = 0;
     int last_block_size = 0; /* for detecting early stops */
@@ -3087,25 +3316,25 @@
         goto cleanup;
     }

+    /* Need to start the recording of audio before the init of the mlv chunk */
+    mlv_rec_call_cbr(MLV_REC_EVENT_STARTING, NULL);
+
     init_mlv_chunk_headers(&raw_info);
-    written_total = written_chunk = write_mlv_chunk_headers(f);
+    written_total = written_chunk = write_mlv_chunk_headers(f, mlv_chunk);
     if (!written_chunk)
     {
         NotifyBox(5000, "Card Full");
         goto cleanup;
     }
-
+   
     hack_liveview(0);
     liveview_hacked = 1;

-    /* this will enable the vsync CBR and the other task(s) */
-    raw_recording_state = pre_record ? RAW_PRE_RECORDING : RAW_RECORDING;
-
     /* try a sync beep (not very precise, but better than nothing) */
-    beep();
-
-    /* signal that we are starting */
-    raw_rec_cbr_starting();
+    if(sync_beep)
+    {
+        beep();
+    }

     /* signal start of recording to the compression task */
     msg_queue_post(compress_mq, INT_MAX);
@@ -3116,6 +3345,9 @@
     int fps = fps_get_current_x1000();
     
     int last_processed_frame = 0;
+
+    /* this will enable the vsync CBR and the other task(s) */
+    raw_recording_state = pre_record ? RAW_PRE_RECORDING : RAW_RECORDING;
     
     /* main recording loop */
     while (RAW_IS_RECORDING && lv)
@@ -3173,6 +3405,7 @@
         int last_grouped = w_head;
         
         int group_size = 0;
+        int meta_slots = 0;
         for (int i = w_head; i != w_tail; INC_MOD(i, COUNT(writing_queue)))
         {
             int slot_index = writing_queue[i];
@@ -3184,13 +3417,21 @@
                 break;
             }

-            /* consistency checks */
-            ASSERT(((mlv_vidf_hdr_t*)slots[slot_index].ptr)->blockSize == (uint32_t) slots[slot_index].size);
-            ASSERT(((mlv_vidf_hdr_t*)slots[slot_index].ptr)->frameNumber == (uint32_t) slots[slot_index].frame_number - 1);
-
-            if (OUTPUT_COMPRESSION)
+            /* consistency checks for VIDF slots */
+            if (!slots[slot_index].is_meta)
             {
-                ASSERT(slots[slot_index].size < max_frame_size);
+                ASSERT(((mlv_vidf_hdr_t*)slots[slot_index].ptr)->blockSize == (uint32_t) slots[slot_index].size);
+                ASSERT(((mlv_vidf_hdr_t*)slots[slot_index].ptr)->frameNumber == (uint32_t) slots[slot_index].frame_number - 1);
+               
+                if (OUTPUT_COMPRESSION)
+                {
+                    ASSERT(slots[slot_index].size < max_frame_size);
+                }
+            }
+            else
+            {
+                /* count the number of slots being non-VIDF */
+                meta_slots++;
             }

             /* TBH, I don't care if these are part of the same group or not,
@@ -3243,7 +3484,7 @@
         {
             int slot_index = writing_queue[i];

-            if (OUTPUT_COMPRESSION)
+            if (OUTPUT_COMPRESSION && !slots[slot_index].is_meta)
             {
                 ASSERT(slots[slot_index].size < max_frame_size);
             }
@@ -3263,7 +3504,7 @@
         idle_time += t0 - last_write_timestamp;

         /* save a group of frames and measure execution time */
-        if (!write_frames(&f, ptr, group_size, num_frames))
+        if (!write_frames(&f, ptr, group_size, num_frames - meta_slots))
         {
             goto abort;
         }
@@ -3287,23 +3528,30 @@
             
             int slot_index = writing_queue[i];

-            if (frame_check_saved(slot_index) != 1)
+            if (!slots[slot_index].is_meta)
             {
-                bmp_printf( FONT_MED, 30, 110,
-                    "Data corruption at slot %d, frame %d ", slot_index, slots[slot_index].frame_number
-                );
-                beep();
+                if (frame_check_saved(slot_index) != 1)
+                {
+                    bmp_printf( FONT_MED, 30, 110,
+                        "Data corruption at slot %d, frame %d ", slot_index, slots[slot_index].frame_number
+                    );
+                    beep();
+                }
+               
+                if (slots[slot_index].frame_number != last_processed_frame + 1)
+                {
+                    bmp_printf( FONT_MED, 30, 110,
+                        "Frame order error: slot %d, frame %d, expected %d ", slot_index, slots[slot_index].frame_number, last_processed_frame + 1
+                    );
+                    beep();
+                }
+                last_processed_frame++;
+            }
+            else
+            {
+                slots[slot_index].is_meta = 0;
             }
             
-            if (slots[slot_index].frame_number != last_processed_frame + 1)
-            {
-                bmp_printf( FONT_MED, 30, 110,
-                    "Frame order error: slot %d, frame %d, expected %d ", slot_index, slots[slot_index].frame_number, last_processed_frame + 1
-                );
-                beep();
-            }
-            last_processed_frame++;
-
             free_slot(slot_index);
         }
         
@@ -3346,18 +3594,16 @@
     /* make sure the user doesn't rush to turn off the camera or something */
     gui_uilock(UILOCK_EVERYTHING);
     
-    /* signal that we are stopping */
-    raw_rec_cbr_stopping();
-   
     /* done, this will stop the vsync CBR and the copying task */
     raw_recording_state = RAW_FINISHING;
+    mlv_rec_call_cbr(MLV_REC_EVENT_STOPPING, NULL);

     /* wait until the other tasks calm down */
     wait_lv_frames(2);

     /* signal end of recording to the compression task */
     msg_queue_post(compress_mq, INT_MIN);
-
+   
     set_recording_custom(CUSTOM_RECORDING_NOT_RECORDING);

     if (!RECORDING_H264)
@@ -3368,6 +3614,28 @@
         /* PauseLiveView breaks UI locks - why? */
         gui_uilock(UILOCK_EVERYTHING);
     }
+   
+    /* write all queued blocks, if any */
+    uint32_t msg_count = 0;
+    msg_queue_count(mlv_block_queue, &msg_count);
+   
+    for(uint32_t msg = 0; msg < msg_count; msg++)
+    {
+        mlv_hdr_t *block = NULL;
+
+        /* there is a block in the queue, try to get that block */
+        if(!msg_queue_receive(mlv_block_queue, &block, 0))
+        {
+            /* when this block will get written, call the CBR */
+            mlv_rec_call_cbr(MLV_REC_EVENT_BLOCK, block);
+           
+            /* use the write func to write the block */
+            write_frames(&f, block, block->blockSize, 0);
+           
+            /* free the block */
+            free(block);
+        }
+    }

     /* write remaining frames */
     /* H.264: we will be recording black frames during this time,
@@ -3388,30 +3656,37 @@
             beep();
         }

-        if (frame_check_saved(slot_index) != 1)
+        /* video frame consistency checks only for VIDF */
+        if(!slots[slot_index].is_meta)
         {
-            bmp_printf( FONT_MED, 30, 110,
-                "Data corruption at slot %d, frame %d ", slot_index, slots[slot_index].frame_number
-            );
-            beep();
+            if (frame_check_saved(slot_index) != 1)
+            {
+                bmp_printf( FONT_MED, 30, 110,
+                    "Data corruption at slot %d, frame %d ", slot_index, slots[slot_index].frame_number
+                );
+                beep();
+            }
+
+            if (slots[slot_index].frame_number != last_processed_frame + 1)
+            {
+                bmp_printf( FONT_MED, 30, 110,
+                    "Frame order error: slot %d, frame %d, expected %d ", slot_index, slots[slot_index].frame_number, last_processed_frame + 1
+                );
+                beep();
+            }
+            last_processed_frame++;
+           
+            /* if it's a VIDF, then it should be smaller than the max frame size when compression is enabled */
+            if (OUTPUT_COMPRESSION)
+            {
+                ASSERT(slots[slot_index].size < max_frame_size);
+            }
         }
-
-        if (slots[slot_index].frame_number != last_processed_frame + 1)
-        {
-            bmp_printf( FONT_MED, 30, 110,
-                "Frame order error: slot %d, frame %d, expected %d ", slot_index, slots[slot_index].frame_number, last_processed_frame + 1
-            );
-            beep();
-        }
-        last_processed_frame++;
-
+       
         slots[slot_index].status = SLOT_WRITING;
-        if (OUTPUT_COMPRESSION)
-        {
-            ASSERT(slots[slot_index].size < max_frame_size);
-        }
+       
         if (indicator_display == INDICATOR_RAW_BUFFER) show_buffer_status();
-        if (!write_frames(&f, slots[slot_index].ptr, slots[slot_index].size, 1))
+        if (!write_frames(&f, slots[slot_index].ptr, slots[slot_index].size, slots[slot_index].is_meta ? 0 : 1))
         {
             NotifyBox(5000, "Card Full");
             beep();
@@ -3466,6 +3741,7 @@
     ResumeLiveView();
     redraw();
     raw_recording_state = RAW_IDLE;
+    mlv_rec_call_cbr(MLV_REC_EVENT_STOPPED, NULL);
}

static REQUIRES(GuiMainTask)
@@ -3639,6 +3915,13 @@
                 .advanced = 1,
             },
             {
+                .name = "Sync beep",
+                .priv = &sync_beep,
+                .max    = 1,
+                .help = "Beeps on recording start for better sync.",
+                .advanced = 1,
+            },
+            {
                 .name   = "Show EDMAC",
                 .priv   = &show_edmac,
                 .max    = 1,
@@ -4026,6 +4309,9 @@
         }
         NotifyBoxHide();
     }
+   
+    /* allocate queue that other modules will fill with blocks to write to the current file */
+    mlv_block_queue = (struct msg_queue *) msg_queue_create("mlv_block_queue", 100);

     lossless_init();

@@ -4068,6 +4354,7 @@
     MODULE_CONFIG(use_srm_memory)
     MODULE_CONFIG(small_hacks)
     MODULE_CONFIG(warm_up)
+    MODULE_CONFIG(sync_beep)
     MODULE_CONFIG(output_format)
     MODULE_CONFIG(h264_proxy_menu)
MODULE_CONFIGS_END()
diff --git a/modules/mlv_rec/mlv.h b/modules/mlv_rec/mlv.h
--- a/modules/mlv_rec/mlv.h
+++ b/modules/mlv_rec/mlv.h
@@ -307,6 +307,7 @@
void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
+void mlv_fill_wavi(mlv_wavi_hdr_t *hdr, uint64_t start_timestamp);  /* provided by mlv_snd */
void mlv_build_vers(mlv_vers_hdr_t **hdr, uint64_t start_timestamp, const char *version_string);

/* randomize the 64 bits passed in parameter using LFSR */
diff --git a/modules/mlv_rec/mlv_dump.c b/modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c
+++ b/modules/mlv_rec/mlv_dump.c
@@ -1917,6 +1917,7 @@
     /* start processing */
     lv_rec_file_footer_t lv_rec_footer;
     mlv_file_hdr_t main_header;
+    mlv_diso_hdr_t diso_info;
     mlv_lens_hdr_t lens_info;
     mlv_elns_hdr_t elns_info;
     mlv_expo_hdr_t expo_info;
@@ -1929,6 +1930,7 @@

     /* initialize stuff */
     memset(&lv_rec_footer, 0x00, sizeof(lv_rec_file_footer_t));
+    memset(&diso_info, 0x00, sizeof(mlv_diso_hdr_t));
     memset(&lens_info, 0x00, sizeof(mlv_lens_hdr_t));
     memset(&elns_info, 0x00, sizeof(mlv_elns_hdr_t));
     memset(&expo_info, 0x00, sizeof(mlv_expo_hdr_t));
@@ -4093,12 +4095,22 @@
                         print_msg(MSG_ERROR, "Failed writing into .WAV file\n");
                         goto abort;
                     }
-
+                   
                     /* init WAV data size, will be grow later block by block (AUDF) */
                     wav_data_size = 0;
                     wav_header_size = file_get_pos(out_file_wav); /* same as sizeof(struct wav_header) */
                 }
             }
+            else if(!memcmp(mlv_block->blockType, "DISO", 4))
+            {
+                diso_info = *(mlv_diso_hdr_t *)mlv_block;
+
+                if(verbose)
+                {
+                    print_msg(MSG_INFO, "     Mode:        %d\n", diso_info.dualMode);
+                    print_msg(MSG_INFO, "     ISO Value:   %d\n", diso_info.isoValue);
+                }
+            }
             else if(!memcmp(mlv_block->blockType, "NULL", 4))
             {
                 /* those are just placeholders. ignore them. */
diff --git a/modules/mlv_rec/mlv_rec.c b/modules/mlv_rec/mlv_rec.c
--- a/modules/mlv_rec/mlv_rec.c
+++ b/modules/mlv_rec/mlv_rec.c
@@ -53,6 +53,8 @@
//#define CONFIG_CONSOLE
//#define TRACE_DISABLED

+#define __MLV_REC_C__
+
#include <module.h>
#include <dryos.h>
#include <property.h>
@@ -79,6 +81,7 @@
#include "../trace/trace.h"

#include "mlv.h"
+#include "mlv_rec_interface.h"
#include "mlv_rec.h"

/* an alternative tracing method that embeds the logs into the MLV file itself */
@@ -123,6 +126,8 @@
static uint32_t mlv_writer_threads = 2;
static uint32_t mlv_max_filesize = 0xFFFFFFFF;
static uint32_t abort_test = 0;
+static uint32_t skip_frames = 0;
+

uint32_t raw_rec_trace_ctx = TRACE_ERROR;

@@ -253,6 +258,83 @@

static volatile int32_t frame_countdown = 0;          /* for waiting X frames */

+/* registry of all other modules CBRs */
+static cbr_entry_t registered_cbrs[32];
+
+uint32_t mlv_rec_register_cbr(uint32_t event, event_cbr_t cbr, void *ctx)
+{
+    if(RAW_IS_RECORDING)
+    {
+        return 0;
+    }
+   
+    uint32_t ret = 0;
+    uint32_t old_int = cli();
+    for(int pos = 0; pos < COUNT(registered_cbrs); pos++)
+    {
+        if(registered_cbrs[pos].cbr == NULL)
+        {
+            registered_cbrs[pos].event = event;
+            registered_cbrs[pos].cbr = cbr;
+            registered_cbrs[pos].ctx = ctx;
+            ret = 1;
+            break;
+        }
+    }
+    sei(old_int);
+   
+    return ret;
+}
+
+uint32_t mlv_rec_unregister_cbr(event_cbr_t cbr)
+{
+    if(RAW_IS_RECORDING)
+    {
+        return 0;
+    }
+   
+    uint32_t ret = 0;
+    uint32_t old_int = cli();
+    for(int pos = 0; (registered_cbrs[pos].cbr != NULL) && (pos < COUNT(registered_cbrs)); pos++)
+    {
+        if(registered_cbrs[pos].cbr == cbr)
+        {
+            int32_t remaining = COUNT(registered_cbrs) - pos - 1;
+           
+            registered_cbrs[pos].cbr = NULL;
+           
+            if(remaining > 0)
+            {
+                memcpy(&registered_cbrs[pos], &registered_cbrs[pos + 1], remaining * sizeof(cbr_entry_t));
+                registered_cbrs[COUNT(registered_cbrs)-1].ctx = NULL;
+                ret = 1;
+                break;
+            }
+        }
+    }
+    sei(old_int);
+   
+    return ret;
+}
+
+static void mlv_rec_call_cbr(uint32_t event, mlv_hdr_t *hdr)
+{
+    for(int pos = 0; (registered_cbrs[pos].cbr != NULL) && (pos < COUNT(registered_cbrs)); pos++)
+    {
+        if(registered_cbrs[pos].event & event)
+        {
+            registered_cbrs[pos].cbr(event, registered_cbrs[pos].ctx, hdr);
+        }
+    }
+}
+
+/* allow modules to set how many frames should be skipped */
+void mlv_rec_skip_frames(uint32_t count)
+{
+    skip_frames = count;
+}
+
+
#if defined(EMBEDDED_LOGGING)
/* START: helper code for logging into MLV files */
static uint8_t *mlv_debg_buffer = NULL;
@@ -921,6 +1003,7 @@
             mlv_hdr_t *write_align_hdr = (mlv_hdr_t *)((uint32_t)vidf_hdr + vidf_hdr->blockSize);
             memset(write_align_hdr, 0xA5, write_size_align);
             mlv_set_type(write_align_hdr, "NULL");
+            write_align_hdr->timestamp = 0;
             write_align_hdr->blockSize = write_size_align;
         }

@@ -1443,6 +1526,8 @@
             mlv_rtci_hdr_t *rtci_hdr = malloc(sizeof(mlv_rtci_hdr_t));
             mlv_fill_rtci(rtci_hdr, mlv_start_timestamp);
             msg_queue_post(mlv_block_queue, (uint32_t) rtci_hdr);
+           
+            mlv_rec_call_cbr(MLV_REC_EVENT_CYCLIC, NULL);
         }

         if(should_run_polling_action(MLV_INFO_BLOCK_INTERVAL, &block_queueing) && (mlv_metadata & MLV_METADATA_CYCLIC))
@@ -1761,9 +1846,12 @@
     }
}

-void mlv_rec_queue_block(mlv_hdr_t *hdr)
+uint32_t mlv_rec_queue_block(mlv_hdr_t *hdr)
{
+    mlv_set_timestamp(hdr, mlv_start_timestamp);
     msg_queue_post(mlv_block_queue, (uint32_t) hdr);
+   
+    return 1;
}

void mlv_rec_set_rel_timestamp(mlv_hdr_t *hdr, uint64_t timestamp)
@@ -1851,6 +1939,7 @@

     /* set old header to a skipped header format */
     mlv_set_type((mlv_hdr_t *)vidf, "NULL");
+    vidf->timestamp = 0;

     /* backup old size into free space */
     ((uint32_t*) vidf)[sizeof(mlv_vidf_hdr_t)/4] = vidf->blockSize;
@@ -2127,10 +2216,14 @@

static void mlv_rec_dma_cbr_r(void *ctx)
{
+   
}

static void mlv_rec_dma_cbr_w(void *ctx)
{
+    /* call the VIDF CBRs */
+    mlv_rec_call_cbr(MLV_REC_EVENT_VIDF, slots[capture_slot].ptr);
+
     mlv_rec_dma_active = 0;
     
     mlv_rec_dma_end = get_us_clock();
@@ -2147,6 +2240,11 @@
         frame_count++;
         return 0;
     }
+   
+    if(frame_count == 1)
+    {
+        mlv_rec_call_cbr(MLV_REC_EVENT_STARTED, NULL);
+    }

     /* where to save the next frame? */
     capture_slot = choose_next_capture_slot(capture_slot);
@@ -2217,6 +2315,13 @@
         return 0;
     }
     
+ /* other modules can ask for some frames to skip, e.g. for syncing audio */
+    if(skip_frames > 0)
+    {
+        skip_frames--;
+        return 0;
+    }
+   
     /* if previous DMA isn't finished yet, skip frame */
     if(mlv_rec_dma_active)
     {
@@ -2229,7 +2334,6 @@
         {
             edmac_timeouts = 0;
             raw_recording_state = RAW_FINISHING;
-            raw_rec_cbr_stopping();
         }
         return 0;
     }
@@ -2248,7 +2352,6 @@
     if(!raw_lv_settings_still_valid())
     {
         raw_recording_state = RAW_FINISHING;
-        raw_rec_cbr_stopping();
         return 0;
     }
     
@@ -2337,7 +2440,7 @@

static int32_t mlv_write_hdr(FILE* f, mlv_hdr_t *hdr)
{
-    raw_rec_cbr_mlv_block(hdr);
+    mlv_rec_call_cbr(MLV_REC_EVENT_BLOCK, hdr);

     uint32_t written = FIO_WriteFile(f, hdr, hdr->blockSize);

@@ -2903,7 +3006,6 @@
         {
abort:
             raw_recording_state = RAW_FINISHING;
-            raw_rec_cbr_stopping();
             NotifyBox(5000, "Recording stopped:\n '%s'", error_message);
             /* this is error beep, not audio sync beep */
             beep_times(2);
@@ -2975,7 +3077,8 @@
             }
             else
             {
-                raw_rec_cbr_mlv_block(block);
+                /* when this block will get prepended, call the CBR */
+                mlv_rec_call_cbr(MLV_REC_EVENT_BLOCK, block);

                 /* prepend the given block if possible or requeue it in case of error */
                 int32_t ret = mlv_prepend_block(slot, block);
@@ -3198,6 +3301,8 @@
     /* init stuff */
     raw_recording_state = RAW_PREPARING;

+    mlv_rec_call_cbr(MLV_REC_EVENT_PREPARING, NULL);
+
     if(DISPLAY_REC_INFO_DEBUG)
     {
         bmp_printf(FONT_MED, 30, 50, "Prepare recording...");
@@ -3226,10 +3331,7 @@
     
     /* disable Canon's powersaving (30 min in LiveView) */
     powersave_prohibit();
-
-    /* signal that we are starting, call this before any memory allocation to give CBR the chance to allocate memory */
-    raw_rec_cbr_starting();
-
+   
     /* allocate memory */
     if(!setup_buffers())
     {
@@ -3239,6 +3341,9 @@
         goto cleanup;
     }

+    /* signal that we are starting */
+    mlv_rec_call_cbr(MLV_REC_EVENT_STARTING, NULL);
+
     msleep(start_delay * 1000);

     hack_liveview(0);
@@ -3339,9 +3444,6 @@
         /* this will enable the vsync CBR and the other task(s) */
         raw_recording_state = RAW_RECORDING;

-        /* some modules may do some specific stuff right when we started recording */
-        raw_rec_cbr_started();
-
         while((raw_recording_state == RAW_RECORDING) || (used_slots > 0))
         {
             /* on shutdown or writers that aborted, abort even if there are unwritten slots */
@@ -3374,7 +3476,6 @@
                 NotifyBox(5000, "Frame skipped. Stopping");
                 trace_write(raw_rec_trace_ctx, "<-- stopped recording, frame was skipped");
                 raw_recording_state = RAW_FINISHING;
-                raw_rec_cbr_stopping();
             }

             /* how fast are we writing? does this speed match our benchmarks? */
@@ -3492,7 +3593,6 @@
                             /* try to free up some space and exit */
                             mlv_rec_release_dummies();
                             raw_recording_state = RAW_FINISHING;
-                            raw_rec_cbr_stopping();
                         }
                         raw_prepare_chunk(handle->file_handle, &handle->file_header);
                     }
@@ -3545,6 +3645,8 @@
             }
         }
         
+        mlv_rec_call_cbr(MLV_REC_EVENT_STOPPING, NULL);
+       
         /* now close all queued files */
         while(1)
         {
@@ -3609,7 +3711,6 @@

         /* done, this will stop the vsync CBR and the copying task */
         raw_recording_state = RAW_FINISHING;
-        raw_rec_cbr_stopping();

         /* queue two aborts to cancel tasks */
         msg_queue_receive(mlv_job_alloc_queue, &write_job, 0);
@@ -3646,7 +3747,7 @@

cleanup:
     /* signal that we are stopping */
-    raw_rec_cbr_stopped();
+    mlv_rec_call_cbr(MLV_REC_EVENT_STOPPED, NULL);

     /*
     if(DISPLAY_REC_INFO_DEBUG)
@@ -3686,7 +3787,6 @@
     {
         abort_test = 1;
         raw_recording_state = RAW_FINISHING;
-        raw_rec_cbr_stopping();
     }
     else
     {
diff --git a/modules/mlv_rec/mlv_rec.h b/modules/mlv_rec/mlv_rec.h
--- a/modules/mlv_rec/mlv_rec.h
+++ b/modules/mlv_rec/mlv_rec.h
@@ -64,7 +64,6 @@
#define MLV_METADATA_CYCLIC   4
#define MLV_METADATA_ALL      0xFF

-
/* one video frame */
struct frame_slot
{
@@ -144,25 +143,8 @@
  *      This function is called on every single raw frame that is received from sensor with a pointer to frame data as parameter.
  *      If the return value is zero, the frame will get save into the saving buffers, else it is skipped
  *      Default: Do not skip frame (0)
- *
- *    uint32_t raw_rec_save_buffer(uint32_t used, uint32_t buffer_count)
- *      This function is called whenever the writing loop is checking if it has data to save to card.
- *      The parameters are the number of used buffers and the total buffer count
- *      Default: Save buffer (1)
- *
- *    uint32_t raw_rec_skip_buffer(uint32_t buffer_index, uint32_t buffer_count);
- *      Whenever the buffers are full, this function is called with the buffer index that is subject to being dropped, the number of frames in this buffer and the total buffer count.
- *      If it returns zero, this buffer will not get thrown away, but the next frame will get dropped.
- *      Default: Do not throw away buffer, but throw away incoming frame (0)
  */
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_starting();
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_started();
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_stopping();
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_stopped();
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_mlv_block(mlv_hdr_t *hdr);
extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_skip_frame(unsigned char *frame_data);
-extern WEAK_FUNC(ret_1) uint32_t raw_rec_cbr_save_buffer(uint32_t used, uint32_t buffer_index, uint32_t frame_count, uint32_t buffer_count);
-extern WEAK_FUNC(ret_0) uint32_t raw_rec_cbr_skip_buffer(uint32_t buffer_index, uint32_t frame_count, uint32_t buffer_count);


/* helpers for reserving disc space */
@@ -204,7 +186,6 @@
static void hack_liveview_vsync();
static void unhack_liveview_vsync(int32_t unused);
static void hack_liveview(int32_t unhack);
-void mlv_rec_queue_block(mlv_hdr_t *hdr);
void mlv_rec_set_rel_timestamp(mlv_hdr_t *hdr, uint64_t timestamp);
int32_t mlv_rec_get_free_slot();
void mlv_rec_get_slot_info(int32_t slot, uint32_t *size, void **address);
diff --git a/modules/mlv_rec/mlv_rec_interface.h b/modules/mlv_rec/mlv_rec_interface.h
new file mode 100644
--- /dev/null
+++ b/modules/mlv_rec/mlv_rec_interface.h
@@ -0,0 +1,62 @@
+/*
+ * Copyright (C) 2013 Magic Lantern Team
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the
+ * Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor,
+ * Boston, MA  02110-1301, USA.
+ */
+
+#ifndef __MLV_REC_INTERFACE_H__
+#define __MLV_REC_INTERFACE_H__
+
+
+/* interface to register callbacks for various stuff */
+typedef void (*event_cbr_t) (uint32_t event, void *ctx, mlv_hdr_t *hdr);
+
+/* event types, can be combined */
+#define MLV_REC_EVENT_STARTING    (1U<<0) /* gets called when recording is going to be started */
+#define MLV_REC_EVENT_STARTED     (1U<<1) /* gets called when recording was started and blocks are being written */
+#define MLV_REC_EVENT_STOPPING    (1U<<2) /* gets called when going to stop recoding for whatever reason. good point to finally push some data before files are closed */
+#define MLV_REC_EVENT_STOPPED     (1U<<3) /* gets called when recording has stopped */
+#define MLV_REC_EVENT_CYCLIC      (1U<<4) /* gets called whenever the cyclic RTCI block is written (usually 2 second interval) */
+#define MLV_REC_EVENT_BLOCK       (1U<<5) /* gets called for every block before it is being written to the file. 'hdr' parameter will contain a pointer to the the block. might get called multiple times per block! */
+#define MLV_REC_EVENT_VIDF        (1U<<6) /* gets called for every VIDF being queued for write. called from EDMAC CBR, so avoid using too much CPU time. */
+#define MLV_REC_EVENT_PREPARING   (1U<<7) /* gets called before any buffer allocation or LV manipulation */
+
+
+#if !defined(__MLV_REC_C__) && !defined(__MLV_LITE_C__)
+
+/* register function as callback routine for all events ORed into 'event' and call that CBR with given 'ctx' */
+extern WEAK_FUNC(ret_0) uint32_t mlv_rec_register_cbr(uint32_t event, event_cbr_t cbr, void *ctx);
+
+/* unregister a previously registered CBR for any event */
+extern WEAK_FUNC(ret_0) uint32_t mlv_rec_unregister_cbr(event_cbr_t cbr);
+
+/* queue a MLV block for writing. timestamp will get set automatically as it requires knowledge of the absolute record starting time */
+extern WEAK_FUNC(ret_0) uint32_t mlv_rec_queue_block(mlv_hdr_t *hdr);
+
+#else
+   
+/* structure entry for registered CBR routines */
+typedef struct
+{
+    uint32_t event;
+    void *ctx;
+    event_cbr_t cbr;
+} cbr_entry_t;
+
+#endif
+
+#endif
diff --git a/modules/mlv_snd/mlv_snd.c b/modules/mlv_snd/mlv_snd.c
--- a/modules/mlv_snd/mlv_snd.c
+++ b/modules/mlv_snd/mlv_snd.c
@@ -30,20 +30,25 @@
#include <beep.h>
#include <propvalues.h>
#include <raw.h>
-#include "ml-cbr.h"
+#include <ml-cbr.h>

#include "../trace/trace.h"
#include "../mlv_rec/mlv.h"
+#include "../mlv_rec/mlv_rec_interface.h"

-#define MLV_SND_BUFFERS 4
+/* allocate that many frame slots to be used for WAVI blocks. two is the minimum to maintain operation */
+#define MLV_SND_SLOTS              2
+/* maximum number of WAVI blocks per slot. the larger, the longer queues will get. should we use more? */
+#define MLV_SND_BLOCKS_PER_SLOT  256

static uint32_t trace_ctx = TRACE_ERROR;

-static CONFIG_INT("mlv.snd.enabled", mlv_snd_enabled, 0);
+static CONFIG_INT("mlv.snd.enabled", mlv_snd_enabled, 1);
static CONFIG_INT("mlv.snd.mlv_snd_enable_tracing", mlv_snd_enable_tracing, 0);
static CONFIG_INT("mlv.snd.bit.depth", mlv_snd_in_bits_per_sample, 16);
static CONFIG_INT("mlv.snd.sample.rate", mlv_snd_in_sample_rate, 48000);
static CONFIG_INT("mlv.snd.sample.rate.selection", mlv_snd_rate_sel, 0);
+static CONFIG_INT("mlv.snd.vsync_delay", mlv_snd_vsync_delay, 1);

extern int StartASIFDMAADC(void *, uint32_t, void *, uint32_t, void (*)(), uint32_t);
extern int SetNextASIFADCBuffer(void *, uint32_t);
@@ -60,15 +65,22 @@
extern int32_t mlv_rec_get_free_slot();
extern void mlv_rec_release_slot(int32_t slot, uint32_t write);
extern void mlv_rec_set_rel_timestamp(mlv_hdr_t *hdr, uint64_t timestamp);
-extern void mlv_rec_queue_block(mlv_hdr_t *hdr);
+extern void mlv_rec_skip_frames(uint32_t count);

-static volatile int32_t mlv_snd_rec_active = 0;
static struct msg_queue * volatile mlv_snd_buffers_empty = NULL;
static struct msg_queue * volatile mlv_snd_buffers_done = NULL;
static volatile uint32_t mlv_snd_in_buffers = 64;
static volatile uint32_t mlv_snd_frame_number = 0;
static volatile uint32_t mlv_snd_in_buffer_size = 0;

+/* for tracking chunks */
+static volatile uint16_t mlv_snd_file_num = UINT16_MAX;
+/* frames issued to mlv_lite for writing */
+static volatile uint32_t mlv_snd_frames_queued = 0;
+/* frames written to previous chunks */
+static volatile uint32_t mlv_snd_frames_saved = 0;
+
+
static uint32_t mlv_snd_rates[] = { 48000, 44100, 22050, 11025, 8000 };
#define MLV_SND_RATE_TEXT "48kHz", "44.1kHz", "22kHz", "11kHz", "8kHz"

@@ -91,13 +103,12 @@
audio_data_t *mlv_snd_next_buffer = NULL;

#define MLV_SND_STATE_IDLE                   0  /* waiting for action, set by writer task upon exit */
-#define MLV_SND_STATE_PREPARE                1  /* recording was started, set by mlv_snd_start() */
-#define MLV_SND_STATE_READY                  2  /* buffers etc are set up, set by mlv_snd_alloc_buffers() */
-#define MLV_SND_STATE_SOUND_RUNNING          3  /* ASIF sound recording was started, set by mlv_snd_vsync() */
-#define MLV_SND_STATE_SOUND_STOPPING         4  /* stop audio recording, set by mlv_snd_stop() */
-#define MLV_SND_STATE_SOUND_STOP_ASIF        5  /* waiting for ASIF to process its last buffer, set by mlv_snd_asif_in_cbr() */
-#define MLV_SND_STATE_SOUND_STOP_TASK        6  /* waiting for thread to stop, set by mlv_snd_asif_in_cbr() */
-#define MLV_SND_STATE_SOUND_STOPPED          7  /* all threads and stuff is stopped, finish cleanup, set by task */
+#define MLV_SND_STATE_READY                  1  /* buffers etc are set up, set by mlv_snd_cbr_starting() */
+#define MLV_SND_STATE_SOUND_RUNNING          2  /* ASIF sound recording was started, set by mlv_snd_vsync() */
+#define MLV_SND_STATE_SOUND_STOPPING         3  /* stop audio recording, set by mlv_snd_stop() */
+#define MLV_SND_STATE_SOUND_STOP_ASIF        4  /* waiting for ASIF to process its last buffer, set by mlv_snd_asif_in_cbr() */
+#define MLV_SND_STATE_SOUND_STOP_TASK        5  /* waiting for thread to stop, set by mlv_snd_asif_in_cbr() */
+#define MLV_SND_STATE_SOUND_STOPPED          6  /* all threads and stuff is stopped, finish cleanup, set by task */

static uint32_t mlv_snd_state = MLV_SND_STATE_IDLE;

@@ -192,29 +203,32 @@
         audio_data_t *entry = NULL;
         if(msg_queue_receive(queue, &entry, 10))
         {
-            trace_write(trace_ctx, "mlv_snd_flush_entries: msg_queue_receive(queue, ) failed");
+            trace_write(trace_ctx, "mlv_snd_flush_entries: msg_queue_receive() failed");
             return;
         }
     
-        trace_write(trace_ctx, "mlv_snd_flush_entries: entry is MLV slot");
         mlv_audf_hdr_t *hdr = (mlv_audf_hdr_t *)entry->mlv_slot_buffer;
         
         if(clear)
         {
-            trace_write(trace_ctx, "mlv_snd_flush_entries: NULL slot %d entry", entry->mlv_slot_id);
+            trace_write(trace_ctx, "mlv_snd_flush_entries:   NULL slot %d entry", entry->mlv_slot_id);
             mlv_set_type((mlv_hdr_t *)hdr, "NULL");
+            hdr->timestamp = 0;
         }
         else
         {
-            trace_write(trace_ctx, "mlv_snd_flush_entries: data %d entry for frame #%d", entry->mlv_slot_id, entry->frameNumber);
+            trace_write(trace_ctx, "mlv_snd_flush_entries:   data %d entry for frame #%d", entry->mlv_slot_id, entry->frameNumber);
             mlv_set_type((mlv_hdr_t *)hdr, "AUDF");
             hdr->frameNumber = entry->frameNumber;
             mlv_rec_set_rel_timestamp((mlv_hdr_t*)hdr, entry->timestamp);
+           
+            /* set the highest frame number for updating header later */
+            mlv_snd_frames_queued = MAX(mlv_snd_frames_queued, entry->frameNumber + 1);
         }
         
         if(entry->mlv_slot_end)
         {
-            trace_write(trace_ctx, "mlv_snd_flush_entries: entry is MLV slot %d (last buffer, so release)", entry->mlv_slot_id);
+            trace_write(trace_ctx, "mlv_snd_flush_entries:   last entry of slot %d -> release for writing", entry->mlv_slot_id);
             mlv_rec_release_slot(entry->mlv_slot_id, 1);
         }
         free(entry);
@@ -254,8 +268,6 @@
     mlv_snd_flush_entries(mlv_snd_buffers_done, 0);
     trace_write(trace_ctx, "mlv_snd_stop: flush mlv_snd_buffers_empty");
     mlv_snd_flush_entries(mlv_snd_buffers_empty, 1);
-
-    mlv_snd_state = MLV_SND_STATE_IDLE;
}

static void mlv_snd_queue_slot()
@@ -280,7 +292,7 @@
     }
     
     /* make sure that there is still place for a NULL block */
-    while((used + block_size + sizeof(mlv_hdr_t) < size) && (queued < 128))
+    while((used + block_size + sizeof(mlv_hdr_t) < size) && (queued < MLV_SND_BLOCKS_PER_SLOT))
     {
         /* setup AUDF header for that block */
         mlv_audf_hdr_t *hdr = (mlv_audf_hdr_t *)((uint32_t)address + used);
@@ -290,9 +302,9 @@
         
         mlv_set_type((mlv_hdr_t *)hdr, "NULL");
         hdr->blockSize = block_size;
-        hdr->frameNumber = 0xFFFFFFFF;
+        hdr->frameNumber = 0;
         hdr->frameSpace = hdr_size - sizeof(mlv_audf_hdr_t);
-        hdr->timestamp = 0xFFFFFFFFFFFFFFFF;
+        hdr->timestamp = 0;
         
         /* store information about the buffer in the according queue entry */
         audio_data_t *entry = malloc(sizeof(audio_data_t));
@@ -308,7 +320,7 @@
         entry->mlv_slot_end = 0;
         
         /* check if this was the last frame and set end flag if so */
-        if((used + block_size + sizeof(mlv_hdr_t) >= size) || (queued >= 128))
+        if((used + block_size + sizeof(mlv_hdr_t) >= size) || (queued >= (MLV_SND_BLOCKS_PER_SLOT - 1)))
         {
             /* this tells the writer task that the buffer is filled with that entry being done and can be committed */
             entry->mlv_slot_end = 1;
@@ -323,13 +335,12 @@
     
     mlv_set_type((mlv_hdr_t *)hdr, "NULL");
     hdr->blockSize = size - used;
-    hdr->timestamp = 0xFFFFFFFFFFFFFFFF;
+    hdr->timestamp = 0;
}


static void mlv_snd_prepare_audio()
{
-    mlv_snd_frame_number = 0;
     mlv_snd_in_sample_rate = mlv_snd_rates[mlv_snd_rate_sel];

     /* some models may need this */
@@ -350,11 +361,10 @@
     mlv_snd_in_buffer_size = (mlv_snd_in_sample_rate * (mlv_snd_in_bits_per_sample / 8) * mlv_snd_in_channels) / fps;
     trace_write(trace_ctx, "mlv_snd_alloc_buffers: mlv_snd_in_buffer_size = %d", mlv_snd_in_buffer_size);
     
-    mlv_snd_queue_slot();
-    mlv_snd_queue_slot();
-
-    /* now everything is ready to fire - real output activation happens as soon mlv_snd_running is set to 1 and mlv_snd_vsync() gets called */
-    mlv_snd_state = MLV_SND_STATE_READY;
+    for(int slot = 0; slot < MLV_SND_SLOTS; slot++)
+    {
+        mlv_snd_queue_slot();
+    }
}

static void mlv_snd_writer(int unused)
@@ -410,6 +420,7 @@
                 {
                     trace_write(trace_ctx, "   --> WRITER: entry is MLV slot %d (last buffer, so release)", buffer->mlv_slot_id);
                     mlv_rec_release_slot(buffer->mlv_slot_id, 1);
+                    mlv_snd_frames_queued = hdr->frameNumber + 1;
                     mlv_snd_queue_slot();
                 }
                 free(buffer);
@@ -430,15 +441,35 @@
     {
         char filename[] = "mlv_snd.txt";
         trace_ctx = trace_start("mlv_snd", filename);
-        trace_format(trace_ctx, TRACE_FMT_TIME_REL | TRACE_FMT_COMMENT, ' ');
+        trace_format(trace_ctx, TRACE_FMT_TIME_REL | TRACE_FMT_COMMENT | TRACE_FMT_TASK_ID | TRACE_FMT_TASK_NAME, ' ');
     }

     trace_write(trace_ctx, "mlv_snd_start: starting");
     
     mlv_snd_prepare_audio();
     task_create("mlv_snd", 0x16, 0x1000, mlv_snd_writer, NULL);
+}
+
+void mlv_fill_wavi(mlv_wavi_hdr_t *hdr, uint64_t start_timestamp)
+{
+    mlv_set_type((mlv_hdr_t*)hdr, "WAVI");
+    hdr->blockSize = sizeof(mlv_wavi_hdr_t);
+    mlv_set_timestamp((mlv_hdr_t *)hdr, start_timestamp);
+
+    if(!mlv_snd_enabled)
+    {
+        /* not recording sound, don't trick MLV decoders :) */
+        mlv_set_type((mlv_hdr_t*)hdr, "NULL");
+        return;
+    }
     
-    mlv_snd_state = MLV_SND_STATE_PREPARE;
+    /* this part is compatible to RIFF WAVE/fmt header */
+    hdr->format = 1;
+    hdr->channels = mlv_snd_in_channels;
+    hdr->samplingRate = mlv_snd_in_sample_rate;
+    hdr->bytesPerSecond = mlv_snd_in_sample_rate * (mlv_snd_in_bits_per_sample / 8) * mlv_snd_in_channels;
+    hdr->blockAlign = (mlv_snd_in_bits_per_sample / 8) * mlv_snd_in_channels;
+    hdr->bitsPerSample = mlv_snd_in_bits_per_sample;
}

static void mlv_snd_queue_wavi()
@@ -448,72 +479,159 @@
     /* queue an WAVI block that contains information about the audio format */
     mlv_wavi_hdr_t *hdr = malloc(sizeof(mlv_wavi_hdr_t));
     
-    mlv_set_type((mlv_hdr_t*)hdr, "WAVI");
-    hdr->blockSize = sizeof(mlv_wavi_hdr_t);
-    mlv_rec_set_rel_timestamp((mlv_hdr_t*)hdr, get_us_clock());
-   
-    /* this part is compatible to RIFF WAVE/fmt header */
-    hdr->format = 1;
-    hdr->channels = mlv_snd_in_channels;
-    hdr->samplingRate = mlv_snd_in_sample_rate;
-    hdr->bytesPerSecond = mlv_snd_in_sample_rate * (mlv_snd_in_bits_per_sample / 8) * mlv_snd_in_channels;
-    hdr->blockAlign = (mlv_snd_in_bits_per_sample / 8) * mlv_snd_in_channels;
-    hdr->bitsPerSample = mlv_snd_in_bits_per_sample;
+    mlv_fill_wavi(hdr, get_us_clock());
     
     mlv_rec_queue_block((mlv_hdr_t *)hdr);
}

-/* public functions for raw_rec */
-uint32_t raw_rec_cbr_starting()
+static void mlv_snd_cbr_starting(uint32_t event, void *ctx, mlv_hdr_t *hdr)
{
     if(!mlv_snd_enabled)
     {
-        return 0;
+        return;
     }
     
+    if(mlv_snd_state != MLV_SND_STATE_IDLE)
+    {
+        return;
+    }
+   
+    /* recording is about to start, everything was set up there, now it is our turn */
+    trace_write(trace_ctx, "mlv_snd_cbr_starting: starting mlv_snd");
+    mlv_snd_start();
+    mlv_snd_queue_wavi();
+    mlv_snd_alloc_buffers();
+   
+    /* reset all variables first */
+    mlv_snd_file_num = UINT16_MAX;
+    mlv_snd_frame_number = 0;
+    mlv_snd_frames_queued = 0;
+    mlv_snd_frames_saved = 0;
+   
+
+    mlv_snd_state = MLV_SND_STATE_READY;
+}
+
+/* will get called from mlv_lite's vsync hook */
+static void mlv_snd_cbr_started(uint32_t event, void *ctx, mlv_hdr_t *hdr)
+{
+    /* the first time we get called from vsync, start recording */
+    if(mlv_snd_state != MLV_SND_STATE_READY)
+    {
+        return;
+    }
+   
+    /* "delaying audio" in the video timeline means to skip video frames */
+    mlv_rec_skip_frames(mlv_snd_vsync_delay);
+   
+    /* fetch buffers to start recording */
+    uint32_t msgs = 0;
+    msg_queue_count(mlv_snd_buffers_empty, &msgs);
+   
+    if(msgs < 2)
+    {
+        trace_write(trace_ctx, "mlv_snd_cbr_started: fatal error, no buffers");
+        bmp_printf(FONT(FONT_MED, COLOR_RED, COLOR_BLACK), 10, 130, "fatal: no buffers");
+        beep();
+        return;
+    }
+   
+    trace_write(trace_ctx, "mlv_snd_cbr_started: starting audio");
+   
+    /* get two buffers and queue them to ASIF */
+    mlv_snd_current_buffer = NULL;
+    mlv_snd_next_buffer = NULL;
+   
+    msg_queue_receive(mlv_snd_buffers_empty, &mlv_snd_current_buffer, 10);
+    msg_queue_receive(mlv_snd_buffers_empty, &mlv_snd_next_buffer, 10);
+   
+    if(!mlv_snd_current_buffer || !mlv_snd_next_buffer)
+    {
+        trace_write(trace_ctx, "mlv_snd_cbr_started: fatal error, no buffers");
+        bmp_printf(FONT(FONT_MED, COLOR_RED, COLOR_BLACK), 10, 130, "fatal: no buffers");
+        beep();
+        return;
+    }
+
+    audio_configure(1);
+    StartASIFDMAADC(mlv_snd_current_buffer->data, mlv_snd_current_buffer->length, mlv_snd_next_buffer->data, mlv_snd_next_buffer->length, mlv_snd_asif_in_cbr, 0);
+   
+    /* the current one will get filled right now */
+    mlv_snd_current_buffer->timestamp = get_us_clock();
+    trace_write(trace_ctx, "mlv_snd_cbr_started: starting audio DONE");
+   
+    mlv_snd_state = MLV_SND_STATE_SOUND_RUNNING;
+}
+
+static void mlv_snd_cbr_stopping(uint32_t event, void *ctx, mlv_hdr_t *hdr)
+{
+    if(mlv_snd_state != MLV_SND_STATE_SOUND_RUNNING)
+    {
+        return;
+    }
+   
+    trace_write(trace_ctx, "mlv_snd_cbr_stopping: stopping");
+    mlv_snd_stop();
+
+    mlv_snd_state = MLV_SND_STATE_IDLE;
+}
+
+static void mlv_snd_cbr_stopped(uint32_t event, void *ctx, mlv_hdr_t *hdr)
+{
     if(mlv_snd_state == MLV_SND_STATE_IDLE)
     {
-        trace_write(trace_ctx, "raw_rec_cbr_starting: starting mlv_snd");
-        mlv_snd_rec_active = 1;
-        mlv_snd_start();
+        return;
     }
     
-    return 0;
+    trace_write(trace_ctx, "mlv_snd_cbr_stopped: seems recording aborted during setup");
+    mlv_snd_stop();
+    mlv_snd_state = MLV_SND_STATE_IDLE;
}

-uint32_t raw_rec_cbr_started()
+
+static void mlv_snd_cbr_mlv_block(uint32_t event, void *ctx, mlv_hdr_t *hdr)
{
-    if(mlv_snd_state == MLV_SND_STATE_PREPARE)
-    {
-        trace_write(trace_ctx, "raw_rec_cbr_started: allocating buffers");
-        mlv_snd_alloc_buffers();
-        mlv_snd_queue_wavi();
-    }
-    return 0;
-}
-
-uint32_t raw_rec_cbr_stopping()
-{
-    if(mlv_snd_state != MLV_SND_STATE_IDLE)
-    {
-        trace_write(trace_ctx, "raw_rec_cbr_stopping: stopping");
-        mlv_snd_stop();
-        mlv_snd_rec_active = 0;
-    }
-    return 0;
-}
-
-uint32_t raw_rec_cbr_mlv_block(mlv_hdr_t *hdr)
-{
-    if(!memcmp(hdr->blockType, "MLVI", 4))
+    if(hdr && !memcmp(hdr->blockType, "MLVI", 4))
     {
         mlv_file_hdr_t *file_hdr = (mlv_file_hdr_t *)hdr;
         
-        /* this block is filled on recording start and when the block gets updates on recording end */
-        file_hdr->audioClass = 1; /* 0=none, 1=WAV */
-        file_hdr->audioFrameCount = mlv_snd_frame_number;
+        uint16_t file_num = file_hdr->fileNum;
+       
+        trace_write(trace_ctx, "mlv_snd_cbr_mlv_block: called for file %d", file_num);
+       
+        /* the MLV block is filled on recording start and when the block gets updates on recording end */
+        if(mlv_snd_state == MLV_SND_STATE_READY)
+        {
+            trace_write(trace_ctx, "mlv_snd_cbr_mlv_block: first chunk");
+            /* first chunk's header is being written, save the number of frames in previous chunks */
+            file_hdr->audioClass = 1; /* 0=none, 1=WAV */
+            file_hdr->audioFrameCount = 0;
+        }
+        else if(file_num == mlv_snd_file_num)
+        {
+            /* block gets updated, capture AUDF count */
+            uint32_t queued = mlv_snd_frames_queued;
+           
+            file_hdr->audioFrameCount = queued - mlv_snd_frames_saved;
+           
+            trace_write(trace_ctx, "mlv_snd_cbr_mlv_block: update: q:%d, s:%d", queued, mlv_snd_frames_saved);
+           
+            mlv_snd_frames_saved = queued;
+        }
+        else if(file_num > mlv_snd_file_num)
+        {
+            trace_write(trace_ctx, "mlv_snd_cbr_mlv_block: next");
+            /* next chunk's header is being written */
+            file_hdr->audioClass = 1; /* 0=none, 1=WAV */
+            file_hdr->audioFrameCount = 0;
+        }
+        else
+        {
+            /* suddenly file_num goes back? unexpected, do not tamper with */
+        }
+       
+        mlv_snd_file_num = file_num;
     }
-    return 0;
}

static void mlv_snd_trace_buf(char *caption, uint8_t *buffer, uint32_t length)
@@ -531,77 +649,42 @@
}


-static unsigned int mlv_snd_vsync(unsigned int unused)
-{
-    if(!mlv_snd_enabled)
-    {
-        return 0;
-    }
-   
-    if(mlv_snd_state != MLV_SND_STATE_READY)
-    {
-        return 0;
-    }
-   
-    /* in running mode, start audio recording here */
-    uint32_t msgs = 0;
-    msg_queue_count(mlv_snd_buffers_empty, &msgs);
-   
-    if(msgs >= 2)
-    {
-        trace_write(trace_ctx, "mlv_snd_vsync: starting audio");
-       
-        /* get two buffers and queue them to ASIF */
-        mlv_snd_current_buffer = NULL;
-        mlv_snd_next_buffer = NULL;
-       
-        msg_queue_receive(mlv_snd_buffers_empty, &mlv_snd_current_buffer, 10);
-        msg_queue_receive(mlv_snd_buffers_empty, &mlv_snd_next_buffer, 10);
-       
-        if(mlv_snd_current_buffer && mlv_snd_next_buffer)
-        {
-            mlv_snd_state = MLV_SND_STATE_SOUND_RUNNING;
-       
-            audio_configure(1);
-            StartASIFDMAADC(mlv_snd_current_buffer->data, mlv_snd_current_buffer->length, mlv_snd_next_buffer->data, mlv_snd_next_buffer->length, mlv_snd_asif_in_cbr, 0);
-           
-            /* the current one will get filled right now */
-            mlv_snd_current_buffer->timestamp = get_us_clock();
-            trace_write(trace_ctx, "mlv_snd_vsync: starting audio DONE");
-        }
-        else
-        {
-            trace_write(trace_ctx, "mlv_snd_vsync: msg_queue_receive(mlv_snd_buffers_empty, ...) failed, retry next time");
-        }
-    }
-   
-    return 0;
-}
-
static struct menu_entry mlv_snd_menu[] =
{
     {
-        .name = "MLV Sound",
-        .priv = &mlv_snd_enabled,
-        .max = 1,
-        .help = "Enable sound recording for MLV.",
-        .submenu_width = 710,
-        .children = (struct menu_entry[])
+        .name       = "Sound recording",
+        .select     = menu_open_submenu,
+        .priv       = &mlv_snd_enabled,
+        .help       = "Sound recording options provided by mlv_snd.",
+        .children   = (struct menu_entry[])
         {
             {
-                .name = "Sampling rate",
-                .priv = &mlv_snd_rate_sel,
-                .min = 0,
-                .max = COUNT(mlv_snd_rates)-1,
-                .choices = CHOICES(MLV_SND_RATE_TEXT),
-                .help = "Select your sampling rate.",
+                .name       = "Enable sound",
+                .priv       = &mlv_snd_enabled,
+                .max        = 1,
+                .help       = "[mlv_snd] Enable sound recording for MLV.",
             },
             {
-                .name = "Trace output",
-                .priv = &mlv_snd_enable_tracing,
+                .name       = "Sampling rate",
+                .priv       = &mlv_snd_rate_sel,
+                .min        = 0,
+                .max        = COUNT(mlv_snd_rates)-1,
+                .choices    = CHOICES(MLV_SND_RATE_TEXT),
+                .help       = "[mlv_snd] Select your sampling rate.",
+            },
+            {
+                .name = "Audio delay",
+                .priv = &mlv_snd_vsync_delay,
                 .min = 0,
-                .max = 1,
-                .help = "Enable log file tracing. Needs camera restart.",
+                .max = 32,
+                .help = "Delay the audio that many frames. (experimental)",
+            },
+            {
+                .name       = "Trace output",
+                .priv       = &mlv_snd_enable_tracing,
+                .min        = 0,
+                .max        = 1,
+                .help       = "[mlv_snd] Enable log file tracing. Needs camera restart.",
             },
             MENU_EOL,
         },
@@ -619,15 +702,32 @@
     //}
     
     trace_write(trace_ctx, "mlv_snd_init: init queues");
-    mlv_snd_buffers_empty = (struct msg_queue *) msg_queue_create("mlv_snd_buffers_empty", 300);
-    mlv_snd_buffers_done = (struct msg_queue *) msg_queue_create("mlv_snd_buffers_done", 300);
+    mlv_snd_buffers_empty = (struct msg_queue *) msg_queue_create("mlv_snd_buffers_empty", MLV_SND_BLOCKS_PER_SLOT * MLV_SND_SLOTS);
+    mlv_snd_buffers_done = (struct msg_queue *) msg_queue_create("mlv_snd_buffers_done", MLV_SND_BLOCKS_PER_SLOT * MLV_SND_SLOTS);
+
+    /* will the same menu work in both submenus? probably not */
+    if (menu_get_value_from_script("Movie", "RAW video") != INT_MIN)
+    {
+        menu_add("RAW video", mlv_snd_menu, COUNT(mlv_snd_menu));
+    }
+    else if (menu_get_value_from_script("Movie", "RAW video (MLV)") != INT_MIN)
+    {
+        menu_add("RAW video (MLV)", mlv_snd_menu, COUNT(mlv_snd_menu));
+    }
+
+    trace_write(trace_ctx, "mlv_snd_init: done");
     
-    menu_add("Audio", mlv_snd_menu, COUNT(mlv_snd_menu));
-    trace_write(trace_ctx, "mlv_snd_init: done");
+    /* register callbacks */
+    mlv_rec_register_cbr(MLV_REC_EVENT_STARTING, &mlv_snd_cbr_starting, NULL);
+    mlv_rec_register_cbr(MLV_REC_EVENT_STARTED, &mlv_snd_cbr_started, NULL);
+    mlv_rec_register_cbr(MLV_REC_EVENT_STOPPING, &mlv_snd_cbr_stopping, NULL);
+    mlv_rec_register_cbr(MLV_REC_EVENT_STOPPED, &mlv_snd_cbr_stopped, NULL);
+    mlv_rec_register_cbr(MLV_REC_EVENT_BLOCK, &mlv_snd_cbr_mlv_block, NULL);
     
     return 0;
}

+
static unsigned int mlv_snd_deinit()
{
     if(trace_ctx != TRACE_ERROR)
@@ -644,7 +744,6 @@
MODULE_INFO_END()

MODULE_CBRS_START()
-    MODULE_CBR(CBR_VSYNC, mlv_snd_vsync, 0)
     MODULE_NAMED_CBR("snd_rec_enabled", mlv_snd_snd_rec_cbr)
MODULE_CBRS_END()

@@ -654,4 +753,5 @@
     MODULE_CONFIG(mlv_snd_in_bits_per_sample)
     MODULE_CONFIG(mlv_snd_rate_sel)
     MODULE_CONFIG(mlv_snd_in_sample_rate)
+    MODULE_CONFIG(mlv_snd_vsync_delay)
MODULE_CONFIGS_END()
diff --git a/modules/trace/trace.c b/modules/trace/trace.c
--- a/modules/trace/trace.c
+++ b/modules/trace/trace.c
@@ -366,6 +366,27 @@
     {
         trace_write_timestamp(ctx, TRACE_FMT_TIME_DATE, tsc, linebuffer, &linebuffer_pos);
     }
+    if(ctx->format & TRACE_FMT_TASK_ID)
+    {
+        char tmp[32];
+       
+        linebuffer[linebuffer_pos++] = ' ';
+        snprintf(tmp, sizeof(tmp), "%d", (current_task->taskId & 0xFF));
+        memcpy(&linebuffer[linebuffer_pos], tmp, strlen(tmp));
+        linebuffer_pos += strlen(tmp);
+        linebuffer[linebuffer_pos++] = ' ';
+        linebuffer[linebuffer_pos] = '\000';
+    }
+    if(ctx->format & TRACE_FMT_TASK_NAME)
+    {
+        char *name = get_current_task_name();
+       
+        linebuffer[linebuffer_pos++] = ' ';
+        memcpy(&linebuffer[linebuffer_pos], name, strlen(name));
+        linebuffer_pos += strlen(name);
+        linebuffer[linebuffer_pos++] = ' ';
+        linebuffer[linebuffer_pos] = '\000';
+    }
     if(ctx->format & TRACE_FMT_COMMENT)
     {
         linebuffer[linebuffer_pos++] = ' ';
diff --git a/modules/trace/trace.h b/modules/trace/trace.h
--- a/modules/trace/trace.h
+++ b/modules/trace/trace.h
@@ -19,6 +19,8 @@
#define TRACE_FMT_TIME_REL        0x0010 /* write the time since start as hh:mm:ss.msec*/
#define TRACE_FMT_TIME_DELTA      0x0020 /* write the relative time as hh:mm:ss.msec since last entry*/
#define TRACE_FMT_TIME_DATE       0x0040 /* write the time of day */
+#define TRACE_FMT_TASK_ID         0x0080 /* write the task id */
+#define TRACE_FMT_TASK_NAME       0x0100 /* write the task name */
#define TRACE_FMT_COMMENT         0x1000 /* headers are C like comments */

#define TRACE_FMT_META            0x0100 /* on start and stop write some metadata (e.g. day, time, ...) */


Hope this is what you was looking for

Tryi with MLVFS or incamera with mlv_play.

You can check metadata with mlv_dump if you compile from this branch or from manual_lens_info

I compiled your branch dfort and when I set the script to run at startup it shows the script but after it disappears.
( before I can select something )

This should be caused from memory malloc as you can see from comments inside mem.c. Maybe a review could solve this issue.
This modification to memory management was needed because an issues which was crashing the camera was observed, It was managed by a1ex as can be found into comments of the merged pull request and solved temporary with a tmp patch which wasn't committed, but revised later with a new commit after merging PR

It doesn't happens on my camera as I can wait as long as I want into lens selection menu.
I used the script last month for a shooting session and if I remember correctly I could see some warning in console regarding timeout during the shoot but the script was running fine.

Can you select a lens from the menu if you quickly press the set button? Also it's strange that your choice is preserved with your script after a camera shutdown, autoload functionality was added to manage this

Dfort, while I was trying to compile from crop_rec_4k, I noticed that lua module wasn't going fine due to a missing reference to "Value2Raw", this isn't also into a1ex's patch

I saw your latest commit and it's missing some diff; let's add the patch below.

Code Select Expand


diff --git a/modules/lua/lua_lens.c b/modules/lua/lua_lens.c
--- a/modules/lua/lua_lens.c
+++ b/modules/lua/lua_lens.c
@@ -17,12 +17,24 @@
static int luaCB_lens_index(lua_State * L)
{
     LUA_PARAM_STRING_OPTIONAL(key, 2, "");
-    /// Get the name of the lens (reported by the lens).
-    // @tfield string name readonly
+    /// Get/Set the name of the lens (reported by the lens)
+    // @tfield string name
     if(!strcmp(key, "name")) lua_pushstring(L, lens_info.name);
-    /// Get the focal length of the lens (in mm). Only updated in LiveView.
-    // @tfield int focal_length readonly
+    /// Get the lens id
+    // @tfield int id readonly
+    else if(!strcmp(key, "id")) lua_pushinteger(L, lens_info.lens_id);
+    /// Get the serial number of the lens
+    // @tfield int serial
+    else if(!strcmp(key, "serial")) lua_pushinteger(L, lens_info.lens_serial);
+    /// Get/Set the focal length of the lens (in mm)
+    // @tfield int focal_length
     else if(!strcmp(key, "focal_length")) lua_pushinteger(L, lens_info.focal_len);
+    /// Get/Set the minimum focal length of the lens (in mm)
+    // @tfield int focal_min
+    else if(!strcmp(key, "focal_min")) lua_pushinteger(L, lens_info.lens_focal_min);
+    /// Get/Set the maximum focal length of the lens (in mm)
+    // @tfield int focal_max
+    else if(!strcmp(key, "focal_max")) lua_pushinteger(L, lens_info.lens_focal_max);
     /// Get the current focus distance (in mm). Only updated in LiveView.
     // @tfield int focus_distance readonly
     else if(!strcmp(key, "focus_distance")) lua_pushinteger(L, lens_info.focus_dist * 10);
@@ -84,7 +96,55 @@
static int luaCB_lens_newindex(lua_State * L)
{
     LUA_PARAM_STRING_OPTIONAL(key, 2, "");
-    if(!strcmp(key, "name") || !strcmp(key, "focal_length") || !strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af") || !strcmp(key, "is_chipped"))
+    if(!strcmp(key, "name"))
+    {
+        LUA_PARAM_STRING(value, 3);
+        strncpy(lens_info.name, value, sizeof(lens_info.name)-1);
+    }
+    else if(!strcmp(key, "exists"))
+    {
+        LUA_PARAM_BOOL(value, 3);
+        lens_info.lens_exists = value;
+    }
+    else if(!strcmp(key, "focal_length"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.focal_len = value;
+    }
+    else if(!strcmp(key, "focal_min"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_focal_min = value;
+    }
+    else if(!strcmp(key, "focal_max"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_focal_max = value;
+    }
+    else if(!strcmp(key, "manual_aperture"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        lens_info.aperture = (int)(value * 10);
+        lens_info.raw_aperture = VALUE2RAW(aperture, lens_info.aperture);
+    }
+    else if(!strcmp(key, "aperture_min"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        int tmp = (int)(value * 10);
+        lens_info.raw_aperture_min = VALUE2RAW(aperture, tmp);
+    }
+    else if(!strcmp(key, "aperture_max"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        int tmp = (int)(value * 10);
+        lens_info.raw_aperture_max = VALUE2RAW(aperture, tmp);
+    }
+    else if(!strcmp(key, "serial"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_serial = value;
+    }
+    else if(!strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af") || !strcmp(key, "is_chipped"))
     {
         return luaL_error(L, "'%s' is readonly!", key);
     }
diff --git a/modules/lua/lua_property.c b/modules/lua/lua_property.c
--- a/modules/lua/lua_property.c
+++ b/modules/lua/lua_property.c
@@ -21,8 +21,6 @@
// !!!DANGER WILL ROBINSON!!!
//#define LUA_PROP_REQUEST_CHANGE

-struct msg_queue * lua_prop_queue = NULL;
-
struct lua_prop
{
     struct lua_prop * next;
@@ -32,100 +30,69 @@
     int prop_handler_ref;
};

-struct lua_prop_msg
-{
-    unsigned property;
-    void * value;
-    unsigned len;
-};
-
-int lua_prop_task_running = 0;
static struct lua_prop * prop_handlers = NULL;

-//TODO: create a new task per script so that scripts don't block each other's prop handlers, necessary?
-static void lua_prop_task(int unused)
+static void lua_prophandler(unsigned property, void * priv, void * buf, unsigned len)
{
-    lua_prop_queue = msg_queue_create("lua_prop_queue", 1);
-    TASK_LOOP
+    struct lua_prop * lua_prop = 0;
+    int found = 0;
+    for (lua_prop = prop_handlers; lua_prop; lua_prop = lua_prop->next)
     {
-        struct lua_prop_msg * msg = NULL;
-        int err = msg_queue_receive(lua_prop_queue, &msg, 0);
-       
-        if(err || !msg) continue;
-       
-        struct lua_prop * lua_prop = NULL;
-        int found = 0;
-        for(lua_prop = prop_handlers; lua_prop; lua_prop = lua_prop->next)
+        if (lua_prop->prop_id == property)
         {
-            if(lua_prop->prop_id == msg->property)
+            found = 1;
+            break;
+        }
+    }
+   
+    if (found)
+    {
+        lua_State * L = lua_prop->L;
+        struct semaphore * sem = NULL;
+        if (lua_take_semaphore(L, 100, &sem) == 0)
+        {
+            ASSERT(sem);
+            if (lua_rawgeti(L, LUA_REGISTRYINDEX, lua_prop->prop_handler_ref) == LUA_TFUNCTION)
             {
-                found = 1;
-                break;
+                lua_rawgeti(L, LUA_REGISTRYINDEX, lua_prop->self_ref);
+                if (len > 4)
+                {
+                    /* fixme: pass proper data types (requires knowledge about each property) */
+                    //long, probably a string
+                    //((char*)(msg->value))[msg->len] = 0x0;
+                    lua_pushstring(L, (char*)buf);
+                }
+                else
+                {
+                    lua_pushinteger(L, *(uint32_t*)buf & (0xFFFFFFFF >> ((4-len) * 8)) );
+                }
+
+                int t0 = get_us_clock();
+
+                if (docall(L, 2, 0))
+                {
+                    fprintf(stderr, "[Lua] prop handler failed:\n %s\n", lua_tostring(L, -1));
+                    lua_save_last_error(L);
+                }
+
+                int t1 = get_us_clock();
+
+                if (t1 - t0 > 10000)
+                {
+                    printf("[%s] slow property handler %X (%d us)\n", lua_get_script_filename(L), lua_prop->prop_id, t1 - t0);
+                }
             }
+            give_semaphore(sem);
         }
-       
-        if(found && msg->value)
+        else
         {
-            lua_State * L = lua_prop->L;
-            struct semaphore * sem = NULL;
-            if (lua_take_semaphore(L, 1000, &sem) == 0)
-            {
-                ASSERT(sem);
-                if(lua_rawgeti(L, LUA_REGISTRYINDEX, lua_prop->prop_handler_ref) == LUA_TFUNCTION)
-                {
-                    lua_rawgeti(L, LUA_REGISTRYINDEX, lua_prop->self_ref);
-                    if(msg->len > 4)
-                    {
-                        //long, probably a string
-                        ((char*)(msg->value))[msg->len] = 0x0;
-                        lua_pushstring(L, (char*)(msg->value));
-                    }
-                    else if(msg->len == 4) lua_pushinteger(L, *((uint32_t*)(msg->value)));
-                    else if(msg->len >= 2) lua_pushinteger(L, *((uint16_t*)(msg->value)));
-                    else lua_pushinteger(L, *((uint8_t*)(msg->value)));
-                    if(docall(L, 2, 0))
-                    {
-                        fprintf(stderr, "[Lua] prop handler failed:\n %s\n", lua_tostring(L, -1));
-                        lua_save_last_error(L);
-                    }
-                }
-                give_semaphore(sem);
-            }
-            else
-            {
-                printf("[Lua] semaphore timeout: prop handler %d (%dms)\n", lua_prop->prop_id, 1000);
-            }
+            printf("[%s] semaphore timeout: prop handler %X (%dms)\n", lua_get_script_filename(L), lua_prop->prop_id, 100);
         }
-        free(msg->value);
-        free(msg);
-    }
-}
-
-static void lua_prophandler(unsigned property, void * priv, void * addr, unsigned len)
-{
-    if (!lua_prop_queue) return;
-    struct lua_prop_msg * msg = malloc(sizeof(struct lua_prop_msg));
-    msg->property = property;
-    msg->len = MIN(len,255);
-    msg->value = malloc(msg->len + 1);
-    if(msg->value)
-    {
-        memcpy(msg->value, addr, MIN(len,255));
-        msg_queue_post(lua_prop_queue, (uint32_t)msg);
-    }
-    else
-    {
-        fprintf(stderr, "[Lua] lua_prophandler: malloc error");
     }
}

static void lua_register_prop_handler(unsigned prop_id)
{
-    if(!lua_prop_task_running)
-    {
-        lua_prop_task_running = 1;
-        task_create("lua_prop_task", 0x1c, 0x10000, lua_prop_task, 0);
-    }
     //check for existing prop handler
     struct lua_prop * current;
     for(current = prop_handlers; current; current = current->next)
diff --git a/src/lens.c b/src/lens.c
--- a/src/lens.c
+++ b/src/lens.c
@@ -1941,7 +1941,11 @@
         factor = 1/2.54;
     }
     
-    if(lens_info.lens_focal_min == lens_info.lens_focal_max)
+    if(!lens_info.lens_focal_min || !lens_info.lens_focal_max)
+    {
+        MENU_SET_VALUE("%d %s", (int)(lens_info.focal_len * factor), unit);
+    }
+    else if(lens_info.lens_focal_min == lens_info.lens_focal_max)
     {
         MENU_SET_VALUE("%d %s", (int)(lens_info.lens_focal_min * factor), unit);
     }
@@ -1955,6 +1959,7 @@
    {
         .name = "Lens Info Prefs",
         .select   = menu_open_submenu,
+ .submenu_width = 700,
         .children =  (struct menu_entry[]) {
             #ifndef CONFIG_FULLFRAME
             {
diff --git a/src/lens.h b/src/lens.h
--- a/src/lens.h
+++ b/src/lens.h
@@ -307,7 +307,12 @@
//~ static const int codes_aperture[] =  {0,13,14,16,19,21,24,27,29,32,35,37,40,44,45,48,51,52,53,56,59,60, 61, 64, 68, 69, 72, 75, 76, 77, 80, 83, 84, 85, 88, 91, 92, 93, 96};

#define RAW2VALUE(param,rawvalue) ((int)values_##param[raw2index_##param(rawvalue)])
-#define VALUE2RAW(param,value) ((int)val2raw_##param(value))
+#define VALUE2RAW(param,value) val2raw_##param(value)
+
+/* prototypes for VALUE2RAW */
+int VALUE2RAW(iso, int);
+int VALUE2RAW(shutter, int);
+int VALUE2RAW(aperture, int);

// UNIT_1_8_EV
#define APEX_TV(raw) ((int)(raw) - 56)
diff --git a/src/mem.c b/src/mem.c
--- a/src/mem.c
+++ b/src/mem.c
@@ -582,7 +582,8 @@

     memcheck_add(ptr, file, line);
     
-    /* keep track of allocated memory and update history */
+    /* keep track of allocated memory and update history */
+    unsigned int state = cli();
     allocators[allocator_index].num_blocks++;
     allocators[allocator_index].mem_used += len + 2 * MEM_SEC_ZONE;
     alloc_total += len;
@@ -590,7 +591,8 @@
     alloc_total_peak_with_memcheck = MAX(alloc_total_peak_with_memcheck, alloc_total_with_memcheck);
     history[history_index] = MIN(alloc_total_with_memcheck / 1024, USHRT_MAX);
     history_index = MOD(history_index + 1, HISTORY_ENTRIES);
-   
+    sei(state);
+
     return (void*)(ptr + MEM_SEC_ZONE);
}

@@ -784,12 +786,64 @@
     /* if we arrive here, you should probably solder some memory chips on the mainboard */
     return -1;
}
+
+static void * check_and_adjust_ptr(void * ptr, size_t size, unsigned int flags, const char* file, unsigned int line, int allocator_index)
+{
+    if (!ptr)
+    {
+        /* didn't work? */
+        snprintf(last_error_msg_short, sizeof(last_error_msg_short), "%s(%s,%x)", allocators[allocator_index].name, format_memory_size_and_flags(size, flags));
+        snprintf(last_error_msg, sizeof(last_error_msg), "%s(%s) failed at %s:%d, %s.", allocators[allocator_index].name, format_memory_size_and_flags(size, flags), file, line, get_current_task_name());
+        dbg_printf("alloc fail, took %s%d.%03d s\n", FMT_FIXEDPOINT3(t1-t0));
+    }
+    else
+    {
+        /* force the cacheable pointer to be the way user requested it */
+        /* note: internally, this library must use the vanilla pointer (non-mangled) */
+        ptr = (flags & MEM_DMA) ? UNCACHEABLE(ptr) : CACHEABLE(ptr);
+
+        dbg_printf("alloc ok, took %s%d.%03d s\n", FMT_FIXEDPOINT3(t1-t0));
+    }
+
+    return ptr;
+}
+
+/* used for property handler / vsync hooks / GUI handlers / other tricky places */
+/* not to be called directly, but exported to modules for test purposes (seltest.mo) */
+void * __fast_malloc(size_t size, unsigned int flags, const char* file, unsigned int line)
+{
+    /* for small sizes only */
+    ASSERT(size < 65536);
+
+    /* Canon code uses AllocateMemory for these, so... let's try */
+    int allocator_index = 1;
+
+    /* is this thread-safe? */
+    void * ptr = memcheck_malloc(size, file, line, allocator_index, flags);
+
+    return check_and_adjust_ptr(ptr, size, flags, file, line, allocator_index);
+}

/* these two will replace all malloc calls */

/* returns 0 if it couldn't allocate */
void* __mem_malloc(size_t size, unsigned int flags, const char* file, unsigned int line)
-{
+{
+    /* running from PROP_HANDLER's will cause trouble
+     * slow allocators (in particular, srm_malloc) will keep the semaphore busy
+     * this would delay Canon's property handlers, possibly overflowing the queue
+     * similar issues may appear with GUI handlers, LiveView "vsync" hooks etc
+     * todo: generic way to figure out whether we are running from such hooks or from regular tasks
+     */
+    const char * current_task_name = get_current_task_name();
+    if (streq(current_task_name, "PropMgr") ||
+        streq(current_task_name, "GuiMainTask") ||
+        streq(current_task_name, "Evf"))
+    {
+        dbg_printf("fast_malloc(%s) from %s:%d task %s\n", format_memory_size_and_flags(size, flags), file, line, get_current_task_name());
+        return __fast_malloc(size, flags, file, line);
+    }
+
     ASSERT(mem_sem);
     take_semaphore(mem_sem, 0);



At this point we should have a changeset which extends lens name as a1ex's patch and a possible lua implementation, but I don't expect to run lens.lua script fine because It misses a lot of old commits from manual_lens_info, which I don't know exactly how to manage as my work is based on top of old version of lens.lua

Anyway it compiles for 5D3 but I can't test it. Can you try it?

Thanks. Added that but it isn't working yet. Need to dig in a little deeper when I've got some time.

Checked again the branch and noticed some missing diff in lua which are responsible to update lens_info structure
Copy these files from manual_lens_info:
- modules/lua/lua_lens.c
- modules/lua/lua_menu.c
- modules/lua/lua_property.c

Refer to this diff and this to see which files has been modified and also remember of a1ex's commits regarding memory backend (crop_rec_4k and lua_fix appears to have the same commits for mem.c and self_test.c, so you can see diff directly from manual_lens_info pull request)

My testing is limited because 50D won't compile in crop_rec_4k and Qemu simulates a chipped lens attached

It should be ok, but it needs "script/lib" folder from manual_lens in order to run lens.lua

Lars, do you need also changes from crop_rec_4k_mlv_snd or building on top of crop_rec_4k is ok for now?

At the moment the experimental manual_lens branch was tested also by a1ex and dfort.
There could be some issues when recording raw video, probably inherited from lua_fix and I would like to know if are present also on top of crop_rec_4k.

Also I would like to know how it behave with different adapter from the cheap one I have, so further testing and feedback from other users is really appreciated

I was thinking to do this on a temporary branch after pushing some little improvements, but I have no time to do this at the moment.

Some time ago a1ex posted a patch for solving merge conflicts and asked to test, but it wasn't recognised by patch tool the I tried.

Hopefully solved. Wasn't easy (tried the same approach when merging the first PR, but could not pass the tests from selftest.mo, so I ended up committing a workaround - that turned out not to be very good); however, today I've found the issue: yet another quirk in Canon's large memory allocators (the reason why it was crashing after ~1000 blocks or so). Seems to work in my quick tests, but I'd still like to sleep on it before including in lua_fix and crop_rec_4k.

I've extracted a patch for crop_rec_4k with only the changes related to ELNS and 64-byte lens name (mlv_dump, mlv*, silent and lens.h), but I'm afraid I have no more energy left to run any tests today (so I'm not committing it yet). If you try it, please report back.

Code Select Expand


# HG changeset patch
# User alex@thinkpad
# Date 1518995384 -3600
#      Mon Feb 19 00:09:44 2018 +0100
# Branch crop_rec_4k
# Node ID 804086b3e47771bd72ed123bdd797c608738bf4f
# Parent  5138c0050854c3c44eb2de21cf63c445baa0b245
* Experimental: 64Byte Extended Lens Name
* mlv_dump.c: Fix duplicate ELNS Block processing
* mlv_rec.c: Fix duplicate ELNS Block processing

diff -r 5138c0050854 -r 804086b3e477 modules/lua/lua_lens.c
--- a/modules/lua/lua_lens.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/lua/lua_lens.c Mon Feb 19 00:09:44 2018 +0100
@@ -72,7 +72,55 @@
static int luaCB_lens_newindex(lua_State * L)
{
     LUA_PARAM_STRING_OPTIONAL(key, 2, "");
-    if(!strcmp(key, "name") || !strcmp(key, "focal_length") || !strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af"))
+    if(!strcmp(key, "name"))
+    {
+        LUA_PARAM_STRING(value, 3);
+        strncpy(lens_info.name, value, sizeof(lens_info.name)-1);
+    }
+    else if(!strcmp(key, "exists"))
+    {
+        LUA_PARAM_BOOL(value, 3);
+        lens_info.lens_exists = value;
+    }
+    else if(!strcmp(key, "focal_length"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.focal_len = value;
+    }
+    else if(!strcmp(key, "focal_min"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_focal_min = value;
+    }
+    else if(!strcmp(key, "focal_max"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_focal_max = value;
+    }
+    else if(!strcmp(key, "manual_aperture"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        lens_info.aperture = (int)(value * 10);
+        lens_info.raw_aperture = VALUE2RAW(aperture, lens_info.aperture);
+    }
+    else if(!strcmp(key, "aperture_min"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        int tmp = (int)(value * 10);
+        lens_info.raw_aperture_min = VALUE2RAW(aperture, tmp);
+    }
+    else if(!strcmp(key, "aperture_max"))
+    {
+        LUA_PARAM_NUMBER(value, 3);
+        int tmp = (int)(value * 10);
+        lens_info.raw_aperture_max = VALUE2RAW(aperture, tmp);
+    }
+    else if(!strcmp(key, "serial"))
+    {
+        LUA_PARAM_INT(value, 3);
+        lens_info.lens_serial = value;
+    }
+    else if(!strcmp(key, "focus_distance") || !strcmp(key, "hyperfocal") || !strcmp(key, "dof_near") || !strcmp(key, "dof_far") || !strcmp(key, "af") || !strcmp(key, "is_chipped"))
     {
         return luaL_error(L, "'%s' is readonly!", key);
     }
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_lite/mlv_lite.c
--- a/modules/mlv_lite/mlv_lite.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_lite/mlv_lite.c Mon Feb 19 00:09:44 2018 +0100
@@ -350,7 +350,8 @@
static GUARDED_BY(RawRecTask)   mlv_rawc_hdr_t rawc_hdr;
static GUARDED_BY(RawRecTask)   mlv_idnt_hdr_t idnt_hdr;
static GUARDED_BY(RawRecTask)   mlv_expo_hdr_t expo_hdr;
-static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
+static GUARDED_BY(RawRecTask)   mlv_lens_hdr_t lens_hdr;
+static GUARDED_BY(RawRecTask)   mlv_elns_hdr_t elns_hdr;
static GUARDED_BY(RawRecTask)   mlv_rtci_hdr_t rtci_hdr;
static GUARDED_BY(RawRecTask)   mlv_wbal_hdr_t wbal_hdr;
static GUARDED_BY(LiveViewTask) mlv_vidf_hdr_t vidf_hdr;
@@ -2807,7 +2808,8 @@

     mlv_fill_idnt(&idnt_hdr, mlv_start_timestamp);
     mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
-    mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+    mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+    mlv_fill_elns(&elns_hdr, mlv_start_timestamp);
     mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
     mlv_fill_wbal(&wbal_hdr, mlv_start_timestamp);

@@ -2827,6 +2829,7 @@
     if (FIO_WriteFile(f, &idnt_hdr, idnt_hdr.blockSize) != (int)idnt_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &expo_hdr, expo_hdr.blockSize) != (int)expo_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &lens_hdr, lens_hdr.blockSize) != (int)lens_hdr.blockSize) return 0;
+    if (FIO_WriteFile(f, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &rtci_hdr, rtci_hdr.blockSize) != (int)rtci_hdr.blockSize) return 0;
     if (FIO_WriteFile(f, &wbal_hdr, wbal_hdr.blockSize) != (int)wbal_hdr.blockSize) return 0;
     if (mlv_write_vers_blocks(f, mlv_start_timestamp)) return 0;
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv.c
--- a/modules/mlv_rec/mlv.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv.c Mon Feb 19 00:09:44 2018 +0100
@@ -55,13 +55,35 @@
     hdr->autofocusMode = af_mode;
     hdr->flags = 0;

+    char name[33];
     char buf[33];
+    snprintf(name, 32, "%s", lens_info.name);
     snprintf(buf, sizeof(buf), "%X%08X", (uint32_t) (lens_info.lens_serial >> 32), (uint32_t)(lens_info.lens_serial & 0xFFFFFFFF));
-   
-    strncpy((char *)hdr->lensName, lens_info.name, 32);
+
+    strncpy((char *)hdr->lensName, name, 32);
     strncpy((char *)hdr->lensSerial, buf, 32);
}

+void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp){
+    /* prepare header */
+    mlv_set_type((mlv_hdr_t *)hdr, "ELNS");
+    mlv_set_timestamp((mlv_hdr_t *)hdr, start_timestamp);
+    hdr->blockSize = sizeof(mlv_elns_hdr_t);
+
+    char bufName[65];
+    snprintf(bufName, 64, "%s", lens_info.name);
+
+    strncpy((char *)hdr->lensName, bufName, 64);
+    hdr->focalLengthMin = lens_info.lens_focal_min;
+    hdr->focalLengthMax = lens_info.lens_focal_max;
+    hdr->apertureMin = RAW2VALUE(aperture, lens_info.raw_aperture_min) / 10.0;
+    hdr->apertureMax = RAW2VALUE(aperture, lens_info.raw_aperture_max) / 10.0;
+    hdr->version = lens_info.lens_version;
+    hdr->extenderInfo = lens_info.lens_extender;
+    hdr->capabilities = lens_info.lens_capabilities;
+    hdr->chipped = lens_info.lens_exists;
+}
+
void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp)
{
     /* prepare header */
@@ -151,7 +173,7 @@
     mlv_set_type((mlv_hdr_t *)hdr, "IDNT");
     mlv_set_timestamp((mlv_hdr_t *)hdr, start_timestamp);
     hdr->blockSize = sizeof(mlv_idnt_hdr_t);
-   
+
     hdr->cameraModel = camera_model_id;
     memcpy(hdr->cameraName, camera_model, 32);
     memcpy(hdr->cameraSerial, camera_serial, 32);
@@ -163,16 +185,16 @@
{
     int block_length = (strlen(version_string) + sizeof(mlv_vers_hdr_t) + 1 + 3) & ~3;
     mlv_vers_hdr_t *header = malloc(block_length);
-   
+
     /* prepare header */
     mlv_set_type((mlv_hdr_t *)header, "VERS");
     mlv_set_timestamp((mlv_hdr_t *)header, start_timestamp);
     header->blockSize = block_length;
     header->length = strlen(version_string);
-   
+
     char *vers_hdr_payload = (char *)&header[1];
     strcpy(vers_hdr_payload, version_string);
-   
+
     *hdr = header;
}

@@ -239,12 +261,12 @@
{
     int mod = -1;
     int error = 0;
-   
+
     do
     {
         /* get next loaded module id */
         mod = module_get_next_loaded(mod);
-       
+
         /* make sure thats a valid one */
         if(mod >= 0)
         {
@@ -252,7 +274,7 @@
             const char *mod_name = module_get_name(mod);
             const char *mod_build_date = module_get_string(mod, "Build date");
             const char *mod_last_update = module_get_string(mod, "Last update");
-           
+
             if(mod_name != NULL)
             {
                 /* just in case that ever happens */
@@ -264,15 +286,15 @@
                 {
                     mod_last_update = "(no version)";
                 }
-               
+
                 /* separating the format string allows us to measure its length for malloc */
                 const char *fmt_string = "%s built %s; commit %s";
                 int buf_length = strlen(fmt_string) + strlen(mod_name) + strlen(mod_build_date) + strlen(mod_last_update) + 1;
                 char *version_string = malloc(buf_length);
-               
+
                 /* now build the string */
                 snprintf(version_string, buf_length, fmt_string, mod_name, mod_build_date, mod_last_update);
-               
+
                 /* and finally remove any newlines, they are annoying */
                 for(unsigned int pos = 0; pos < strlen(version_string); pos++)
                 {
@@ -281,23 +303,23 @@
                         version_string[pos] = ' ';
                     }
                 }
-               
+
                 /* let the mlv helpers build the block for us */
                 mlv_vers_hdr_t *hdr = NULL;
                 mlv_build_vers(&hdr, mlv_start_timestamp, version_string);
-               
+
                 /* try to write to output file */
                 if(FIO_WriteFile(f, hdr, hdr->blockSize) != (int)hdr->blockSize)
                 {
                     error = 1;
                 }
-               
+
                 /* free both temporary string and allocated mlv block */
                 free(version_string);
                 free(hdr);
             }
         }
     } while(mod >= 0 && !error);
-   
+
     return error;
}
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv.h
--- a/modules/mlv_rec/mlv.h Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv.h Mon Feb 19 00:09:44 2018 +0100
@@ -165,6 +165,22 @@
}  mlv_lens_hdr_t;

typedef struct {
+    uint8_t     blockType[4];    /* ELNS - Extended LENS block with longer lens name and optional fields, depending on camera */
+    uint32_t    blockSize;
+    uint64_t    timestamp;
+    uint16_t    focalLengthMin;  /* shortest focal length in mm                       */
+    uint16_t    focalLengthMax;  /* longest focal length in mm                        */
+    uint16_t    apertureMin;     /* lowest f-number * 100                             */
+    uint16_t    apertureMax;     /* highest f-number * 100                            */
+    uint32_t    version;         /* lens internal version number, if available        */
+    uint8_t     extenderInfo;    /* extender information, if provided by camera       */
+    uint8_t     capabilities;    /* capability information, if provided by camera     */
+    uint8_t     chipped;         /* when not zero, lens is communicating with camera  */
+    uint8_t     lensName[64];     /* full lens string, null terminated                 */
+    // TODO: Review Specs
+}  mlv_elns_hdr_t;
+
+typedef struct {
     uint8_t     blockType[4];       /* RTCI: real-time clock metadata */
     uint32_t    blockSize;          /* total frame size */
     uint64_t    timestamp;          /* hardware counter timestamp for this frame (relative to recording start) */
@@ -287,6 +303,7 @@
void mlv_fill_rtci(mlv_rtci_hdr_t *hdr, uint64_t start_timestamp);
void mlv_fill_expo(mlv_expo_hdr_t *hdr, uint64_t start_timestamp);
void mlv_fill_lens(mlv_lens_hdr_t *hdr, uint64_t start_timestamp);
+void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp);
void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv_dump.c
--- a/modules/mlv_rec/mlv_dump.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv_dump.c Mon Feb 19 00:09:44 2018 +0100
@@ -1894,6 +1894,7 @@
     lv_rec_file_footer_t lv_rec_footer;
     mlv_file_hdr_t main_header;
     mlv_lens_hdr_t lens_info;
+    mlv_elns_hdr_t elns_info;
     mlv_expo_hdr_t expo_info;
     mlv_idnt_hdr_t idnt_info;
     mlv_wbal_hdr_t wbal_info;
@@ -1904,6 +1905,7 @@
     /* initialize stuff */
     memset(&lv_rec_footer, 0x00, sizeof(lv_rec_file_footer_t));
     memset(&lens_info, 0x00, sizeof(mlv_lens_hdr_t));
+    memset(&elns_info, 0x00, sizeof(mlv_elns_hdr_t));
     memset(&expo_info, 0x00, sizeof(mlv_expo_hdr_t));
     memset(&idnt_info, 0x00, sizeof(mlv_idnt_hdr_t));
     memset(&wbal_info, 0x00, sizeof(mlv_wbal_hdr_t));
@@ -3652,6 +3654,46 @@
                     print_msg(MSG_INFO, "     Flags:       0x%08X\n", lens_info.flags);
                 }
             }
+            else if(!memcmp(buf.blockType, "ELNS", 4))
+            {
+                uint32_t hdr_size = MIN(sizeof(mlv_elns_hdr_t), buf.blockSize);
+
+                if(fread(&elns_info, hdr_size, 1, in_file) != 1)
+                {
+                    print_msg(MSG_ERROR, "File ends in the middle of a block\n");
+                    goto abort;
+                  }
+
+                /* skip remaining data, if there is any */
+                file_set_pos(in_file, position + elns_info.blockSize, SEEK_SET);
+
+                lua_handle_hdr(lua_state, buf.blockType, &elns_info, sizeof(elns_info));
+
+                if(verbose)
+                {
+                    //TODO: Check Aperture Min/Max and if other fields are needed
+                    print_msg(MSG_INFO, "     Name:                '%s'\n", elns_info.lensName);
+                    print_msg(MSG_INFO, "     Focal Length Min:    %d mm\n", elns_info.focalLengthMin);
+                    print_msg(MSG_INFO, "     Focal Length Max:    %d mm\n", elns_info.focalLengthMax);
+                    print_msg(MSG_INFO, "     Aperture Min:        f/%.2f\n", (double)elns_info.apertureMin);
+                    print_msg(MSG_INFO, "     Aperture Max:        f/%.2f\n", (double)elns_info.apertureMax);
+                    print_msg(MSG_INFO, "     Version:             %d\n", elns_info.version);
+                    print_msg(MSG_INFO, "     Extender Info:       0x%02X\n", elns_info.extenderInfo);
+                    print_msg(MSG_INFO, "     Capabilities:        0x%02X\n", elns_info.capabilities);
+                    print_msg(MSG_INFO, "     Chipped:             0x%02X\n", elns_info.chipped);
+                }
+
+                if(mlv_output && !no_metadata_mode && (!extract_block || !strncasecmp(extract_block, (char*)elns_info.blockType, 4)))
+                {
+                    /* correct header size if needed */
+                    elns_info.blockSize = sizeof(mlv_elns_hdr_t);
+                    if(fwrite(&elns_info, elns_info.blockSize, 1, out_file) != 1)
+                    {
+                        print_msg(MSG_ERROR, "Failed writing into .MLV file\n");
+                        goto abort;
+                    }
+                }
+            }
             else if(!memcmp(mlv_block->blockType, "INFO", 4))
             {
                 mlv_info_hdr_t block_hdr = *(mlv_info_hdr_t *)mlv_block;
diff -r 5138c0050854 -r 804086b3e477 modules/mlv_rec/mlv_rec.c
--- a/modules/mlv_rec/mlv_rec.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/mlv_rec/mlv_rec.c Mon Feb 19 00:09:44 2018 +0100
@@ -195,6 +195,7 @@

static mlv_expo_hdr_t last_expo_hdr;
static mlv_lens_hdr_t last_lens_hdr;
+static mlv_elns_hdr_t last_elns_hdr;
static mlv_wbal_hdr_t last_wbal_hdr;
static mlv_styl_hdr_t last_styl_hdr;

@@ -1449,14 +1450,17 @@
             trace_write(raw_rec_trace_ctx, "[polling_cbr] queueing INFO blocks");
             mlv_expo_hdr_t *expo_hdr = malloc(sizeof(mlv_expo_hdr_t));
             mlv_lens_hdr_t *lens_hdr = malloc(sizeof(mlv_lens_hdr_t));
+            mlv_elns_hdr_t *elns_hdr = malloc(sizeof(mlv_elns_hdr_t));
             mlv_wbal_hdr_t *wbal_hdr = malloc(sizeof(mlv_wbal_hdr_t));

             mlv_fill_expo(expo_hdr, mlv_start_timestamp);
             mlv_fill_lens(lens_hdr, mlv_start_timestamp);
+            mlv_fill_elns(elns_hdr, mlv_start_timestamp);
             mlv_fill_wbal(wbal_hdr, mlv_start_timestamp);

             msg_queue_post(mlv_block_queue, (uint32_t) expo_hdr);
             msg_queue_post(mlv_block_queue, (uint32_t) lens_hdr);
+            msg_queue_post(mlv_block_queue, (uint32_t) elns_hdr);
             msg_queue_post(mlv_block_queue, (uint32_t) wbal_hdr);
         }

@@ -2567,6 +2571,7 @@
         mlv_rtci_hdr_t rtci_hdr;
         mlv_expo_hdr_t expo_hdr;
         mlv_lens_hdr_t lens_hdr;
+        mlv_elns_hdr_t elns_hdr;
         mlv_idnt_hdr_t idnt_hdr;
         mlv_wbal_hdr_t wbal_hdr;
         mlv_styl_hdr_t styl_hdr;
@@ -2574,6 +2579,7 @@
         mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
         mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
         mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+        mlv_fill_elns(&elns_hdr, mlv_start_timestamp);
         mlv_fill_idnt(&idnt_hdr, mlv_start_timestamp);
         mlv_fill_wbal(&wbal_hdr, mlv_start_timestamp);
         mlv_fill_styl(&styl_hdr, mlv_start_timestamp);
@@ -2585,10 +2591,12 @@
         idnt_hdr.timestamp = 4;
         wbal_hdr.timestamp = 5;
         styl_hdr.timestamp = 6;
+        elns_hdr.timestamp = 7;

         mlv_write_hdr(f, (mlv_hdr_t *)&rtci_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&expo_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&lens_hdr);
+        mlv_write_hdr(f, (mlv_hdr_t *)&elns_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&idnt_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&wbal_hdr);
         mlv_write_hdr(f, (mlv_hdr_t *)&styl_hdr);
@@ -3100,13 +3108,16 @@

         mlv_expo_hdr_t old_expo = last_expo_hdr;
         mlv_lens_hdr_t old_lens = last_lens_hdr;
+        mlv_elns_hdr_t old_elns = last_elns_hdr;

         mlv_fill_expo(&last_expo_hdr, mlv_start_timestamp);
         mlv_fill_lens(&last_lens_hdr, mlv_start_timestamp);
+        mlv_fill_elns(&last_elns_hdr, mlv_start_timestamp);

         /* update timestamp for comparing content changes */
         old_expo.timestamp = last_expo_hdr.timestamp;
         old_lens.timestamp = last_lens_hdr.timestamp;
+        old_elns.timestamp = last_elns_hdr.timestamp;

         /* write new state if something changed */
         if(memcmp(&last_expo_hdr, &old_expo, sizeof(mlv_expo_hdr_t)))
@@ -3123,6 +3134,13 @@
             memcpy(hdr, &last_lens_hdr, sizeof(mlv_lens_hdr_t));
             msg_queue_post(mlv_block_queue, (uint32_t) hdr);
         }
+        /* write new state if something changed */
+        if(memcmp(&last_elns_hdr, &old_elns, sizeof(mlv_elns_hdr_t)))
+        {
+            mlv_hdr_t *hdr = malloc(sizeof(mlv_elns_hdr_t));
+            memcpy(hdr, &last_elns_hdr, sizeof(mlv_elns_hdr_t));
+            msg_queue_post(mlv_block_queue, (uint32_t) hdr);
+        }
     }

     if(mlv_update_styl && (mlv_metadata & MLV_METADATA_SPORADIC))
diff -r 5138c0050854 -r 804086b3e477 modules/silent/silent.c
--- a/modules/silent/silent.c Sat Feb 17 22:37:51 2018 +0100
+++ b/modules/silent/silent.c Mon Feb 19 00:09:44 2018 +0100
@@ -28,6 +28,7 @@
extern WEAK_FUNC(ret_0) void mlv_fill_rtci(mlv_rtci_hdr_t *hdr, uint64_t start_timestamp);
extern WEAK_FUNC(ret_0) void mlv_fill_expo(mlv_expo_hdr_t *hdr, uint64_t start_timestamp);
extern WEAK_FUNC(ret_0) void mlv_fill_lens(mlv_lens_hdr_t *hdr, uint64_t start_timestamp);
+extern WEAK_FUNC(ret_0) void mlv_fill_elns(mlv_elns_hdr_t *hdr, uint64_t start_timestamp);
extern WEAK_FUNC(ret_0) void mlv_fill_idnt(mlv_idnt_hdr_t *hdr, uint64_t start_timestamp);
extern WEAK_FUNC(ret_0) void mlv_fill_wbal(mlv_wbal_hdr_t *hdr, uint64_t start_timestamp);
extern WEAK_FUNC(ret_0) void mlv_fill_styl(mlv_styl_hdr_t *hdr, uint64_t start_timestamp);
@@ -365,6 +366,7 @@
     mlv_rtci_hdr_t rtci_hdr;
     mlv_expo_hdr_t expo_hdr;
     mlv_lens_hdr_t lens_hdr;
+    mlv_elns_hdr_t elns_hdr;
     mlv_idnt_hdr_t idnt_hdr;
     mlv_wbal_hdr_t wbal_hdr;
     mlv_styl_hdr_t styl_hdr;
@@ -478,6 +480,7 @@
     mlv_fill_rtci(&rtci_hdr, mlv_start_timestamp);
     mlv_fill_expo(&expo_hdr, mlv_start_timestamp);
     mlv_fill_lens(&lens_hdr, mlv_start_timestamp);
+    mlv_fill_elns(&elns_hdr, mlv_start_timestamp);

     expo_hdr.isoValue = metadata.iso;
     expo_hdr.shutterValue = 1000000000 / metadata.tvr;
@@ -486,7 +489,8 @@
     if (FIO_WriteFile(save_file, &rtci_hdr, rtci_hdr.blockSize) != (int)rtci_hdr.blockSize) goto write_error;
     if (FIO_WriteFile(save_file, &expo_hdr, expo_hdr.blockSize) != (int)expo_hdr.blockSize) goto write_error;
     if (FIO_WriteFile(save_file, &lens_hdr, lens_hdr.blockSize) != (int)lens_hdr.blockSize) goto write_error;
-   
+    if (FIO_WriteFile(save_file, &elns_hdr, elns_hdr.blockSize) != (int)elns_hdr.blockSize) goto write_error;
+
     memset(&vidf_hdr, 0, sizeof(mlv_vidf_hdr_t));
     mlv_set_type((mlv_hdr_t *)&vidf_hdr, "VIDF");
     mlv_set_timestamp((mlv_hdr_t *)&vidf_hdr, mlv_start_timestamp);
diff -r 5138c0050854 -r 804086b3e477 src/lens.h
--- a/src/lens.h Sat Feb 17 22:37:51 2018 +0100
+++ b/src/lens.h Mon Feb 19 00:09:44 2018 +0100
@@ -37,7 +37,7 @@
struct lens_info
{
         void *                  token;
-        char                    name[ 32 ];
+        char                    name[ 64 ];
         unsigned                focal_len; // in mm
         unsigned                focus_dist; // in cm
         unsigned                IS; // PROP_LV_LENS_STABILIZE


50D if needed:

Code Select Expand


PB_Palette - the one maybe I need?
bmp_vram_real() - in bmp.c same as above
bmp_vram_info - should not be necessary as isn't asked during compilation if I remember correctly
sd_device - probably minor priority


How to find ram address for those?

I made a quick test defining CONFIG_40D (to solve some minor problems following difference in code for the others cameras running vxworks) and got a compilation without errors but no signs of ML getting loaded in qemu as it doesn't replace firmware version number and it doesn't draw ML Menu.

So I expect to find those pieces to be able to print text and draw ML menu.

Also tried to match gui_task_list from 40D using

Code Select Expand

./run_canon_fw.sh 40D,firmware="boot=0" -d ram and managed to found which subroutine return the value found into stubs.S, done the same with 1000D and also found what to appears the same code in rom's dump but as the function's address is called more than one times and return different values each time, i checked for the same value for 1000D and it shows up in console but not use if is correct to use it...

[lua_fix.2018Mar31.50D]

I tested again this time with lua_fix.2018Mar31.50D and camera crashes also with this branch. So I think there is something wrong with memory management.



How to reproduce:
1. enable a lot of modules and mlv_rec.mo with or without lua.mo enabled
2. start a raw recording
3. camera freezes with black screen (sometimes red led on) after stopping recording
if lua.mo is enabled recording it crash immediately. Also noticed some strange behaviour of the text next to the green camera icon, it is suppose to give you current recording time? or an estimated remaining time? Because it seems to print random numbers...

Using mlv_lite.mo instead  (with or without lua.mo loaded) it can record multiple clip in a row but debug menu will show a overflow error

@a1ex Can you take a look please? Any others have this issue with 50D or other camera?

EDIT: Merging lua_fix into manual_lens appears to solve camera crash. I need to retest this branch without the merge a d dig a little more

Regarding Dual Iso movie for 50D there is an old post of tecgen. This could be a starting point.

Another missing feature is flexinfo.
I made a pull request that enable it on 50D. Current issue is the flicker ofelements inside the center part of the screen. Also occurring with console screen (See below).
I was unable to test into unified due to compiling error I get with vanilla branch

Other bugs:
B1. Magic Zoom flicker a little bit. Maybe it is less evident In nighly than lua_fix derivatives. Need to retest a bit. See issue
2682

B2. Stopping h264 recording is done by pressing SET button instead of using half-shutter press. See issue #2332

B3. Console's windows is flickering in Photo display screen.
This should be a particularity of 50D. This was introduced (previously it wasn't evident because canon's display refresh was blocked) after refactoring console in lua_fix branch

[EDIT:]

i have downloaded magiclantern-manual_lens_info.2018Feb19.1100D105.zip for my 1100d, but camera can't start with this version.

Maybe is related to memory constraint resource of 1100D.

I tried to merge latest changes from lua_fix to see if my previous reported issue could be solved with newer memory backend but camera won't show any signs apart the MPU handlings buttons and lens...

EDIT: I was using a workspace with CONFIG_QEMU defined from a previous work and didn't noticed, so this is why it wasn't running on camera.