Naming conventions for RAW and MLV clips

[kfprod]

It took me a while to understand why MLV Mystic and RAW Magic was crashing for me. It turned out to be when I had been running 2 cameras on the same minute, the names of the files were the same.

I'd like to suggest a naming convention scheme similar to what RED has implemented on their cameras.

"The aim is to both uniquely and unambiguously reference each clip/take which has ever been shot every day by each camera, as well as aiding during the online editing and post-production phases to discern which reel, camera and shooting date combinations that clip "belongs" to."

http://en.wikipedia.org/wiki/REDCODE
(under "Naming convention")

For the time being I've just added seconds to my module, I'm not a strong enough programmer to implement a system close to what RED has.

Code: [Select]

snprintf(filename, sizeof(filename), "%s/M%02d-%02d%02d-%02d.RAW", get_dcim_dir(), now.tm_mday, now.tm_hour, COERCE(now.tm_min + number, 0, 99), COERCE(now.tm_sec + number, 0, 99));
I found some stuff online that I'd like to put in but I can't compile it, again because I'm a noob but let me know what changes I'd need to make to get this to run in raw_rec.c and I could possibly put it together myself.

Code: [Select]

string RandomString(int len)
{
   srand(time(0));
   string str = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
   int pos;
   while(str.size() != len) {
    pos = ((rand() % (str.size() - 1)));
    str.erase (pos, 1);
   }
   return str;
}

string random_str = RandomString(4);

[g3gg0]

you can only name files in the DOS fashion 8.3 style.

[kfprod]

OK I'm able to use longer file names, must be because I'm using exFAT on all my cards. I found that's a nicer way to set it as you don't get split files but you can't format cards in the camera... Still it could be implemented but with a warning if you're on FAT and not exFAT.

[g3gg0]

which model supports longer names?

[kfprod]

I'm on a 5d mk3.

[kfprod]

The 5d3.123 release. I've made a file called "M14-2333-55-LONGFILENAMEHERE.RAW" without problem and it shows up in the raw playback also. Could it be a problem using longer file names even if it works?

If you could help me modify the RandomString function I wrote about earlier so it compiles I could set that up and then write a menu item in the style of

if (exFAT)
                .name = "Unique file names",
                .priv = &unique_name,
                .max = 1,
                .help = "Create unique file names",
                .advanced = 1,

[a1ex]

What about autodetecting the filesystem (at startup maybe), and if it's ExFAT, allow ML to use enhancements like long file names and large files without cluttering the menu?

Autodetection can be done with the low-level routines from bootflags.c (from the code used to make the card bootable).

[dmilligan]

Personally I'd be in favor of sticking to 8.3 for everything to avoid cluttering the code base. 8 chars is plenty to have unique names if you use all the letters (i.e. encode the date in base 37). Just my 2 cents.

[kfprod]

Interesting… Thought about this a bit now and can’t come up with an ideal solution. Basically it would be better for the clutter but worse for the user. Everytime you want to make sense of the name you'd have to have a base 36 parser. Checking the file name on your PC or camera wouldn't make sense on first sight. And they wouldn't end up in order unless you sort by date. Also reconnecting shots and knowing which was which would be hard after you've "unpacked" the name.

If we did abbreviations for the months like

JA
FE
MR
AP
MY
JN
JL
AU
SE
OC
NO
DE

An exFAT name would be
14DE2014-1003-XXX.raw (where XXX is base 36 to make it unique)

That wouldn’t work for the DOS name but the DOS name could be
DE14LIL0

(where LIL0 is base 36 for 1003860, ie 10:03.860 hour minute and 3 milisecond digits) - that should make it unique enough for almost everything while still giving some information about what’s there.

I used http://www.translatorscafe.com/cafe/EN/units-converter/numbers/c/

[swinxx]

or if possible.. give the camera a unique identification number - cam 01,02,03,04,... or readout serialnumber and use the last 2 or 3 numbers?

[kfprod]

I can come up with hundreds of solutions for that for exFAT but with 8 characters it becomes difficult. What would you suggest?

A good solution for exFAT would be to have a user input that defaults to the last 3 digits of the serial number as you say but that you could also set to just 1, 2, 3 or whatever number you want to identify the camera. The file name then has the addition CAM1, CAM2, CAM3 (or CAM555 if you didn't change the field and the serial number ended with 555).

File name then could be 31DE2014-1003-CAM1-XXX.raw (or 31DE2014-1003-CAM555-XXX.raw).

[ansius]

IMHO, we all have our own style for workflow, but what I have learned - to utilize os file sorting, and there for I'm mostly using CAM-XX_YYYY-MM-DD_HH:MM:SS_CLIP-XXXX, and preferably CAM and date fields should be possible to swap, it can come in handy when shooting multi-camera for more than one day, if you are not organized and don't put footage in the right folders. So far when shooting on my 7D in h264 i'm always resting the counter, thus creating a new folder. And in photography too, especially for timelapses, handy.

[g3gg0]

thanks for the constructive feedback and discussion.

ill throw in another option.
why does the filename at all have to contain information?
why not give it just the/a unique id like the uid field in MLV header.

the long filename then can be restored on computer using a simple script/tool.
on PC side you can specify the exact format like this:
 "${cam_name}_${cam_serial}_${take}_${HH}_${MM}_${SS}"
and all fields are available in MLV metadata.

what's in a name? its just decoration

[kfprod]

I agree that would be a nice solution. That does mean another app before running the unpacking app or asking the MLV Mystic programmers to implement it though.

But keeping the file name the way it is now and extracting the advanced file name that ansius suggests in postproduction would work.

[ansius]

Name matters, because when you start to dig trough days of material, it is much faster to find it if named in human readable form. Of course, for that there are media management solutions (ingest, tagging, proxy generation) witch are out of the reach for most mortals, and for us, simple folk -  there are folders and file names.

I've noticed ML communities obsession with scripts, but to be honest - creative professionals would avoid them if possible. "Why I have to run script for what was possible in camera?" The ones benefiting form ML development are not the IT oriented folks, but ones that shoot, ingest, edit, export people. They have already enough on their plates content wise, and they hate all middle steps, to get to the edit. Other problem is that, most of camera operators, editors, colorists hate using console, we learn it just to be able to deal with it, but if there is a GUI or a plugin, we would stick to that.

[kfprod]

DEVs: Is there a global variable that says if the file system is exFAT? Currently mlv_rec.c only checks that its not one of the cameras that doesn't have the option (cam_5d2, cam_50d, cam_7d). It doesn't actually check what file system the card is.

[g3gg0]

quoting:

Autodetection can be done with the low-level routines from bootflags.c (from the code used to make the card bootable).

this implies: no there is nothing like that, but some code that can be re-used

[kfprod]

Thanks g3gg0 I checked now and that's a bit over my current skill. But I've put all the other changes I wanted into mlv_rec.c and the final names are in this style

20140618_232314_c1_t3_djhc.MLV

Added a menu item for "camera" 0-99. c1 and t1 is only added if you have camera and take enabled. The last 4 are just randomized "0123456789abcdefghijklmnopqrstuvwxyz".

It checks is_camera("5D3",  "1.2.3") || is_camera("5D3", "1.1.3") but doesn't check for exFAT.

Would you like to check the changes, should I make a pull request from web browser like this?
http://www.magiclantern.fm/forum/index.php?topic=7940.0

It would be great if you could have a look at this post too: a1ex thought maybe you knew the answer to something at Reply #34.
http://www.magiclantern.fm/forum/index.php?topic=12022
I have a shoot next week where it would be amazing if I could get the trigger talked about there to work along with the new file names... Thanks!