Messages

I also noticed for FRSP there are black borders on the top and left of the image, i heard that MLV_DUMP doesn't do this

@eatbuckshot
Sorry, i was unavailable. I will research this.

np!

just to note, from what i read, i think theoretically changing the qscale which the prores_ks encoder allows, may make the output video technically not compliant with the rates apple set for prores which i read has the possibility to cause issues with anything decoding it, but it seems to work fine for me in after effects

There is an issue where, if you make a preset with a blank name(on accident? or invalid name?) it creates a file with only an extension ".pst1" and then will display a blank entry in the presets menu, but also will cause any new presets even if they're named correctly, to be blank as well

there is another issue with setting default output directory to a path with '[' or ']' in it. 'D:\OFFLOADS\[2015.11.07]Technica' would be truncated to 'D:\OFFLOADS\[2015.11.07' after restarting mlvproducer and checking the settings, I suspect this has something to do with how the settings.dat stores configuration using [] as delimiters

Also I can't seem to get the export and import queue to work, it saves a 2.24mb file regardless of how many i have in the queue and claims to be an unknown format on import

edit: I see  that Prores 444 FAST actually outputs 422 according to media player classic at around 250 MBps

what exactly is the difference between Prores fast and non fast?

edit:
I sought after x264 10-bit output support as it tends to be more efficient than 8-bit output with the same source, is of higher quality, and it handles gradients without needing to dither.
I discovered that 8-bit or 10-bit output depends on the specific build of ffmpeg.exe which can only output one or the other.  Here are builds with ffmpeg that use the 10-bit x264 https://ffmpeg.zeranoe.com/blog/?p=435 (i just replaced the old ffmpeg.exe that's alongside mlvproducer)

I prefer x264 over prores or dnxhd since they have a set compression rate and aren't that efficient at their bitrates. (https://ffmpeg.zeranoe.com/forum/viewtopic.php?f=10&t=2270)
(https://video.stackexchange.com/questions/13164/encoding-422-in-10-bit-with-libx264)  (http://x264.nl/x264/10bit_02-ateme-why_does_10bit_save_bandwidth.pdf)
(http://unix.stackexchange.com/questions/111886/prores-444-output-using-ffmpeg)
Though there seems to be some issues with adobe premiere pro cs6 or after effects in importing any x264 encoded source besides 420
edit:
Since I can't seem to get them to work in ppro or ae, I opted for prores444 but with reduced bitrate, currently it's being encoded with a qscale of 0 for lossless by default in MLVProducer, but that produces 250+ mbps which is a little too much as I'm sharing these "masters" I discovered where I could set the qscale 9 by editing the parameters MLVProducer sends to FFMPEG http://i.imgur.com/x9aBdO4.png which allows around 142 mbps instead

Working on creating a Mac Version (via Wine Bottler and Wine Tricks). This app looks like it is the new best thing for "developing" MLV files into Prores files for editing (since MLrawViewer stopped working for me). Got everything working UNTIL I click on the "..." to set the output path, which results in a "Runtime error '445'" and the app crashing. Very keen to get this working as it outstrips other programs by a mile.

I would be fine to continue working on packaging MLVP using Wine Bottler, but if you're using Free Basic and VB6 then I might be able to compile a native Mac app using the sources (Free Basic is cross platform and Mono supports VB for Mac). Would love to collaborate and make MLV Producer the end all utility!

Alice

I think some of the .dlls might use some variant of BASIC that's not vb6 and the source for those were provided somewhere in the thread, though Mono IIRC is for .net languages, which vb6 came before

It would be neat to be open source though.  I am a fan of it being able to run natively (probably statically linked vb6 library though) or without a framework to install.

Awesome work AWPStar!

Are you looking into render output settings customization? And then perhaps NVENC or quicksync support?

I can't seem to find the options/settings dialog in 1772,  I was curious if I could set the ram usage to be more than 1024 by default, (i think the limit was 1536 in the older versions, i suppose i can check the settings file though...).  I used to write a lot of VB6 back in the day, and I believe it's only 32 bit which may limit the process to 2-3gb max... Perhaps multiple processes to get around this? or utilize 64-bit dll etc idk


I wonder how the multithreading is implemented. Is it alternating frames are assigned to render in a new thread? or is it more granular on groups of pixels at a time (perhaps to make debayering a single frame faster etc)

Edit: Nevermind, I forgot the settings is under the gear icon to the left of File

When attempting to do full res silent pics on 50d using the may or june nightlies using "take pics like crazy" I also get individual mlv files per frame, and it cannot be stopped without taking out of battery.

Has there been any progress on this?

Ahh it's just possible that in the IDX code, you may have checked for existence earlier, for the current opening .mlv/.raw file, then didn't expect the path to differ in idx?? idk

i'm not sure what happens if you load a .mlv/.raw then move it to another folder and reopen it

@eatbuckshot It is definitely OS problem. Command line should work.

@kutu
weird bug.
If you will try to resize main window, will it refresh these tools?

Nevermind, I think I fixed it.

I thought it had something to do with accessing the MLV files via a network drive that was previously mapped to some unrelated network path. Although explorer would correctly browse to R:\50d1 mapped to \\192.168.1.25\g$\50d1 etc, it would display the old \\some\really\old\path\here in the address bar, which was due to remnants in the Registry under a MountPoints2 key under Explorer.  But as suspected, removing this didn't solve the issue since it was only for display only.

The real culprit was, I had accessed the particular MLV file when mapped to drive K:, then I remapped it to drive R:
I received this error any time I tried to access it.
I didn't realize it was only this file, since I happened to have also accessed yet another file with the same error so I didn't try anymore files I suppose...


Anyway, I did a little troubleshooting and discovered that it was referencing the old path in the IDX files using HXD and procmon
http://i.imgur.com/DQ0OvQM.png

It seems that I cannot open a MLV file using Open With... and selecting MLVProducer in this version
(nor can I use command line MLVProducer.exe somemlvfile.mlv)

If i select a predefine, such as 50d, nothing is displayed, (but selecting a colorspace brings it back)

There are many free online IDE's with a linux vm, I frequently use https://c9.io or https://nitrous.io

can someone tell me why my liveview image auto  adjusts exposure as i pan it around ? i don't have ml installed yet. is this a camera feature i have to disable?

You probably need to turn on exposure simulation https://vimeo.com/1944877

I also have a 50d, but I vaguely do remember something about the crop not being in the center by default... I may be wrong but isn't there a electronic "dolly" feature that allows you to move the crop around with ML?

@Redrocks indeed I'm talking Canon H264 here, not ML files. So my understanding is that Canon does change ADTG/CMOS configuration for 160, 320, 640 and doesn't for 100, 200, 400 which are obtained in software post processing. Does this sound correct?

I believe native ISOs refer to the available analog ISO gains to the sensor, which are the 100,200,etc. (set in the CMOS #0 register)

The ADTG/CMOS configuration can adjust only in those native increments, and the third stop gains are applied digitally after the nearest native ISO settings have been set in the configuration.

If I recall correctly, some videos that demonstrated the lower noise in 320, 640, etc were due to the fact that these are negative digital gain applied to the closest ISO of 400, 800, etc... which on our canon dslrs result in better shadow noise due to the fact that it is effectively "exposing to the right" then stopping down.  Even the PDF says that Canons are notorious for having bad shadow recovery.  This is why 125, 250, 500 yield higher noise since they are taking ISO 100,200,400 and digitally gaining +1/3 EV

(I'm not sure if this is the right place to post this,so if not, my apologies!)

I was thinking about trading in my t2i for a 50d to take advantage of the raw capabilities. If I wanted to film a long project, would a hacked 50d work better than what I've got? I've been researching specs but I'm still a bit fuzzy on how stable this thing is.

I have both the 50d and T2i right now and I'll have to say for the most part it depends on what you'd like to do for your "long project"

Currently I'd say the 50d ML is pretty stable for now and I hadn't had issues recording for long periods of time.  Granted, it's still experimental in a way, but for the purposes of your situation let's just say it's perfectly stable.

With that said, you'll be getting into a lot more required post processing and setup when it comes to recording raw. 
- External Mic and recording device (if you need sound that is)
- 1000x minimum speed CF card (64GB for about 15 min of 1584x896 res footage at 24p, ) (or slower if you're only shooting a few second clips)
- Ample amounts of hard drive space, preferably ssds for storage and transcoding
- fast cpu and lots of ram for video editing and stuff (typically that is)
- transcoding every clip
- learn to conform and grade
Also there's only a max of 1584x896 res (though upscaled to 1080p seems pretty good people say) or 1080p24 in 5x zoom
With raw you do get a minor increase in sharpness and resolution(though moire and aliasing might crop up easily, where special care needs to be taken) (or a good amount in 1080p crop), a lot in dynamic range (for grading), but you do need to know how to use it

If this is going to be a feature length film, just 1 hour would require at least 512 GB of space, and if it's a short film... well that's somewhere along the lines of footage you'd record and cut from... That is just all the raw data... Still needs to be extracted into DNGs or turned into some intermediary codec...

Anyway it'll require a LOT of work, so if you aren't familiar with the workflow or have the necessary hardware for it.. it'll be a pain..

I shoot a lot of events which are hours in length and I find that the T2i h264 at 1080p or 720p is more than sufficient since I don't need to do grading and it just gets cut together for youtube uploading.  I got the 50d and a komputerbay 1000x 64gb for $650 (i paid like 100-200 more than necessary) because I want to learn about the capabilities, raw workflow, and possible further developments with ML since I am a comp eng.

If you're technically able to find grading footage shot in h264 limiting and restricting, then it's a good consideration to make, else it might be better to stick with the T2i instead of swapping it for the 50d

I went to Great Falls Park, and tried out a couple res on some waterfalls and friends

I made the final video have 2048x1152 so it would trigger youtube's original res for 18-26 MBPS

Edit: I've also uploaded a Final uncompressed output with 8bit lagarith lossless codec (5.6 GB) available here: https://docs.google.com/file/d/0BxmF1M7jtUvFZzhhOVlHUV9Ub0U/edit?usp=sharing

Just be careful of rolling shutter i suppose...

Wonder if HDR raw capture is possible directly to a fast CF...
I don't really understand how he got the HDR stream in Live View.
It almost looks to me like he's using that board to remote control magic lantern functions on the camera via USB.
Someone got any ideas on how his set-up works?

Computer engineer here,
I've recently worked on digilent FPGA boards, and what you see is real time image processing being done on the 60d' video output.  (FPGA's is a chip that allows you to reconfigure it to do different things, almost like you're programming a CPU)  FPGA's are good because they can run very fast in their configuration since it's all physical. He said that it stores the different exposed frames in a buffer and processes them for HDR and outputs it to the monitor.
(the fpga is acting as an image processing device inbetween the camera and the monitor)

I'm not sure if he's remote controlling it perse, but it isn't necessary... It seems it's just the regular ML hdr video mode (maybe he modified some ML code to just activate it when not recording)

The mac version doesn't even exist. I honestly don't believe it works. No word since posting...

Have faith!
Also I don't think he'd go through the trouble of pretending to show a commandline where he just compiled his program just to make it seem like he got it working..

Also I was thinking about making a similar program as it's much easier to do on a computer rather than in camera.  The raw format is already open sourced.. so writing something to play back is not too difficult.  Also with sufficient storage device access speed, playback is possible, the complications maybe from the debayering algorithms and color correction but other than that it should work

It would be great if there were a windows version

[F4 21 00 00]

wow hahaha just a couple hours ago I also discovered this fix using the same exact program, hxd
trying to save some footage

I am a computer engineer though, and it's helpful to use
https://bitbucket.org/hudson/magic-lantern/src/1222c61e8120e24f2d8fd4836179c5891a4bc20f/modules/lv_rec/lv_rec.h?at=unified

Code Select Expand

#ifndef _lv_rec_h_
#define _lv_rec_h_

#include "raw.h"

/* file footer data */
typedef struct
{
    unsigned char magic[4];
    unsigned short xRes;
    unsigned short yRes;
    unsigned int frameSize;
    unsigned int frameCount;
    unsigned int frameSkip;
    unsigned int sourceFpsx1000;
    unsigned int reserved3;
    unsigned int reserved4;
    struct raw_info raw_info;
} lv_rec_file_footer_t;

#endif

where each char is 1 byte (2 hex characters), short is 2 bytes, and int is 4 bytes, that way you can crossreference the struct definition with the number's you are editing in hex.

the raw hex stores these numbers in "little endian" which means that if you want to obtain the real value of an integer from raw hex you would have to write it in reverse( raw hex: "52 41 57 4D 80 07 38 04 00 5F 37 00 F4 21 00 00 ..."  is 00 00 21 F4 or just 0x21F4)
I used windows calculator in programmer mode to convert from hex to dec quickly (f4 f5)

I used footers from raw videos that were correct to calculate how many frames i had using the framesize field which is the size of a frame in bytes.  I took the file size and divided by the frame size to get a rough rounded down frame count that i would translate into hex, into little endian and write into the raw hex in that location in the footer.

Does anybody have any specs on the actual throughput of these gigabit ethernet peripherals?

It seems like there's some sort of dock port/connection directly into the camera body suggesting that it may be possible to use the full gigabit speeds as it seems tightly integrated, but I'm just speculating. If it does support full speeds, it may be easier and possible to find a mating connection and manufacture our own board to connect to it for cheaper than the grip..
http://www.cameralabs.com/reviews/Canon_WFT_E3A/outdoor_results.shtml

It also has a usb port to connect a storage device but i imagine that'd be too slow.

While there's always a possibility of doing that, it is a bit too impractical.  The BGA memory chips would have to be the exact same pin out and voltage requirements.  Then we'd have to make sure that the entire memory is addressable and not conflict with other peripherals (speculating, maybe possible if there's some slack, and with ML).  Even if we could theoretically increase the Memory, it would only minimally increase recording time because the buffer offloading speed to cards would have to be fast enough. 

If we could find a way to funnel the output through the cameras HDMI-out, I find it highly plausible that we could find a manufacturer with an HDMI interfaced ARM board to team up with.

I registered just now just to suggest this idea, but you already have haha.

Anywho, as a computer engineer w/ some minor embedded systems background, I say it could be feasible if we could directly access the HDMI data stream... I haven't been able to look into the code base for ML closely yet, so I'm not sure if this is possible.  I heard that the existing HDMI out is interlaced and not progressive? (unsure) But either way, we'd have to find a way to losslessly encode the RAW data into the HDMI stream, if the Digic's are fast enough... On the other end  there could be hardware solution that takes hdmi to SATA or CF (might be able to try this out with my FPGA if i find it..)