Messages

I'm not sure what you mean by "fixing" live-view but A1ex recently announced a new crop mode feature which allows you to shoot in 1:1 crop mode at 1080p in colour. Check out THIS thread. It's not currently part of the nightly builds so you'll have to download from that page.

As for getting continuous 48fps at 1080p I believe that's not possible due to the data rate required. Have you tried shooting at 38fps at a high shutter speed (say 1/300 or faster depending on your subject - if you're filming something that moves quite quickly then you may even need 1/4000 or faster to capture "crisp" frames) and then using optical flow slow-mo in post to slow it down to 60fps or more? Most NLEs now have optical flow as an option when doing slow-mo. I've had good results from filming running shots at 32fps with a fast shutter and slowing down by 50% in FCPX with optical flow enabled.

Are you able to see the graphs from my post?

I'm afraid not. They're just dead-image placeholders on my computer. Your textual explanation was enough though. Thank you again, A1ex.

Having thought long and hard about things I think I'm going to stick to recording 1920x803 in 3x3 binning mode instead of worrying about recording 2.5K in 1:1 crop mode like some idiotic megapixel junkie as from my non-scientific tests it seems that the quality of each pixel seems better in the 3x3 binning mode over the 1:1 crop mode - the result of which (now that you are educating me a little) I'm sure is more than likely down to the lower noise per pixel, but of course I could be completely wrong.

Cheers,

Mark.

Thanks for sharing. May I ask what ISO you shot at for both? Could just be the vimeo compression but looks like there's a fair bit of noise there in both shots (even in the ungraded footage). May I ask how you lit the scene?

Thanks again,

Mark.

Noise:
- 1:1 crop is more noisy (3x3 binning captures 9x as many photons)
- DR is similar because, on Canons, the averaging is done in hardware, before the read noise gets introduced;

Thanks A1ex. Did a quick and dirty noise test here if you're interested. Those photos you attached don't seem to show up on my computer though.

So, if the 3x3 binning mode captures 9x as many photons, does that mean that for equal ISOs the 3x3 binning mode will have potentially 9x less noise than in 1:1 crop mode or is that being simplistic?

Cheers,

Mark.

Hi guys and gals,

Since playing around with 1:1 crop mode a few weeks back I've been pseudo-obsessed with thinking about perhaps re-shooting the three short scenes I've done so far in my movie @ 2560x1072 (instead of the original 1920x804 that they were shot at) with the mind-logic that it should (surely) look better scaled down to 1080p rather than recording 1080p natively. Surely.

However, none of this was with any testing, just hypothetical logic in my ickle brain, so tonight I thought I'd try a quick and dirty shoot to test out the noise between 1:1 and 3x3 binning after A1ex's reply in this thread. It's clear to see at 3200 ISO that 5x zoom mode is decidedly more noisy. In fact, in contrast the 1920x804 non-crop image is very useable at 3200 ISO. Still plenty of detail.

Note: All I'm testing here is noise.

All I did with these shots were bring them into Resolve, add a Rec.709 color space and desaturate them to make them legal in the vectorscope:

1920x804 @ 3200 ISO on 1920 timeline


2560x1072 @ 3200 ISO scaled down to 1920 timeline


1920x804 @ 3200 ISO scaled up to 2560 timeline


2560x1072 @ 3200 ISO on 2560 timeline


1920x804 @ 3200 ISO scaled up to 4K timeline


2560x1072 @ 3200 ISO scaled up to 4K timeline


1920x804 @ 3200 ISO scaled up to 4K timeline (left corner)


2560x1072 @ 3200 ISO scaled up to 4K timeline (left corner)



Now, obviously there are differences in the setup of this test. Even though both times I focused on the nail varnish bottle (not mine, honest) there will be differences in the DOF as these tests were done using the same lens with me just moving back a bit to try and frame the shot the same as the non-crop image in 1:1 crop mode (mainly because I don't have a super-wide lens to get the same perspective in 1:1 crop). Not scientific at all.

If I was doing this scientifically (and if it wasn't so late now) then I'd use my 35mm lens for the 1:1 crop mode and then match that shot with a zoom lens at around the 100mm mark without moving the tripod etc. I may do that tomorrow if I've got time and add a selection of different ISOs into the mix too.

Right now though, at 3200, and with regards to noise, even with the different DOF it looks clear to me that the advantages the extra photons bring to the table in 3x3 binning mode ("normal" recording) far outweigh the extra pixels of the 5x zoom recording. Not just on the 1920 timeline but when upscaled to 4K too. It'll be interesting to see if that's the case at lower ISOs too.

Not rushing out to re-shoot that footage at the moment though...

The binning patterns are described here: http://www.magiclantern.fm/forum/index.php?topic=16516

Just a quick question if I may, A1ex... If I understand it correctly, does that mean that the "normal" (non-crop mode) recording in ML uses pixel binning but crop-mode recording does not as it's a 1:1 crop of the sensor? If that is indeed the case does that mean that the image quality is potentially better in crop mode?

Please feel free to slap me with a wet fish if I've misunderstood.

Thanks,

Mark.

Please feel free to mark this request as "not possible" as I now understand why from the above linked post and A1ex's explanation. I was thinking of possibilities in terms of pixels instead of the RGB pre-debayered sensor information.

Cheers,

Mark.

Thanks for the clarification and information, A1ex.

Cheers,

Mark.

The binning patterns are described here: http://www.magiclantern.fm/forum/index.php?topic=16516

Ah. Makes sense now as we're not dealing with debayered pixels but the RGB "blobs" that make up the pixels, of course. Thanks, A1ex. I take it from the photo you attached that the non binned bits are the ones with the green rectangles around them?

Can I take it from the binning pattern post that one must have a pattern in 3s so that equal amounts of red and blue are removed?

[1x3 Binning]

Thank you to @dmilligan for pointing me to this thread from my own in-camera anamorphic request. I immediately downloaded the build to try it out! 

After doing some tests using with the 1x3 Binning mode, and whilst excellent(!), I think that the 3:1 compression (not sure if compression is the right word to use here. I mean the height:width ratio.) is perhaps a little too severe for critical shots but great for shots with less detail. Here are two stills to illustrate:

Here's a still from a video taken with the new 1:1 (3x) crop mode @ 1920x1080 (so great being able to see the framing properly in colour! Thank you!):



And here's a still taken from a video shot @ 1024x1290 in 1x3 Binning mode with the width resized to 3072 and the height cropped by 40 pixels to remove the top black line:



As you can see, the binned image is perfectly useable for non-detail shots and in those circumstances it would be fine (especially considering the lower data rate requirement of shooting at 1024x1290!), but as to be expected with a compression of 3:1 the image is noticeably softer than the native 1:1 image and no longer "looks" like RAW. More like H.264.

Because of this I'm wondering if it's possible to create a binning mode of 1x2 or even a binning mode of 1x1.5 or 1x1.33?

If that is technically possible I think that would greatly add to the functionality you've included here as for example we'd be able to record 1804x1152 @ 1x1.5 to give an aspect ratio of 2.35:1 @ 2707x1152 when stretched. I'd imagine that the image would not suffer too much with such a small squeeze.

Thanks again for your hard work.

Cheers,

Mark.

p.s.

- 1x3 binning: read all lines, bin every 3 columns (extremely squashed image)

Just as a matter of interest, is this correct with regards to the pixels that are thrown away or kept?

Code Select Expand


Column 1 - discard
Column 2 - discard
Column 3 - discard
Column 4 - keep
[repeat]


<I'm re-doing this post after more tests... Will update soon>

http://www.magiclantern.fm/forum/index.php?topic=17021.msg165260#msg165260

Wow, did I read that correctly? It's already been implemented?! :-O

Thanks for the reply, Walter. From your reply I guess what you're saying is that it's impossible(?) but as ML already does this with the vertical resolution when shooting in 60fps (from the Canon menu) - the 1080p is squeezed by a 1.67x vertical anamorphic factor during recording so that it's recorded at 1920x646:

http://www.magiclantern.fm/forum/index.php?topic=5472.0

...can it not be squeezed horizontally to 1920 from 2538 (2.35:1) so that it's recorded at 1920x1080 or is there a technical limitation in 1x recording mode where the widest available horizontal pixels are 1920?

Cheers,

Mark.

Incidentally, I do know that I can record at a higher pixel count in 3x mode but it'd be great to be able to use my 24mm lens at 24mm and still have 1080 vertical pixels at 2.35:1 in 1x mode - hence my request. (In case no-one is replying as they think I've overlooked 3x mode! )

Hi guys,

It occurred to me the other day when doing some high-fps test shots on my 5D III with ML that the vertical image is "squeezed" and needs to be "unsqueezed" in post to bring it back to 1080p. This lit a bulb in my head. A lightbulb, if you will... Ping!

So... would it be possible to squeeze the HORIZONTAL pixels by 1.34x to mimic the anamorphic lens squeeze so that it's still recorded at 1920x1080 but when it's stretched back out it becomes 2581x1080? If it's possible then perhaps a variable multiplication factor choice could be used, i.e. 1.34x for 2.39:1, 1.32x for 2.35:1 etc.

If this is indeed possible then the higher pixel count would be a great boon for ML and cinematography as a whole and would do away with the need of buying an anamorphic lens. Obviously you forego other advantages of an actual anamorphic lens such as horizontal flare, shallower DOF etc but at least we wouldn't be throwing away pixels when shooting at 2.35 / 2.39. Ya know?

Answers on a postcard to...

Mark.

Is there a way to find the White Balance information so that I can input it into Resolve? Currently, "As shot" returns 2770.

Thanks,

Mark.

[Mac OS Extended (Journaled)]

Cannot reproduce. Spaces and brackets work fine here. Upload the whole crash log and also ~/mlvfs.log. Can you reproduce the issue if you change the name back to what it was? What exactly was the drive name and what format?

Can't seem to find mlvfs.log on my system but changing the name back resulted in the same error:




The volume format is Mac OS Extended (Journaled) and the name was "MLV Backup (Samsung 2TB)". Maybe it's the length of the name?

Here's the latest crash report:

Code Select Expand

Process:               mlvfs [427]
Path:                  /Users/USER/Library/Services/*/mlvfs
Identifier:            mlvfs
Version:               0
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           mlvfs [427]
User ID:               501

Date/Time:             2016-03-26 15:55:21.382 +0000
OS Version:            Mac OS X 10.11.3 (15D21)
Report Version:        11
Anonymous UUID:        2F2B3BBB-6F28-0BF2-82FE-21ED37D41433


Time Awake Since Boot: 75 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000027800000010
Exception Note:        EXC_CORPSE_NOTIFY

VM Regions Near 0x27800000010:
    Process Corpse Info    000000010a53b000-000000010a73b000 [ 2048K] rw-/rwx SM=COW 
-->
    MALLOC_TINY            00007fc382c00000-00007fc382e00000 [ 2048K] rw-/rwx SM=PRV 

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_c.dylib              0x00007fff93f54152 strlen + 18
1   libsystem_c.dylib              0x00007fff93faeb79 strdup + 18
2   mlvfs                          0x0000000107af5d25 main + 309
3   libdyld.dylib                  0x00007fff8a2395ad start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000001  rbx: 0x0000000000000000  rcx: 0x0000027800000019  rdx: 0x0000027800000019
  rdi: 0x0000027800000010  rsi: 0x00007fff64bea248  rbp: 0x00007fff5810caa0  rsp: 0x00007fff5810caa0
   r8: 0x0000000000000000   r9: 0x00007fc382d00145  r10: 0x00007fff9bcb7f01  r11: 0x00007fff93faeb67
  r12: 0x0000000000000000  r13: 0x0000000000000000  r14: 0x0000027800000019  r15: 0x0000000000000000
  rip: 0x00007fff93f54152  rfl: 0x0000000000010202  cr2: 0x0000027800000010
 
Logical CPU:     2
Error Code:      0x00000004
Trap Number:     14


Binary Images:
       0x107af3000 -        0x107b61ff7 +mlvfs (0) <FE4AA6CB-2C80-3284-BEEC-C38549A7E38A> /Users/USER/Library/Services/*/mlvfs
       0x10a2d8000 -        0x10a2f6fe7 +libosxfuse_i64.2.dylib (10.3) <B953F9DD-EF0F-3FA6-AB08-D8D96131C9CE> /usr/local/lib/libosxfuse_i64.2.dylib
    0x7fff64bea000 -     0x7fff64c21007  dyld (360.19) <9D05FDF4-65CE-3B53-86D4-ABE1A5BF35F3> /usr/lib/dyld
    0x7fff84f05000 -     0x7fff84f21ff7  libsystem_malloc.dylib (67) <9EECAB18-F025-34C4-8E32-7EFFA6720EFC> /usr/lib/system/libsystem_malloc.dylib
    0x7fff8798f000 -     0x7fff87997fff  libcopyfile.dylib (127) <F5133269-0B22-388C-A57C-079667B6291E> /usr/lib/system/libcopyfile.dylib
    0x7fff87f9d000 -     0x7fff87f9dff7  libunc.dylib (29) <1D0F8265-F026-3CBD-93D3-F8DF14FFCE68> /usr/lib/system/libunc.dylib
    0x7fff89117000 -     0x7fff8911aff7  libsystem_sandbox.dylib (460.30.1) <3E0036AF-FC64-3352-8DA4-6B550C2C2562> /usr/lib/system/libsystem_sandbox.dylib
    0x7fff8977a000 -     0x7fff897a9ffb  libsystem_m.dylib (3105) <26655445-CA97-321E-B221-801CB378D1AA> /usr/lib/system/libsystem_m.dylib
    0x7fff899ae000 -     0x7fff89aa0ff7  libiconv.2.dylib (44) <F05A0A5A-92A9-3668-8F20-F27CBDA26BE9> /usr/lib/libiconv.2.dylib
    0x7fff89aa1000 -     0x7fff89b02ff7  libsystem_network.dylib (583.20.10) <865FE79A-A22D-3733-A14F-FC7B37F3AECD> /usr/lib/system/libsystem_network.dylib
    0x7fff89b31000 -     0x7fff89b48ff7  libsystem_asl.dylib (322.30.1) <9B500E4E-E462-321E-828E-5524DC984C1B> /usr/lib/system/libsystem_asl.dylib
    0x7fff89c92000 -     0x7fff89e9fffb  libicucore.A.dylib (551.41) <CFFD7342-A7D6-323A-AC14-B9EECF6EFFED> /usr/lib/libicucore.A.dylib
    0x7fff8a236000 -     0x7fff8a239ffb  libdyld.dylib (360.19) <AA629043-C6F6-32FE-8007-E3478E99ACA7> /usr/lib/system/libdyld.dylib
    0x7fff8b048000 -     0x7fff8b08eff7  libauto.dylib (186) <999E610F-41FC-32A3-ADCA-5EC049B65DFB> /usr/lib/libauto.dylib
    0x7fff8b4ed000 -     0x7fff8b50bfff  libsystem_kernel.dylib (3248.30.4) <9CEB6C3B-1CAF-3C32-A9FD-93BC72CBCEA1> /usr/lib/system/libsystem_kernel.dylib
    0x7fff8b581000 -     0x7fff8b581ff7  liblaunch.dylib (756.20.4) <EDF719D6-D2BB-38DD-8C94-4272BEFDA2CD> /usr/lib/system/liblaunch.dylib
    0x7fff8b5b0000 -     0x7fff8b5b0ff7  libkeymgr.dylib (28) <09397E01-6066-3179-A50C-2CE666FDA929> /usr/lib/system/libkeymgr.dylib
    0x7fff8b5e7000 -     0x7fff8b5f0ff7  libsystem_pthread.dylib (138.10.4) <327CECD0-B881-3153-8FCC-4FD4818B7F16> /usr/lib/system/libsystem_pthread.dylib
    0x7fff8ba5c000 -     0x7fff8ba89fff  libdispatch.dylib (501.20.1) <324C9189-2AF3-3356-847F-6F4CE1C6E901> /usr/lib/system/libdispatch.dylib
    0x7fff8c76a000 -     0x7fff8c76bffb  libSystem.B.dylib (1226.10.1) <5A4257EF-3145-3BB3-87A4-0D2404A9462D> /usr/lib/libSystem.B.dylib
    0x7fff8cbef000 -     0x7fff8d065fff  com.apple.CoreFoundation (6.9 - 1256.14) <768A7FB7-9143-3148-8591-7C6ED3162D35> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
    0x7fff8d06c000 -     0x7fff8d074fe7  libsystem_platform.dylib (74.10.3) <D3A27E10-7F08-3603-ACC8-7A92B2C04BAB> /usr/lib/system/libsystem_platform.dylib
    0x7fff8d58e000 -     0x7fff8d590ff7  libquarantine.dylib (80) <163CF63A-7455-3D1F-AE57-8C4475A9204C> /usr/lib/system/libquarantine.dylib
    0x7fff8d591000 -     0x7fff8d5e4ff7  libc++.1.dylib (120.1) <8FC3D139-8055-3498-9AC5-6467CB7F4D14> /usr/lib/libc++.1.dylib
    0x7fff901b1000 -     0x7fff901b3ff7  libsystem_configuration.dylib (802.20.7) <5FD79070-36CC-3D02-BEA7-BB5D2AE97D5D> /usr/lib/system/libsystem_configuration.dylib
    0x7fff910de000 -     0x7fff910e2fff  libcache.dylib (75) <6B245C0A-F3EA-383B-A542-5B0D0456A41B> /usr/lib/system/libcache.dylib
    0x7fff937bb000 -     0x7fff937e4fff  libsystem_info.dylib (477.20.1) <6513635B-4ADE-3B45-BF63-ED7AC565B0C9> /usr/lib/system/libsystem_info.dylib
    0x7fff93f44000 -     0x7fff93f45fff  libsystem_secinit.dylib (20) <FD6ECF2C-1489-32CA-981B-9045B5EB1FAA> /usr/lib/system/libsystem_secinit.dylib
    0x7fff93f53000 -     0x7fff93fe0fff  libsystem_c.dylib (1082.20.4) <EAB38A6C-8671-3B13-B500-90EC1B912063> /usr/lib/system/libsystem_c.dylib
    0x7fff94718000 -     0x7fff94720fff  libsystem_networkextension.dylib (385.20.6) <DC8A102A-BF02-31A4-8914-65C34DF6B592> /usr/lib/system/libsystem_networkextension.dylib
    0x7fff9478c000 -     0x7fff9478dfff  libDiagnosticMessagesClient.dylib (100) <4243B6B4-21E9-355B-9C5A-95A216233B96> /usr/lib/libDiagnosticMessagesClient.dylib
    0x7fff9526b000 -     0x7fff95273ffb  libsystem_dnssd.dylib (625.20.4) <945B5FB1-DA91-3D45-A961-A8FAD53C1E7E> /usr/lib/system/libsystem_dnssd.dylib
    0x7fff95508000 -     0x7fff95509ffb  libremovefile.dylib (41) <B8D1A5FC-CFD5-3AAB-8A10-14DDC129710A> /usr/lib/system/libremovefile.dylib
    0x7fff955ee000 -     0x7fff95617fff  libc++abi.dylib (125) <DCCC8177-3D09-35BC-9784-2A04FEC4C71B> /usr/lib/libc++abi.dylib
    0x7fff957bc000 -     0x7fff957bdfff  libsystem_blocks.dylib (65) <49D42329-7DE9-3413-92C3-A473A7E9CF35> /usr/lib/system/libsystem_blocks.dylib
    0x7fff95bf7000 -     0x7fff95c00ff3  libsystem_notify.dylib (150.20.3) <243FADE1-255A-3B78-8033-F336CD64B817> /usr/lib/system/libsystem_notify.dylib
    0x7fff9615f000 -     0x7fff96164ff7  libmacho.dylib (875.1) <CB745E1F-4885-3F96-B38B-2093DF488FD5> /usr/lib/system/libmacho.dylib
    0x7fff96629000 -     0x7fff9662bfff  libsystem_coreservices.dylib (19.2) <1B3F5AFC-FFCD-3ECB-8B9A-5538366FB20D> /usr/lib/system/libsystem_coreservices.dylib
    0x7fff96a33000 -     0x7fff96a38ff3  libunwind.dylib (35.3) <124E0F05-2350-3774-A32C-7F5BF38EDE73> /usr/lib/system/libunwind.dylib
    0x7fff96ca0000 -     0x7fff96cabff7  libcommonCrypto.dylib (60075.20.1) <766BC3F5-41F3-3315-BABC-72718A98EA92> /usr/lib/system/libcommonCrypto.dylib
    0x7fff96f8d000 -     0x7fff96f94ff7  libcompiler_rt.dylib (62) <D3C4AB40-23B4-3BC6-8C38-5B8758D14E80> /usr/lib/system/libcompiler_rt.dylib
    0x7fff97028000 -     0x7fff97039ff7  libz.1.dylib (61.20.1) <B3EBB42F-48E3-3287-9F0D-308E04D407AC> /usr/lib/libz.1.dylib
    0x7fff97840000 -     0x7fff97851ff7  libsystem_trace.dylib (201.10.3) <B485369F-E3A1-319E-998C-89AAF606079E> /usr/lib/system/libsystem_trace.dylib
    0x7fff9896f000 -     0x7fff98cda657  libobjc.A.dylib (680) <58CB8CFC-7DBD-3A53-BD72-A42FF799B21E> /usr/lib/libobjc.A.dylib
    0x7fff99bbb000 -     0x7fff99be4fff  libxpc.dylib (756.20.4) <61AB4610-9304-354C-9E9B-D57198AE9866> /usr/lib/system/libxpc.dylib
    0x7fff9ae26000 -     0x7fff9ae3cff7  libsystem_coretls.dylib (83.20.8) <75C97D88-0A63-3093-AE83-DE33EB7405CE> /usr/lib/system/libsystem_coretls.dylib
    0x7fff9aea3000 -     0x7fff9af1afe7  libcorecrypto.dylib (335.20.1) <C6BD205F-4ECE-37EE-BCAB-A76F39CDCFFA> /usr/lib/system/libcorecrypto.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 131
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=106.9M resident=0K(0%) swapped_out_or_unallocated=106.9M(100%)
Writable regions: Total=61.8M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=61.8M(100%)

                                VIRTUAL   REGION
REGION TYPE                        SIZE    COUNT (non-coalesced)
===========                     =======  =======
Activity Tracing                  2048K        2
Kernel Alloc Once                    4K        2
MALLOC                            10.2M        8
MALLOC guard page                   16K        4
Process Corpse Info               2048K        2
STACK GUARD                       56.0M        2
Stack                             8192K        2
VM_ALLOCATE                          4K        2
__DATA                            42.2M       50
__LINKEDIT                        91.3M        5
__TEXT                            15.6M       48
__UNICODE                          552K        2
shared memory                        8K        3
===========                     =======  =======
TOTAL                            227.8M      119



[Do extensive tests]

Wow! Lovely grading on your video. Top stuff! 

As far as your pink frames go, in my experience they only rear their ugly head if I'm "pushing" ML. For example, recording sound in ML is now something I never do, always externally with sync mark to an in-shot hand clap as the less you ask ML to do, the better. Another culprit I've found is Global Draw. I now only turn it on if I absolutely have to and when I do have to (for focus peeking or framing etc) I know that there's a risk of pinkies appearing so I try and compensate for that by shooting twice if possible.

Another thing I've found to help (and this could be purely co-incidence) is to always have "fps override" set to 23.976 - unless I'm shooting at a higher frame-rate, of course.

In a nutshell...

- Only load the modules you absolutely need
- Don't record sound in ML
- Don't use Global Draw unless you absolutely have to
- Keep your fps at 23.976
- Use a Komputerbay 128Gb CF card (in my tests they came out on top)
- exFAT format is a must and format your card after every data dump. DO NOT just delete the files.

And this is the most crucial...

- Do extensive tests so that you know how much you can push ML before heading out to shoot and use MLRawViewer in "frame-by-frame" mode (not real time) to check your footage for pinkies after your tests so that you know what settings to avoid in ML to avoid or reduce them.

In my own personal experience I wouldn't hesitate shooting professionally with a 5D III and ML providing the above points are met. It really is a fantastic piece of software and on that note, if I may be so frank, calling it a "hack" is doing somewhat of an injustice to all the developers who have spent thousands of hours of their time developing this free tool for the community - not to mention how terrible it sounds to your clients if you tell them you're using a "hacked" camera.  Perhaps "modified" would be more appropriate...?

Cheers,

Mark.

[EDIT:]

99.999% of the time spent by the bad pixel fix feature is *looking* for bad pixels (you have to check each and every pixel, and there are millions, and analyze all of it's neighbors)

Ah, of course. Makes sense.

An ideal algorithm would probably pick a handful of frames and use all of them *together* to find stuck pixels, but such a complex algorithm is beyond the amount of free time I have to work on this project.

I understand. How about a third option, that is to keep the "Aggressive" flag as it was (i.e., checking every frame) and add another setting to the UI called "Aggressive (first frame check only)"? That way if we experience hot pixels due to camera heat / cooling down etc then we could choose the "every frame" option but otherwise just use the "quick" option... Thoughts?

Incidentally, I have a problem running MLVFS from a certain drive. It results in a crash:

Code Select Expand


Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000027800000010
Exception Note:        EXC_CORPSE_NOTIFY


I can send you the full crash report if you want.

Basically I moved all of my MLV files to an external Raid 0 drive and tried to mount the MLV folder from there. It goes through the process of showing the "choose your mount point" window but as soon as I do it opens Safari to a blank page saying that it could not open the page.

I've tried having the mount folder locally and on the external drive but to no avail.

I've set the privileges on the drive to "Read & Write" for me, "Staff" and "Everyone" as I assume it's a write error?

This does not happen with all external drives. Works fine with another external SSD drive that I have.

Cheers,

Mark.

EDIT: I've found the problem with the error. It seems that MLVFS doesn't like spaces (or it could be brackets) in the drive name. I renamed the drive to use underscores instead of spaces (and removed the brackets) and all now works as expected.

We can actually make the bad pixel fix considerably faster by only analyzing one frame and then simply applying the correction to those pixels in all frames. This should work in theory since bad pixels are typically fixed. I've updated the Mac download with a build that implements this. Please let me know how it's working. Also fixed is the TC issue @dfort found.

But does the bad pixel feature not take into account corresponding colours when it "blends"?

May I ask what is the difference between "On" and "Aggressive" in terms of what's happening to the image?

Thanks,

Mark.

I already have so if you don't seen version info at the bottom of the page, you must be running an old version.

Ha. No problem. Have just updated.

A good way to measure speed is to cat the DNGs to /dev/null and time that.

Great tip, thanks! I shall use that in future. For now though as I have some historic time data I re-ran my jobs after re-linking the media to cDNGs that I'd copied from the MLVFS volume with the "Aggressive" flag set. As to be expected it has made a massive difference doing it that way. A job which took 12.5hrs to render now took only 1.5hrs.

There's no real noticeable difference in rendering speed as far as I can see (in my admittedly un-scientific tests) between jobs that are rendered via cDNGs that are opened directly via MLVFS with the Fix Pixel flag set to OFF or copied cDNGs. The only slow-down seems to be when setting the Fix Pixel flag to Aggressive. In that instance it seems much quicker to first copy out the cDNGs from MLVFS to a new folder with the flag set to "aggressive" (took 2hrs) and then relink the media to the copied files rather than opening them directly through MLVFS.

This is on my machine of course and with the clips fully graded in Resolve - i.e., they're not simply "spat out" the other side in delivery from the source clips. I'd imagine that would speed things up dramatically too.

Cheers,

Mark.

I've fixed some memory leaks recently, maybe I haven't updated the build, I'll check.

Ah ok. I'm not sure which build I have. Perhaps you could amend the http://localhost:8000 control page to display the current build of MLVFS installed to make checking if we're running the latest version easier?

If bad pixel fix slows down your process much you can get rid of the bad pixel with a secondary node in resolve instead. You make super small masks that blur out the stuck pixels. It's quite tiresome and fiddely to make masks for all the bad pixels but once you have done it you can save all the masks as a preset that you can reuse later since the bad/stuck pixels will be in the same places.

Funny you should mention this as I was going to shoot a black matte at high ISO to make just a power grade with the technique you mentioned! Great minds an' all that...

But I think I've discovered the culprit, speed wise. I did some tests with a 60Gb cDNG folder and copying it to various places.

1. Bad Pixel Fix = "Aggressive". Copying from MLVFS > HDD took 130 minutes.
2. Bad Pixel Fix = "Off". Copying from MLVFS > HDD took 11 minutes.
3. Copying the copied folder to another location on my HDD took 7 minutes.

So from my quick test it takes 11 times longer with "Aggressive" but when set to "Off" it's only 1.5 longer than copying from HDD to HDD (only SATA II on this laptop unfortunately) so MLVFS is incredibly fast when set to "Off". Obviously, 130/7 = 18 times quicker when opening the copied DNGs directly from my HDD.

I'm now going to do another test in Resolve whereby I open the copied folders instead of the MLVFS folders to see if there's any significant difference in UI / Playback / Rendering speed over opening from MLVFS directly.

Cheers,

Mark.

Firstly, thank you @dmilligan for developing this solution for the community. It really is most excellent. I wonder if it would be possible for you to answer these questions for me as my search on this forum didn't bring up any:

1. What is the difference between "On" and "Aggressive" for bad pixel correction in terms of use and speed differences (if any)?
2. What are the differences in speed between opening the cDNGs using MLVFS or by first copying out the cDNG files to a folder and opening them there?

I ask because rendering 1080p clips from my 5D III in DaVinci Resolve 12 Lite is painfully slow for me. I have a MBP i7 8Gb RAM / 512Mb Nvidia with 512Gb SSD system drive and 2TB SSD media drive installed and rendering a 2 minute clip to ProRes 4444 with "Aggressive" bad pixel correction (as "on" didn't get rid of them) as well as colour grading in Resolve took 6 hours. I'm currently rendering another timeline with various clips and a total duration of 4 minutes and the estimated render time is SIXTEEN hours - it's 11 hours in right now so I'm not going to stop it to do some tests at this point.

I'd just like to get to the bottom of where the bottleneck is (aside from my mediocre Mac system compared to today's standards!) as I've read that others are getting 40fps render times. Is it the "Aggressive" mode or is it opening the cDNG files directly instead of first copying them?

Thanks for your help,

Mark.

EDIT: I just stopped the render as it was ridiculous so I'm going to do some tests. Incidentally, MLVFS is using a lot of RAM...

<removed>

or AE for better details IMO at the cost of longer rendering times especially @ ProRes4444XQ

I've not tried the AE approach yet but I will give it a go. As I'm right at the beginning of shooting my feature I think it's worth exploring all avenues to get the best workflow that works for me before getting set into a way of working which I regret later - although having said that, I don't particularly like AE so I may try Lightroom instead - I take it they both use ACR in the same way?

Can't see MLRawViewer not being part of the workflow though! 

Cheers,

Mark.