crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView

[Danne]

Tested some more around ffmpeg and blackdetect filter. I´m sure there are more ways to go on about this but this seems to work as a oneliner in bash to cut of the black frames from the MOV file. MOV file should this way match start of MLV file and after this extracting the wav file is easy. Will work on automation details in Switch.
FFmpeg needs to be installed if anyone wants to test.

How this monkeywrench oneliner is working:
The INPUT.MOV is your H.264 proxy and OUTPUT.MOV will be the shortened MOV file without the blackframes. To speed up processing I do "killall ffmpeg" after one second of processing to get the the black frame detected. It then proceeds with copying out the "cleaned" OUTPUT.MOV file.

Code: [Select]

ffmpeg -i INPUT.MOV -ss $(sleep 1 && killall ffmpeg & ffmpeg -i INPUT.MOV -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) -vcodec copy -acodec copy OUTPUT.MOV
Do a for loop inside a folder with MOV files and get them all processed:

Code: [Select]

for i in *.MOV; do
ffmpeg -i "$i" -ss $(sleep 1 && killall ffmpeg & ffmpeg -i "$i" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) -vcodec copy -acodec copy new_"$i"
done
Seems to be a few blackframes at the end as well. Not sure if we want to cut those out. Might check into it later.

[Lars Steenhoff]

Nice work
Yes there are also a few black frames at the end. for the most easy sync they could be cut too.

[Lars Steenhoff]

Also when using 1080 with x3 crop the frames turn out red in the h264. ( RAW lite + h254 )

[Lars Steenhoff]

@danne

can ffmpeg also extract just the audio with the right length and save as a WAV file ?

[Danne]

Code: [Select]

ffmpeg -i INPUT.MOV -c copy -map 0:a OUTPUT.WAV
Extract from the "cleaned" proxy file

Working on detecting the blackframes at the end of the files...

[Danne]

Here is a for loop which will trim and "clean" beginning and end of the H.264 proxy file. It needs some testing so if anyone could check if the new files match the MLV file it would be nice to know. If running into issues please upload the LOG.txt file.

*Not really working with longer files regarding chopping of the ending. Start of files works with any file with the former for loop. Gonna check further when I have time...

cd into a folder with proxy MOV files and put following in terminal:

Code: [Select]

exec &> >(tee -a LOG.txt >&2 )
for i in *.MOV; do
start=$(sleep 1 && killall ffmpeg & ffmpeg -i "$i" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) ;
end1=$(sleep 10 && killall ffmpeg & ffmpeg -i "$i" -vf reverse -t 2 -f matroska pipe:1 | ffmpeg -i - -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) ;
end=$(echo $start + $end1 | bc -l) ;
duration=$(ffmpeg -i "$i" 2>&1 | grep -o 'Duration: .*' | cut -d "," -f1 | cut -d ":" -f2,3,4 | tr -d ':')
ffmpeg -i "$i" -ss $start -t $(echo $duration - $end | bc -l) -vcodec copy -acodec copy new_"$i"
done

[AndreasKielb]

Hi,

do you think it might be possible to include resolution settings for anamorphic? I already experimented with the EOS 50D and anamorphic but it is limited to lower resolutions. With the 5D3 it might be possible to go higher and according to the first crop_rec on steroids posts it's not unrealistic that it might work at continous recording.

 /*   24p   25p   30p   50p   60p */
    [CROP_PRESET_3X_TALL]       = { 1920, 1728, 1536,  960,  800 }, /* 1920 */
    [CROP_PRESET_3x3_1X]        = { 1290, 1290, 1290,  960,  800 }, /* 1920 */
    [CROP_PRESET_3x3_1X_48p]    = { 1290, 1290, 1290, 1080, 1080 }, /* 1920; 1080p45/48 <- 50/60p in menu */
    [CROP_PRESET_3K]            = { 1920, 1728, 1504,  760,  680 }, /* 3072 */
    [CROP_PRESET_UHD]           = { 1536, 1472, 1120,  640,  540 }, /* 3840 */
    [CROP_PRESET_4K_HFPS]       = { 2560, 2560, 2500, 1440, 1200 }, /* 4096 half-FPS */
    [CROP_PRESET_FULLRES_LV]    = { 3870, 3870, 3870, 3870, 3870 }, /* 5796 */

The aspect ratio does not necessarily have to be 4:3. The new Arri Alexa models allow for a new 6:5 ratio. With 2x anamorphics that leads to 2.4 : 1 and is perfect for anamorphic shooters. http://www.arri.com/de/camera/alexa/kameras/kamera_details/alexa-sxt-plus/subsection/new_sxt_features/

A ML resolution of 2048 horizontal and 1716 vertical would be the best option. With 2x anamorphics that would be 4096 : 1716 after desqueeze just like the new Alexa SXT 6:5 ProRes 4K cine anamorphic mode . If you have the time it would be great if you could add some other resolution options in that area in order to find what works at continous recording. 1920 horizontal and 1600 vertical would also be great for 3840 : 1600 desqueezed.

[a1ex]

@Danne: here's my attempt:

Code: [Select]

#!/bin/bash
# Trim black frames at start and end of a H.264 clip

readarray blacks < <(ffmpeg -i "$1" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect")
first_black=${blacks[0]}
last_black=${blacks[-1]}

start=$(echo $first_black | grep -oP "(?<=black_end:)[^ ]*")
end=$(echo $(echo $last_black | grep -oP "(?<=black_start:)[^ ]*"))
echo "$1: cutting at $start and $end..."

# with -ss $start after -i, it also trims the first good frame, why?
# in this case, we have to use -ss start -i input.mov -to end-start
# with short clips, bc outputs .1234 rather than 0.1234, but ffmpeg doesn't like it
# also, without trickery, it may leave one extra black frame
trimmed=$(echo $end - $start - 0.01 | bc)
ffmpeg -ss 0$start -i "$1" -to 0$trimmed -vcodec copy -acodec copy new_"$1" -loglevel quiet

Only tested with 2 very short clips.

For long clips, you may want to scan only the first and the last 2-3 seconds. With some trickery, the analysis can probably be done in one single ffmpeg command (see e.g. https://superuser.com/a/898765 ), if you know the clip duration in advance ( http://ffmpeg.gusari.org/viewtopic.php?f=11&t=1284 ).

[Danne]

Nice. Works good on Ubuntu. Shortly tested on a virtual box set up with limited drive etc. Mac cries about grep syntax as usual but with some tweaking it will work on mac too.
The beginning frames are easier to remove. I have some ideas about the ending when it comes to longer files to avoid long scan delays.

*All filters(-vf) seems to require re-encoding which most likely will delay the trim filter solution(yet to be verified). That is why my reverse effect filter won´t work since it seems to encode audio first then video which will delay processing a lot. What works is -ss and -t with the lossless -copy function splitting files at the end on longer files for example. Also putting the -ss before the -i input file as suggested in above script.

Code: [Select]

ffmpeg -ss 01:19:00 -i Input.MOV -t 01:21:00 -c copy Output.MOVFetching the duration of the file is easy with ffmpeg:

Code: [Select]

ffmpeg -i Input.MOV 2>&1 | grep -o 'Duration: .*' | cut -d "," -f1 | cut -d ":" -f2,3,4More digging/testing todo...

Also curious what the array from a1ex script will do with longer takes. Probably scan the whole file though which will slow down processing a lot.

Code: [Select]

readarray blacks < <(ffmpeg -i "$1" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect")I simply kill the process after one second of running and it would probably work with the array as well.

Code: [Select]

readarray blacks < <(sleep 1 && killall ffmpeg & ffmpeg -i "$1" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect")Then again it seems the array is fetching blackframe detection for both beginning and end parts so in this case we´ll have to wait for the whole process to finish.

[a1ex]

Right - here's a version with two ffmpeg commands for analysis (and one ffprobe):

Code: [Select]

#!/bin/bash
# Trim black frames at start and end of a H.264 clip

duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$1")
echo -n "$1: duration=$duration; "

# analyze first and last 3 seconds
# clips shorter than 3 seconds require extra care
# bc outputs short values as .1234 rather than 0.1234, but ffmpeg doesn't like this; prepending a 0 does the trick
near_end=0$(echo "($duration > 3) * ($duration - 3)" | bc)
first_black=$(ffmpeg -i "$1" -to 3 -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect" | head -n1)
last_black=$(ffmpeg -ss $near_end -i "$1" -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect" | tail -n1)

# keep from first black_end (or 0, if the clip doesn't start with black frames)
# to last black_start (or to $duration, if the clip doesn't end with black frames)
start=0$(echo $first_black | grep -oP "(?<=black_end:)[^ ]*")
end0=0$(echo $last_black | grep -oP "(?<=black_start:)[^ ]*")
end=0$(echo "($end0 > 0) * (0$end0 + $near_end) + ($end0 == 0) * $duration" | bc)

if [ "$end0" == "00" ]; then
    echo "please increase near_end"
    exit
elif [[ "$start" == "0" && "$end0" == "0" ]]; then
    [ $(echo "$end - $duration" | bc) == "0" ] || echo assert
    echo "nothing to do"
    exit
else
    echo "cutting at $start and $end..."
fi

# with -ss $start after -i, it also trims the first good frame, why?
# in this case, we have to use -ss start -i input.mov -to end-start
# also, without trickery, it may leave one extra black frame
trimmed=0$(echo $end - $start - 0.01 | bc)
ffmpeg -ss $start -i "$1" -to $trimmed -vcodec copy -acodec copy new_"$1" -loglevel error -n

Usage:

Code: [Select]

for f in *.MOV; do ./cut_black.sh $f; done

Sample output:

Code: [Select]

3U3B6192.MOV: duration=3.128125; cutting at 01.29296 and 02.210545...
3U3B6194.MOV: duration=2.502500; cutting at 01.20954 and 01.626620...
BIG__3GB.MOV: duration=71.740000; nothing to do
BLACK_4GB.MOV: duration=90.300000; please increase near_end
MVI_4134.MOV: duration=46.946900; nothing to do

Shouldn't be affected by black frames in the middle of the clip, but might get confused by H.264 dark frames

The threshold may have to be increased at high ISOs and/or with low-contrast picture styles (didn't check).

[Danne]

Nice!
There goes my "killall" roundtripping. Thank god .

Code: [Select]

first_black=$(ffmpeg -i "$1" -to 3 -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep "blackdetect" | head -n1)

Code: [Select]

# clips shorter than 3 seconds require extra careexactly.

Code: [Select]

Shouldn't be affected by black frames in the middle of the clip, but might get confused by H.264 dark framesDetect this through ffmpeg as it will be treated as one long blackframe?

Code: [Select]

ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i Input.MOVyes, workable format. Can´t ffmpeg output this too. Gonna check. Nah, seems ffprobe is the way to go. Another dependency...
Static version of ffprobe here:
https://github.com/joshwnj/ffprobe-static

[Danne]

Here is my latest take on a proxy cleansing script after chewing a1ex refinements. Much faster than the initial script.

Code: [Select]

for i in *.MOV; do
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
first_black=$(ffmpeg -i "$i" -to 3 -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
near_end=$(echo $duration - 3 | bc -l)
last_black=$(echo $duration - $(ffmpeg -ss $near_end -i "$i" -vf "blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) | bc -l)
trimmed=$(echo $last_black - $first_black - 0.01 | bc -l)
ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
done
Note the change for detection of ending blackframes(Had a few issues with proxy files only having 0.495 blackframes):

Code: [Select]

-vf "blackdetect=d=0.1:pix_th=0.01"
Still seems to be one last black frame survivor at the end. Soon...

[Lars Steenhoff]

If we dont want to confuse black with a dark scene, perhaps it would be better to use another color instead of black?

[Kharak]

I just tried Latest Build (2017-08-26 21:25)

On 5D3 123. Everytime I Q button (sub menu) anything in the menu, after that Shutter Scroll wheel and Apertures wheels are Yielding "Busy" when moved. The Aperture I can change in ML Menu, shutter speed will change in ML menu, but the busy sign comes on and it does not change to the shutter speed after all.

If I don't touch anything in the menus, I can keep using the Shtt wheel and Ap wheel.

Also Shutter Fine Tuning was behaving very weirdly, but that most likely has something to do with this.

Sorry to say, but I can't work with this build at all in this state.

[Danne]

I noticed the busy sign as well. Exiting and reentering liveview again seemed to get rid of it. Not a fix of course.

@Lars Steenhoff
I'd be surprised if ffmpeg would mix dark footage with blackframes. Do test against some very dark shots and tell us if the script breaks.

[a1ex]

If we dont want to confuse black with a dark scene, perhaps it would be better to use another color instead of black?

If you know how configure the sensor to output some other solid color, why not?

(pretty sure it's possible, some fiddling with adtg_gui should reveal how; for the current implementation, setting the exposure to 0 was the easiest way)

Everytime I Q button (sub menu) anything in the menu, after that Shutter Scroll wheel and Apertures wheels are Yielding "Busy" when moved. The Aperture I can change in ML Menu, shutter speed will change in ML menu, but the busy sign comes on and it does not change to the shutter speed after all.

Reproduced - the BUSY message appears when pressing half-shutter (e.g. to close ML menu), but it's not related to submenus. Workaround: press Q twice outside ML menu. Or, enter ML menu and exit it by pressing DELETE rather than half-shutter. Related bug: ETTR sometimes locks up for the same reason.

The "fix" would be to re-enable the SRM use-after-free (not straightforward, but not hard either). Reason: keeping SRM memory allocated causes Canon code to show the BUSY message (and I don't know where/how to patch it).

However, using unallocated memory is not exactly a good practice. Therefore, if one can find a solution to keep the SRM memory allocated (marked as used, so Canon won't overwrite it, no matter what), but without the BUSY message, that would be the best way to solve this.

[Kharak]

Pressing Q twice "releases" the wheels again. Nice workaround.

Btw!! another tip for people with Manual Lenses. When starting the camera in crop mode with a manual lens attached, liveview wont start, the camera does not detect a lens and therefore does not turn on the liveview. Instead of entering photo to movie mode and turning off and On crop mode etc. A workaround: when you turn on Camera with a manual lens after 1-2 secs the message "Ensure a lens is attached..." It blinks quickly about 2-3 times, for the duration this message is blinking, you can press "Start/Stop" button and it will force in to liveview in Crop Mode boy did i save a couple thousand shutter actuations after learning this

Sometimes you might have to go in to Canon Menu and back for the shutter speeds to sync.

Just press Start once, if you press twice h264 recording starts immediatly.

But also thinking if this is something a ML/Script could do?

[a1ex]

With a manual lens, without pressing anything, LiveView starts directly into crop mode. With a Canon lens, it flickers a bit (starts in regular mode, then switches to crop mode after 1-2 seconds). Not sure what the issue is...

edit: sometimes the screen remains black when switching modes around (not sure how to reproduce); in this case, you don't need to go to photo mode; just press MENU twice (see first post).

[Lars Steenhoff]

Do you get an h264 with the black frames when recoding in crop mode? 3x crop 1080 the first option in the crop menu?

[a1ex]

Yes, except they are red, just like in your report. The same happens in Canon's 5x zoom, so it's not a misconfiguration in crop_rec.

I can set exposure time to about 1/23000 (or maybe 1/46000? didn't measure it); however, that may confuse the script if the scene contains bright lights (so it won't be much better than red).

[Lars Steenhoff]

Ok I see, well its not a problem because its easy to manually cut the red frames.
just i was wondering if i did something wrong. thanks for comfirming its normal

[a1ex]

Here's a quick and dirty patch for the BUSY issue:

Code: [Select]

--- a/src/exmem.c
+++ b/src/exmem.c
@@ -357,6 +357,11 @@
     /* unlock the shutter button */
     /* (only touch the shutter-related bits, just in case) */
     gui_uilock(icu_uilock & ~UILOCK_SHUTTER);
+
+    /* 5D3 1.2.3 */
+    void (*SRM_ClearBusy)(void *, int) = (void *) 0xFF1F1F68;
+    void * RscMgr = 0x241C8;
+    SRM_ClearBusy(RscMgr, 1);
 }

Press shutter fully to see why it's not committed

[Danne]

Just tried some really dark shots and ffmpeg correctly did not detect them as blackframes. I think we´re cool with black.
Updated my script with some analysis. Having some difficulties comprehending A1ex advanced pieces but I think I got something similar in another way. bc causes parse errors as expected when missing a zero. It works anyway even if it looks shitty in terminal:

Code: [Select]

for i in *.MOV; do
duration=$(ffprobe -loglevel quiet -of compact=p=0:nk=1 -show_entries format=duration -i "$i")
first_black=$(ffmpeg -i "$i" -to 3 -vf "blackdetect=d=0.5:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2)
near_end=$(echo $duration - 3 | bc -l)
last_black=$(echo $duration - $(ffmpeg -ss $near_end -i "$i" -vf "blackdetect=d=0.1:pix_th=0.01" -an -f null - 2>&1 | grep -o "black_duration:.*" | cut -d ":" -f2) | bc -l)
trimmed=$(echo $last_black - $first_black - 0.01 | bc -l)
if ! [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
elif ! [ x"$first_black" = x ] && [ x"$last_black" = x ]; then
ffmpeg -ss $first_black -i "$i" -t $duration -vcodec copy -acodec copy new_"$i"
elif [ x"$first_black" = x ] && ! [ x"$last_black" = x ]; then
ffmpeg -i "$i" -t $trimmed -vcodec copy -acodec copy new_"$i"
fi
done
Still have to think something out with shorter than 3 s files.

[a1ex]

The bc trickery goes like this:

Code: [Select]

echo "0.2+0.2" | bc
.4
echo "2.5+2.5" | bc
5.0

The first value is not accepted by ffmpeg (wherever it may appear); let's prepend a 0:

Code: [Select]

echo 0$(echo "0.2+0.2" | bc)
0.4
echo 0$(echo "2.5+2.5" | bc)
05.0

The leading zero from the second example is harmless - both values are accepted by ffmpeg.

Now the conditionals: I've looked through the man page of bc (it's the first time I use it) for some conditional operator, or min/max functions, but couldn't find any. As I didn't want to figure out the advanced syntax, I've stumbled upon the relational operators - for example (1 < 2) evaluates as 1 and (1 == 2) evaluates as 0. That makes it easy to build conditional expressions without knowing the if/then syntax.

After a second look over the man page, the if/then syntax no longer looks that hard:

Code: [Select]

echo "if (1 == 2) 5 else 7" | bc

BTW, a nice alternative to bc is pythonpy.

[Kharak]

With a manual lens, without pressing anything, LiveView starts directly into crop mode. With a Canon lens, it flickers a bit (starts in regular mode, then switches to crop mode after 1-2 seconds). Not sure what the issue is...

edit: sometimes the screen remains black when switching modes around (not sure how to reproduce); in this case, you don't need to go to photo mode; just press MENU twice (see first post).

Okey, my experience with manual lenses without electronics is that the liveview wont start. I'll say for the slow mo crop modes atleast. I have not used the high res modes for quite some time.

Edit: 2x menu works aswell! I retreat my bug. But Start/Stop button thing works too.