crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView

[domasa]

Today I had crash during record:

CRASH00.LOG
ASSERT: IsSuiteSignature( hSuite )
at ./PackMemory/PackMem.c:983, task MoviePlay
lv:0 mode:3


Magic Lantern version : crop_rec_4k.2017Apr29.5D3123
Mercurial changeset   : d55423f1e270 (crop_rec_4k) tip
Built on 2017-04-29 14:31:28 UTC by jenkins@nightly.
Free Memory  : 164K + 3811K

ASSERT00.LOG
ML ASSERT:
fullsize_buffers[1] == UNCACHEABLE(raw_info.buffer)
at mlv_lite.c:1379 (free_buffers), task raw_rec_task
lv:1 mode:3


Magic Lantern version : crop_rec_4k.2017Apr29.5D3123
Mercurial changeset   : d55423f1e270 (crop_rec_4k) tip
Built on 2017-04-29 14:31:28 UTC by jenkins@nightly.
Free Memory  : 144K + 3821K

Info
I used H.264 proxy (H264->SD with ML; RAW 1920x1080 12-bit->CF)

Magic Yoom Large TL 3>1, Waveform small, Red Crossout, DIGIC peaking: Slightly sharper, Zoom with old button..

[hjfilmspeed]

@pc_bel, @goldenchild9to5:

Have a look at the preview options; the help text has some hints for your use cases (also discussed earlier in this thread).

@a1ex this feature changed everything! So awesome!

[DeafEyeJedi]

Some fun stuff w DarkFraming Avg Process in 12-bit lossless @ ISO 6400 from 5D3.123 shot w Helios 44-2.

Cinelog_Rec709 has been applied for comparison viewing purpose. Full album can be found here: https://flic.kr/s/aHskVk85io

Pre DF (pushed exposure up to +2.50 in ACR):


Post DF (pushed exposure up to +2.50 in ACR):


Pre DF (pushed exposure up to +2.50 in ACR):


Post DF (pushed exposure up to +2.50 in ACR):


Pre DF (pushed exposure up to +2.50 in ACR):


Post DF (pushed exposure up to +2.50 in ACR):


Pre DF (pushed exposure up to +2.50 in ACR):


Post DF (pushed exposure up to +2.50 in ACR):


Such a relief that @bouncyball, @Danne, @g3gg0 and the rest of you all have made this possible to bring back this useful feature in DarkFraming Average Processing. Especially for lossless compressions.

Yes, all of this was solely done with the latest and greatest cr2hdr.app (Thanks @Danne for standing by with all of your love & support!)

[janvkem]

Looking good DeafEyeJedi! Do you create the darkframe by shooting (with the same settings) with the lenscap attached? Can you do it anytime after the shot (like when i go on holiday and i make the darkframes when i come back home) or does it have to be close together in time? For windows users mlv_dump can do this right?

Quote (with pics removed):

Some fun stuff w DarkFraming Avg Process in 12-bit lossless @ ISO 6400 from 5D3.123 shot w Helios 44-2.

Cinelog_Rec709 has been applied for comparison viewing purpose. Full album can be found here: https://flic.kr/s/aHskVk85io

Such a relief that @bouncyball, @Danne, @g3gg0 and the rest of you all have made this possible to bring back this useful feature in DarkFraming Average Processing. Especially for lossless.

Yes, all of this was solely done with the latest and greatest cr2hdr.app (Thanks @Danne for standing by with all of your love & support!)

[DeafEyeJedi]

Do you create the darkframe by shooting (with the same settings) with the lenscap attached?

Correct.

Can you do it anytime after the shot (like when i go on holiday and i make the darkframes when i come back home) or does it have to be close together in time?

Anytime works.

For windows users mlv_dump can do this right?

I believe so, not sure how tho.

[janvkem]

Thanks for the reply, going to implement this into my workflow!

[pc_bel]

Quote from: a1ex on Yesterday at 10:07:47 PM

    @pc_bel, @goldenchild9to5:

    Have a look at the preview options; the help text has some hints for your use cases (also discussed earlier in this thread).

Thank you!!! I didn't see that feature.
Hope it worked in 50p3x3 for correct framing.
@A1ex
It's easy to implement it in other button different from the shutter release?...
Thanks again.

[a1ex]

There are a few other buttons that send the same event (it's hard to tell the difference between them, other than waiting about 0.5 seconds to see whether it starts autofocusing or not). Use any of them

[pc_bel]

 
THANKS!!!!

[goldenchild9to5]

Thanks @a1ex

[whysodifficult]

Hello everyone,

Could someone explain, please, what i do wrong? My brain is just exploding of complexity.

I've been trying to shoot 3k or 4k or 5k for a few days using instructions here, but everything either work 5 sec or switches to 1920 automatically. Every day it behaves differently. One day i managed to work each resolution but time was different - 3 sec mostly. Couldn't do what i've seen others shot.

Right now i tried to shoot 3,5K crop according to instructions and it allowed me to shoot only 5 sec, also preview freezes. Maybe i've done something wrong with FPS override? I tried 23.976 (from 25), Exact FPS - 23.976 and Low light - 23.980.

My goal is to find the maximum resolution and aspect ratio with reasonable time (say, from 5 min of recording).

I have 5DM3 + 1.1.3 Firmware + Sandisk 30 mb/s (exFAT) with ML (crop from 29 April) + KomputerBay 64GB 1000x (exFAT).

Thank you!

[giarcpnw]

@whysodifficult:

Are you choosing a lossless compression? 3.5k only seems to work continuously with 12bit lossless. 3k maybe with 14bit lossless. You'll have to lower frame rates down significantly to get 4 or 5k. They are not feasible options for traditional frame rates.

[hjfilmspeed]

@whysodifficult I am currently making a video tutorial for 3k if you can give me to the end of the day

[rob_6]

@whysodifficult I am currently making a video tutorial for 3k if you can give me to the end of the day

@hjfilmspeed

Really looking forward to your video tutorial! Thanks so much in advance!

[Danne]

Been fiddling around mlv_dump code in crop_rec_4k branch and by doing two minor changes it works a little better around 10bit12bit files.
https://bitbucket.org/Dannephoto/magic-lantern/commits/b1e226a565287ae563944cc035957bd213852855?at=crop_rec_4k_ver2

What seems to work now is following:

MLV files
- Compression(lj92) of all flavours of mlv files including 10bit12bit14bit, compressed raw, regular raw. Compression level is very efficient, almost 50%.

- Decompression of mlv files into decompressed dng files or back to decompressed mlv. This works with all MLV flavours going from compressed straight to decompressed dng files. This is really great since we do not need to decompress the MLV file once it´s compressed.

DNG files
- Compression of 14bit dng files, all flavours works very good. Efficient compression around 40-50%.

Not working
- Compression of 10bit12bit files. DNG output comes out all black but compressed and with metadata present.

I did some minort changes to get 10bit12bit working at all. There is also some testing with decompressing code since the files will come out with incosistent sizes when decompressing. Depends on if coming from mlv_block or outputting dng files. Check code here.
https://bitbucket.org/Dannephoto/magic-lantern/commits/b1e226a565287ae563944cc035957bd213852855?at=crop_rec_4k_ver2

I´m trying to see what is missing around 10bit12bit now. One funny note is that if erasing bitdepth = 14; mlv_dump will output compressed 10bit12bit files fine into original 10bit12bit. Of course this will not work with vetical stripes code since it´s written in 14bit but then again it means it´s halfway working when run_compressor is set.

If any hints to these findings I´d be greatful since I hit a wall now.

Here are some testfiles by the way.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/10_12_14_bit.zip

[ju64]

Hi everyone,

I spent a few days trying to shoot with the crop mode 3.5K but nothing conclusive.

When I use crop mode 3.5K 1:1 centered (resolution 3504x1320 and fps overide 23.976) if I use 12 bit lossless I have the overexposed zone pink, with 14 bits lossless no problem.

Any idea where is my problem ? Did I miss something ?

Thank you !

[whysodifficult]

@whysodifficult:

Are you choosing a lossless compression? 3.5k only seems to work continuously with 12bit lossless. 3k maybe with 14bit lossless. You'll have to lower frame rates down significantly to get 4 or 5k. They are not feasible options for traditional frame rates.

If lossless compression means 10-12 bit instead of 14 bit, then yes, i tried it as well. Thank you for the tip!

I wonder can there be a difference between 1.1.3 and 1.2.3 firmware versions.... I'm using 1.1.3.

@whysodifficult I am currently making a video tutorial for 3k if you can give me to the end of the day

Thank you very much! I am looking forward to watch it, so as other people, i'm sure!

[pc_bel]

Been fiddling around mlv_dump code in crop_rec_4k branch and by doing two minor changes it works a little better around 10bit12bit files.

@danne Thank you very much!!!

[hjfilmspeed]

@whysodifficult @rob_6
This video is not for those new to ML. Try this at your own risk. I am not responsible for any damage this video may bring if your camera explodes or bricks. Magic Lantern is not responsible either. It will probably just brick not explode though.
Magic Lantern 3072 2.35:1 12 bit Lossless RAW Recording from the CROP_REC_4K build from 4-29-2017
Once you have ML installed and the correct build on your SD card, this video should help you set up your 5DIII to record 3K almost continuously depending on the scene. Remember ML on SD and Rec to CF.
Remember this build is experimental and not for those that are new to ML. Try at your own risk.

[g3gg0]

Been fiddling around mlv_dump code in crop_rec_4k branch and by doing two minor changes it works a little better around 10bit12bit files.
https://bitbucket.org/Dannephoto/magic-lantern/commits/b1e226a565287ae563944cc035957bd213852855?at=crop_rec_4k_ver2

What seems to work now is following:

hi danne
thanks for your feedback and testing!

does that mean with that changes you managed to compress 10/12 bit images correctly?
made test cases for all combinations of compression/decompression/recompression/pass-thru-unprocessed-lossless to check mlv_dump behavior.

though i didn't complete that to cover 10/12 bits, just made some sloppy dev tests.

here is the test case i had for these tests:

Code Select Expand


mlv_dump_test: mlv_dump
rm *.dng
./mlv_dump -v 14bit_lossless.mlv -o mlv_dump_test_original.mlv
./mlv_dump -v 14bit_lossless.mlv -o mlv_dump_test_decompressed.mlv -d
./mlv_dump -v 14bit_lossless.mlv -o mlv_dump_test_recompressed.mlv -c
./mlv_dump -v 14bit_lossless.mlv -o mlv_dump_test_recompressed_double.mlv -c -c
./mlv_dump -v mlv_dump_test_original.mlv              --dng -o mlv_dump_test_0_exp_0
./mlv_dump -v mlv_dump_test_decompressed.mlv          --dng -o mlv_dump_test_1_exp_0
./mlv_dump -v mlv_dump_test_recompressed.mlv          --dng -o mlv_dump_test_2_exp_0
./mlv_dump -v mlv_dump_test_recompressed_double.mlv   --dng -o mlv_dump_test_3_exp_0
./mlv_dump -v mlv_dump_test_original.mlv              --dng -o mlv_dump_test_4_exp_1  -c
./mlv_dump -v mlv_dump_test_decompressed.mlv          --dng -o mlv_dump_test_5_exp_1  -c
./mlv_dump -v mlv_dump_test_recompressed.mlv          --dng -o mlv_dump_test_6_exp_1  -c
./mlv_dump -v mlv_dump_test_recompressed_double.mlv   --dng -o mlv_dump_test_7_exp_1  -c
./mlv_dump -v mlv_dump_test_original.mlv              --dng -o mlv_dump_test_8_exp_2  -c -c
./mlv_dump -v mlv_dump_test_decompressed.mlv          --dng -o mlv_dump_test_9_exp_e  -c -c
./mlv_dump -v mlv_dump_test_recompressed.mlv          --dng -o mlv_dump_test_a_exp_3  -c -c
./mlv_dump -v mlv_dump_test_recompressed_double.mlv   --dng -o mlv_dump_test_b_exp_3  -c -c
md5sum mlv_dump_test_*.dng


the result then i manually checked if that is what i expected.
all with exp_0 should have the same hash, same for exp_1, exp_2 and exp_3.
the exp_e is expected to throw an error.

[Danne]

Hi g3gg0. Thanks for getting back on this. I'm out at the moment but will describe from memory.
As you describe 14bit compression(-c --dng) is working nicely with dng exports. When exporting 10bit12bit the dng files comes out all black.
What is working is compression of MLV files and that works for all flavours, 10bit12bit14bit.

What is adressed in my code changes is to firstly get regular uncompressed 10bit12bit working again at all since it was broken in this build. Next was to try -c with 10bit12bit but always comes out black.

My last code changes made it possible to export to uncompressed dng files coming from compressed MLV. That works with all bits and flavours.
Sorry for these messy allround tests.
If I would suggest for one priority fix is to get -c --dng option working with 10bit12bit files if possble at all?

[whysodifficult]

@hjfilmspeed

Thank you very much for the video, i did everything you said. It works now!! I really appreciate it!

[D_Odell]

Anyone else's experienced that 12-but is unavailable to choose in latest build 1.1.3? It says in mine that it's only available in photo mode.

[a1ex]

It says in mine that it's only available in photo mode.

There's no code path that could display this message in the raw video menu, and there are no lower bit depths available in photo mode.

http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

When I use crop mode 3.5K 1:1 centered (resolution 3504x1320 and fps overide 23.976) if I use 12 bit lossless I have the overexposed zone pink, with 14 bits lossless no problem.

See commit messages from 2028d73, 945aadb and 2aee5f8.

[rob_6]

@hjfilmspeed

Amazing! Thank you so much for the tutorial video. Everything makes sense now! So seriously cool you made this for us! I love magic lantern! You guys are all awesome!

Rob