Author Topic: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView  (Read 414106 times)

sylvain_c

  • New to the forum
  • *
  • Posts: 39
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #750 on: May 23, 2017, 10:56:16 PM »
crop_rec_4k Apr29 not working here. I have followed exactly the video tutorial, but it result in random multicolored pixels (viewed in Mlrawviewer and raw2cdng1.7.9). also tested with and without fps overide.

dng sample:
https://mega.nz/#!DocmzRJJ!-A96kskYpm7ZqzFynl_4MZH9yd0cfc9O9bNCcrPlM5k

Edit: Ok sorry, I succeed with mlv_dump.exe
5D MKIII 1.1.3

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5789
Issue with crop_rec_4k mlv_dump
« Reply #751 on: May 24, 2017, 02:21:02 PM »
I would like to focus on 10bit12bit files in recent mlv_dump again. Seems the fix is very close but if it the problem passes unnoticed now it will probably be buried into other functions adding up in time.
I test mlv_dump with the following command(latest crop_rec_4k code)
mlv-dump --dng INPUT.MLV

12bit test file
https://bitbucket.org/Dannephoto/magic-lantern/downloads/12bit.MLV


The last commit that will export 10bit12bit files correctly is this
1738cb0
https://bitbucket.org/hudson/magic-lantern/commits/1738cb06e2652d9751f9407490de507fe30ef520?at=crop_rec_4k


Next commit is where things are getting wrong with 10bit12bit footage. Compiling from this commit will cause segmentation fault 11, however the next commit partly fix this but will export frames with missing information(see examples below)
92d1b4d
https://bitbucket.org/hudson/magic-lantern/commits/92d1b4d2958975f50ba428853545f64db8a61205?at=crop_rec_4k

Following commit, "re-adding 14bpp enforcement for DNG" produces black 10bit/12bit dng files cause it won´t fill the frame entirely(see examples). Dcraw exports the file and it shows where the image information is missing.
8b31c98
https://bitbucket.org/hudson/magic-lantern/commits/8b31c98380d5dc27573886d0f84ac7ed0c5d0f2b?at=crop_rec_4k

Looking at the code changes I see a lot of changes around frame_buffer_size, read, prev_frame_size so I guess somewhere in there this could be fixed. Unfortunately I couldn´t get this right with the example file provided but hopefully a dev could take a look or give a hint on how to fix it.


dcraw -T INPUT.dng gives:

Working commit
1738cb0




Not working



Kharak

  • Hero Member
  • *****
  • Posts: 899
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #752 on: May 24, 2017, 03:49:27 PM »
Can I update from 113 to 123 Canon firmware, if I install Apr. 29 Build? Or do I have to update firmware via EOS utility? I am currently on 113... Its 2 years since I did firmware update  :-\
once you go raw you never go back

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 6834
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #753 on: May 24, 2017, 03:58:08 PM »
You can install *Canon* firmware 1.2.3 using Canon menu. If you do this you have to run ML builds for *Canon* firmware 1.2.3.
ML is not firmware.
All builds should be available for 1.1.3 and 1.2.3. There is no need to upgrade Canon firmware if you don't have to for other reasons.
Photogs and videographers: Assist in proof reading upcoming in-camera help!. Your input is wanted and needed!

Kharak

  • Hero Member
  • *****
  • Posts: 899
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #754 on: May 24, 2017, 04:37:10 PM »
I meant if the 123 firmware is also included in the Apr 29 build. As in i could update to 123 from 113 by updating the build and simaltaneously update the firmware?
once you go raw you never go back

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 6834
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #755 on: May 24, 2017, 04:40:26 PM »
Best way to semi-brick/stalling your cam.
Photogs and videographers: Assist in proof reading upcoming in-camera help!. Your input is wanted and needed!

Savely

  • Freshman
  • **
  • Posts: 52
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #756 on: May 24, 2017, 04:54:53 PM »
@Danne thanks, you clarify issue pretty well. Just to remind you and those who develop some software solutions within magic lantern - currently mlv_dump didn't work properly with Dual ISO MLVs from crop_mode. Sometimes frames corrupted with looking-like-labirynth pattern and it seems randomly.

So it seems that currently there are just one reliable (but long) workflow for HDR-video when you shoot in crop_mode - MLV to DNG-sequence then DNG-sequence to MOV then MOV with Avysinth (in HDR-workflow pack) processed to A, B and C jpg-sequence then this C-sequence again assembled in some final video format.

Kharak

  • Hero Member
  • *****
  • Posts: 899
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #757 on: May 24, 2017, 05:36:06 PM »
Best way to semi-brick/stalling your cam.

So the simple answer is: No. Thank you
once you go raw you never go back

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3154
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3154
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #759 on: May 24, 2017, 11:14:15 PM »
reason for asking is this:
Code: [Select]
Block: RAWI
  Offset: 0x00000034
  Number: 1
    Size: 180
    Time: 212.444000 ms
    Res:  960x540
    raw_info:
      api_version      0x00000001
      height           1181
      width            1888
      pitch            2832
      frame_size       0x003B8A48
      bits_per_pixel   12
      black_level      512
      white_level      4050
      active_area.y1   26
      active_area.x1   152
      active_area.y2   1181
      active_area.x2   1886
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1

Block: VIDF
  Offset: 0x0000027c
  Number: 9
    Size: 3344624
    Time: 550.688000 ms
   Frame: #0000
    Crop: 536x332
     Pan: 536x333
   Space: 0
   depth: 12 -> 14, size: 777600 -> 907200 (116.67%)
   black: 512 -> 2048
   white: 4050 -> 16200

Res:  960x540 -> 777600 bytes frame size.
but the frame seems 1181x1888 -> 3344592 bytes.

looks like the whole raw buffer was written instead of the specified frame size.
patching the fields results in proper frames.

any idea where this file came from?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5789
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #760 on: May 24, 2017, 11:43:14 PM »
It's an early build put into a 600D. Wonder if it was called raw_rec(mlv_lite). Stupid of me not to download a later build when testing. Right now I don't have the camera nor the firmware version until monday.
On another note. Is recent 10bit12bit files working with latest crop_rec_4k like it should? Last I tried it wouldn't work. If so, I guess there is no issue. Don't have any cam here unfortunately to verify.
By the way. That particular file will process with earlier builds. Guess there is some other buffer detectors in previous code?

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5789
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #761 on: May 25, 2017, 12:12:34 AM »
Just tested with mlv files on my eos m coming from mlv_lite and regular mlv_rec and both transcodes 10bit12bit with wrong buffer(black bottom). Downloaded experimental raw_video_10bit_12bit 30 april build. Ran latest mlv_dump (crop_rec_4k) code.

Here is the thing. If I replace
Code: [Select]
if(read_size != (int)frame_buffer_size)with
Code: [Select]
if(read_size > (int)frame_buffer_size)later 10bit12bit files will process correctly. Is this the fix we could use? It still won´t work to compress 10bit12bit files(mlv_dump -c setting produces black output) but I guess they are already compressed being 10bit12bit so maybe compression for lower bits should be excluded?

erek

  • New to the forum
  • *
  • Posts: 26
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #762 on: May 25, 2017, 09:30:09 AM »
Here's my first 3072x1320 24 fps recording:



so with 14-bit lossless 3K i am seeing about 6.5 minutes of record time on a 64GB CF card... does this seem accurate?  also what is a recommended Aspect Ratio?

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3154
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #763 on: May 25, 2017, 09:43:33 PM »
Here is the thing. If I replace
then you will get crashes :)

but thanks for your testing. just pushed a fix that should fix that issue.
after converting bit depth, frame size was updated, but not data size.

reason for having both is obvious when compression is used. frame size wont change with or without compression, whereas the resulting binary data varies in size.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3154
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #764 on: May 25, 2017, 09:48:39 PM »
Here's my first 3072x1320 24 fps recording:
so with 14-bit lossless 3K i am seeing about 6.5 minutes of record time on a 64GB CF card... does this seem accurate?  also what is a recommended Aspect Ratio?
3072 * 1320 * 24 * 60 * 6.5 * 14/8 * 0.58
gives ~38GiB
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5789
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #765 on: May 25, 2017, 10:35:45 PM »
@g3gg0
Code: [Select]
                        frame_buffer_size = new_size; :D
Tested, working. Thanks a million.

lostfeliz

  • Freshman
  • **
  • Posts: 54
  • 5diii. 60d.
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #766 on: May 26, 2017, 03:54:40 AM »
@erek, your shadows look a little noisy. maybe you underexposed or maybe you had a high ISO?

bazidl

  • New to the forum
  • *
  • Posts: 6
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #767 on: May 26, 2017, 09:51:46 PM »
What program are you guys using post production (after exporting via mlv_dump). I would normally use Premier pro cc 2015 but it doesn't recognize the .dngs whenever I use any new feature settings...although I can view the individual .dngs fine in photoshop or any image viewer.

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5789
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #768 on: May 26, 2017, 11:19:08 PM »
Mlvfs or cr2hdr.app will produce cdng files

bazidl

  • New to the forum
  • *
  • Posts: 6
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #769 on: May 27, 2017, 04:35:30 AM »
I tried cr2hdr.app before and still got the same message as using mlv_dump. I'm prob missing a setting that allows for the proper export. Regardless just tried Mlvfs and it worked great!

Thanks for the help

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5789
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #770 on: May 27, 2017, 05:51:20 AM »
I see. What settings did you use for it not to work? I would like to reproduce this issue. Do you have a sample file to test with?

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5789
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #771 on: May 27, 2017, 06:37:10 AM »
@g3gg0
Code: [Select]
mlv_dump -c -c is not working healthy right now. dng files comes out with 0 black and white level for instance. I narrowed the issue down to this change.
Code: [Select]
                            if(new_depth != old_depth)reverting back to this and it works again.
Code: [Select]
                            if(new_depth)

There is also one of these beauties left
Code: [Select]
LJ92: Failed
which could go into nifty --relaxed to let mlv_dump produce compressed dng files.

Code: [Select]
                                /* set new compressed size and copy buffers */
                                frame_buffer = realloc(frame_buffer, compressed_size);
                                assert(frame_buffer);
                                memcpy(frame_buffer, compressed, compressed_size);
                                frame_buffer_size = compressed_size;
                            }
                            else
                            {
                                print_msg(MSG_ERROR, "    LJ92: Failed (%d)\n", ret);
                                goto abort;
                            }
to
Code: [Select]
                                /* set new compressed size and copy buffers */
                                frame_buffer = realloc(frame_buffer, compressed_size);
                                assert(frame_buffer);
                                memcpy(frame_buffer, compressed, compressed_size);
                                frame_buffer_size = compressed_size;
                            }
                            else
                            {
                                print_msg(MSG_ERROR, "    LJ92: Failed (%d)\n", ret);
                                if(relaxed)
                                {
                                    goto skip_block;
                                }
                                goto abort;
                            }

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3154
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #772 on: May 27, 2017, 09:25:34 AM »
first one done, really have to rework some code to make such changes happen just in one point.

the second thing, does this ever happen?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5789
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #773 on: May 27, 2017, 09:39:06 AM »
Quote
the second thing, does this ever happen?
You´re right. I´m working a version which turns on the compression switch when selecting -c -c. Not a problem in crop_rec_4k version.

goldenchild9to5

  • Hero Member
  • *****
  • Posts: 534
Re: crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView
« Reply #774 on: May 27, 2017, 07:03:08 PM »
@g3gg0 @Danne Great job guy's.. ya'll working.