Author Topic: [experiment] [5D3, others?] massive mlv_play speedup  (Read 29924 times)

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3143
[experiment] [5D3, others?] massive mlv_play speedup
« on: September 01, 2014, 02:23:08 AM »
recently i've been trying hard to use hardware engines to process raw video for playback.
unfortunately i did not get far enough to say we have realtime playback.

but using an hardware engine, i was able to speed up processing at least a bit.

a example video with 600 frames, 24fps, 1920x1080, of 25 seconds length
takes 96 seconds using "color" and "all", which means it will play all frames with nice colors and no skipping.

using a tweak module "raw_twk" (see source in unified), which adds new methods for the latest
mlv_play to play raw with improved speeds.

warning:
a) use it on your own risk
b) only compatible with mlv_play
c) THIS IS REALLY EXPERIMENTAL, DONT DARE TO POST HERE IF IT RUINED SOME SHOOT YOU MADE!
d) 5D3 only yet

maybe this is stable enough to make use of it in ML core?

it is making use of "ProcessPathForFurikake" DSUNPACK/DARK/PACK16/WDMAC16 engines which receive 14bpp raw stream and align it correctly into 16bpp.
this eases up the way we can read out the pixel data and basically the most CPU expensive thing is rgb->yuv.

for this reason ive also improved rgb2yuv:

left: original code, right: handcrafted assembly (there are also 6 words of constans not shown)


source
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

MA Visuals

  • Freshman
  • **
  • Posts: 83
Re: [experiment] [5D3] mlv_play speedup
« Reply #1 on: September 01, 2014, 06:06:20 AM »
I ran a few tests using the experimental raw_twk module on my 5d3 with both .raw and .mlv clips

Clip Info:   248 frames, 23.976fps, 1920x1080, 10.3 second clip (playback mode: Color & All)

                                       Before       After    
   Playback in-camera        38 sec       22 sec     73% speed increase
   Playback via HDMI       174 sec       97 sec      80% speed increase

Observations
- HDMI playback benefited a bit more than in-camera playback (an additional 7% speed increase)
- Relative speed increases were the same for both .raw and .mlv files
- The popup mlv_play menu was noticeably more sluggish to appear and dissappear (even more so via hdmi)
- I did not notice any other odd behavior with recording, playback, deleting or navigating clips
- No smoke or explosions (at least for me)

Overall a really nice improvement in playback speed :)

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 1124
Re: [experiment] [5D3] mlv_play speedup
« Reply #2 on: September 01, 2014, 07:55:50 AM »
Wow the posted results above are really a noteworthy performance boost. Congrats.
70D.112 & 100D.101

jpaana

  • New to the forum
  • *
  • Posts: 28
Re: [experiment] [5D3] mlv_play speedup
« Reply #3 on: September 01, 2014, 10:57:22 PM »
Out of curiosity tried this on 5D3.123 (ProcessPathForFurikake is at 0xFF3CCC14) and it seemed to work fine and visibly faster than without the module. Also looked up the function from EOSM.202 (0xFF425150) but had problem with task creation.

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3143
Re: [experiment] [5D3] mlv_play speedup
« Reply #4 on: September 01, 2014, 11:28:39 PM »
yeah the EOS M is afair at the limit of tasks to be run in parallel
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

vstrglv

  • Member
  • ***
  • Posts: 173
Re: [experiment] [5D3] mlv_play speedup
« Reply #5 on: September 04, 2014, 04:52:37 PM »
Very useful but does not work on NB 18July 5D3 133. Old API
Canon 5D3,1.1.3 CF-SanDisk Extreme PRO,160MB/s, 256GB, SD-SanDisk Extreme, 90MB/s, 64GB.

Kharak

  • Hero Member
  • *****
  • Posts: 869
Re: [experiment] [5D3] mlv_play speedup
« Reply #6 on: September 04, 2014, 08:40:24 PM »
If playback is set to "fast", do you get realtime playback?

Anyways, really nice improvement, I will try this on 1.1.3 if thats not a problem? As above post states an issue?
once you go raw you never go back

kgv5

  • Hero Member
  • *****
  • Posts: 643
Re: [experiment] [5D3] mlv_play speedup
« Reply #7 on: September 04, 2014, 08:59:51 PM »
on 5d3 1.1.3 works fine, no problems so far. Really nice improvement :)
www.pilotmovies.pl   5D Mark III, 6D, 550D

Kharak

  • Hero Member
  • *****
  • Posts: 869
Re: [experiment] [5D3] mlv_play speedup
« Reply #8 on: September 04, 2014, 10:09:05 PM »
Installed on 1.1.3. on latest nightly. See significant improvement

Played a clip that took 64 seconds to play before raw_twk and after it was 35 secs.

But latest nightly wouldn't play .mlv files that were recorded before I updated to latest nightly. raw files played fine though.
Nothing to do with raw_twk though, it was not loaded when I started playing the files and noticed that the mlv files would not play, just mentioning.

Thanks a lot for this geggo! Really nice improvement. Haven't really done any recordings with the raw_twk loaded, but also just updated to latest nightly so will be hard for me to tell if I encounter problems, if it is the raw_twk module or latest nightly that causes issues.
once you go raw you never go back

ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: [experiment] [5D3] mlv_play speedup
« Reply #9 on: September 04, 2014, 10:17:21 PM »
But latest nightly wouldn't play .mlv files that were recorded before I updated to latest nightly. raw files played fine though.
Nothing to do with raw_twk though, it was not loaded when I started playing the files and noticed that the mlv files would not play, just mentioning.
Can you try manually deleting the associated IDX file and letting mlv_play re-create it?  I adjusted the IDX format recently, but the change should only affect "exact" playback.  And even then, playback should still work.  But, perhaps some bug slipped past my testing.
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

Kharak

  • Hero Member
  • *****
  • Posts: 869
Re: [experiment] [5D3] mlv_play speedup
« Reply #10 on: September 04, 2014, 10:19:46 PM »
Yes, makes sense that the idx is interferring as these mlv files have been played before on the earlier nightly that I was using.

Thank you, I will try this.
once you go raw you never go back

ayshih

  • Contributor
  • Senior
  • *****
  • Posts: 266
Re: [experiment] [5D3] mlv_play speedup
« Reply #11 on: September 04, 2014, 10:28:30 PM »
Actually, more useful would be if you save a copy of the old IDX file instead of deleting it.  If re-creating the IDX file fixes your issue, send me both the old and new IDX files.  Thanks!
Canon EOS 50D | 17–40mm f/4L & 70–300mm f/4.5–5.6 DO IS | Lexar 1066x

vstrglv

  • Member
  • ***
  • Posts: 173
Re: [experiment] [5D3] mlv_play speedup
« Reply #12 on: September 05, 2014, 09:27:51 PM »
Good speed-up! But magiclantern-Nightly.2014Sep01.5D3113 and raw_twk.mo  has a problem with 5x zoom ML Grayscale. White lines on the screen and  camera hangs sometimes.
Canon 5D3,1.1.3 CF-SanDisk Extreme PRO,160MB/s, 256GB, SD-SanDisk Extreme, 90MB/s, 64GB.

Kharak

  • Hero Member
  • *****
  • Posts: 869
Re: [experiment] [5D3] mlv_play speedup
« Reply #13 on: September 05, 2014, 10:08:54 PM »
Good speed-up! But magiclantern-Nightly.2014Sep01.5D3113 and raw_twk.mo  has a problem with 5x zoom ML Grayscale. White lines on the screen and  camera hangs sometimes.

I got same build, I'll see if I get same results with Crop mode and ml grayscale.

Edit:

Confirmed
once you go raw you never go back

vstrglv

  • Member
  • ***
  • Posts: 173
Re: [experiment] [5D3] mlv_play speedup
« Reply #14 on: September 05, 2014, 10:21:49 PM »
What is confirmed? Is there this issue or not?
Canon 5D3,1.1.3 CF-SanDisk Extreme PRO,160MB/s, 256GB, SD-SanDisk Extreme, 90MB/s, 64GB.

Kharak

  • Hero Member
  • *****
  • Posts: 869
Re: [experiment] [5D3] mlv_play speedup
« Reply #15 on: September 05, 2014, 10:42:38 PM »
What is confirmed? Is there this issue or not?

I confirm that I also get stripes in Crop mode with ML Grayscale ;)
once you go raw you never go back

vstrglv

  • Member
  • ***
  • Posts: 173
Re: [experiment] [5D3] mlv_play speedup
« Reply #16 on: September 05, 2014, 10:51:02 PM »
Thank you!
Canon 5D3,1.1.3 CF-SanDisk Extreme PRO,160MB/s, 256GB, SD-SanDisk Extreme, 90MB/s, 64GB.

Markus

  • Senior
  • ****
  • Posts: 338
Re: [experiment] [5D3] mlv_play speedup
« Reply #17 on: September 06, 2014, 04:15:11 PM »
Got error when trying to load module on latest nightly for 5d3  123 FW. Only tried it with zacuto evf attached.

kgv5

  • Hero Member
  • *****
  • Posts: 643
Re: [experiment] [5D3] mlv_play speedup
« Reply #18 on: September 06, 2014, 08:02:26 PM »
I confirm that I also get stripes in Crop mode with ML Grayscale ;)

The same here. When i enter crop mode camera also crashes (and need to take battery out). 5D3 113
www.pilotmovies.pl   5D Mark III, 6D, 550D

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3143
Re: [experiment] [5D3] mlv_play speedup
« Reply #19 on: September 07, 2014, 01:00:46 PM »
thanks for the feedback.
so you cannot enter crop mode with mlv_rec enabled?
preview mode is enabled, right?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Kharak

  • Hero Member
  • *****
  • Posts: 869
Re: [experiment] [5D3] mlv_play speedup
« Reply #20 on: September 07, 2014, 09:06:47 PM »
I am able to enter and exit crop mode when ML grayscale is activated. But the entire screen is filled with tick white and black chunky pixel stripes that acts like heavy noise, seems like a failed attempt by ML grayscale to output the screen.

The same here. When i enter crop mode camera also crashes (and need to take battery out). 5D3 113

The camera doesn't crash for me, but I haven't tried recording in crop mode like that. Did you record in crop mode?

1.1.3
Sep 1st Nightly


once you go raw you never go back

kgv5

  • Hero Member
  • *****
  • Posts: 643
Re: [experiment] [5D3] mlv_play speedup
« Reply #21 on: September 07, 2014, 09:30:57 PM »
so you cannot enter crop mode with mlv_rec enabled?

Thats correct, mlv_rec (and mlv_play) enabled and raw_twk also enabled. When i press zoom button white mess appears, sometimes it crashes just after pushing once and sometimes when i press zoom second time. It crashes every time though and cannot leave crop mode, every time i need to take battery out.

preview mode is enabled, right?

Right, preview is on auto.

I am able to enter and exit crop mode when ML grayscale is activated.

The same, when ML grayscale is on. When its on 'auto' it crashes.

But the entire screen is filled with tick white and black chunky pixel stripes that acts like heavy noise, seems like a failed attempt by ML grayscale to output the screen.

The camera doesn't crash for me, but I haven't tried recording in crop mode like that. Did you record in crop mode?

1.1.3
Sep 1st Nightly

No, i did not try recording because of the crashes. I am on the same build:  2014Sep01.5D3113
www.pilotmovies.pl   5D Mark III, 6D, 550D

N/A

  • Hero Member
  • *****
  • Posts: 576
  • Dreaming in 14 bit
Re: [experiment] [5D3] mlv_play speedup
« Reply #22 on: September 21, 2014, 03:32:31 AM »
Any more development for this? Would be interested to see if the 7d's dual digics could be used to speed up playback.
7D. 600D. Rokinon 35 cine. Sigma 30 1.4
Audio and video recording/production, Random Photography
Want to help with the latest development but don't know how to compile?

Greg

  • Contributor
  • Hero Member
  • *****
  • Posts: 580
Re: [experiment] [5D3] mlv_play speedup
« Reply #23 on: September 21, 2014, 04:03:35 PM »
500D does not have ProcessPathForFurikake
550D has, so newer cameras also should

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: [experiment] [5D3] mlv_play speedup
« Reply #24 on: September 21, 2014, 07:53:14 PM »
6D 113 FF5ABF84 ProcessPathForFurikake
7D 203 FF3A325C ProcessPathForFurikake 

6D module loaded, 5x broken, 10x ok. DNG playback looks like 5x. Sometimes crashes after playing a file. Playback def faster.
7D module loaded, 5x ok, 10x ok, recording ok. Playback a bit faster. Taking a full silent pic output is black + hang, playback is black + hang.

Live previews do not work on either camera, neither color nor BW. I'm guessing MLV preview set to auto was entering this mode for 5X.

Something to do with defect correction processing? Maybe thats why zoom is broken? What else can it do?