Author Topic: New writing strategy - variable buffering  (Read 75918 times)

xNiNELiVES

  • Senior
  • ****
  • Posts: 461
  • 5D II, 17-35mm F2.8L, 50mm F1.4, 70-200 F4L
Re: New writing strategy - variable buffering
« Reply #100 on: June 22, 2013, 05:35:06 AM »
I meant to say that I started in video mode NOT photo mode.

I have the newest version of ML, with Alex's new findings..
I update it almost every day.

Yeah It seems with these other users that they are experiencing similar problems. Wait for a fix. Sorry.

budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: New writing strategy - variable buffering
« Reply #101 on: June 22, 2013, 06:46:28 AM »
¨The kaos is an order to resolve¨.

(See you in a couple of months).

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: New writing strategy - variable buffering
« Reply #102 on: June 22, 2013, 08:28:05 AM »
The don't click trick is only from Tragic Lantern and it's camera-specific. I don't consider it safe enough for normal builds, because it breaks Canon UI and you can't undo it without rebooting.

aaphotog

  • Senior
  • ****
  • Posts: 356
Re: New writing strategy - variable buffering
« Reply #103 on: June 22, 2013, 09:35:18 PM »
It slowed down for me on the newest build.
I don't have the option to have the preview set to 'HaCKed'
The "preview option is no longer there.
FPS over ride set to 23.976
I got over 10 seconds with the 3 files you posted(258 frames)
with the newest build that Lourenco posted(last commit: 363a968), I'm only getting 202 frames.

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: New writing strategy - variable buffering
« Reply #104 on: June 23, 2013, 12:41:33 AM »
It can be undone now, just have to close lv after unhacking.

Also for 50D at least they can chose timers from 1024-32768 and disabled... so I have to port this part.. on SD cameras the less aggressive hack may be enough.

I too think it got *slightly* slower and the quick ramp up is hit or miss in terms of speed.



ted ramasola

  • Moderators
  • Hero Member
  • *****
  • Posts: 1251
Performance gains with variable buffer and speculative start.
« Reply #105 on: June 25, 2013, 07:15:50 AM »
Performance gains with variable buffer and speculative start.
Using Lexar 1000x 32gb on 5d2. Each resolution was tested twice, results separated by ( / ).
Card is always formatted in camera before a new build is tested.
All tests done with: GD=off, preview=hacked, FPS over ride= 23.976

resolutions at 1x -- frames 1st take/2nd take

jun 19 by a.d.

1880 x 1058 -- 319/326
1880 x 1016 -- 532/425
1880 x 940  -- 779/988
1880 x 854  --continuous

alex variable buffer jun 21

1872 x 1054 -- 486/480
1872 x 1012 -- 755/732
1872 x 936  -- 2,391/2,272
1872 x 850  -- continuous
1856 x 1044 -- 521
1856 x 928  -- continuous

jun 22 by a.d. spec start off

1872 x 1054 -- 485/582
1872 x 1012 -- 932/876
1872 x 936  -- 4,085/3,626
1872 x 850  -- continuous
1856 x 1044 -- 644/563
1856 x 928  -- continuous

jun 22 by a.d. spec start ON

1872 x 1054 -- 504/531
1872 x 1012 -- 949/893
1872 x 936  -- 2,833/2,369
1872 x 850  -- continuous
1856 x 1044 -- 572/639
1856 x 928  -- continuous

jun 24 by a.d.

1872 x 1054 -- 572/506
1872 x 1012 -- 871/858
1872 x 936  -- 2,868/4,009
1872 x 850  -- continuous
1856 x 1044 -- 557/555
1856 x 928  -- continuous

CROP modes in the following builds;

jun 18-19-21-24 were all Unpredictable and always have pinkish cast in all frames.
I have the number of frames tabulated but since they're mostly pinkish I didn't bother posting.

last build with no pink cast in most resolutions was Jun 16 by a.d.
5DmkII  / 7D
www.ramasolaproductions.com
Texas

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: New writing strategy - variable buffering
« Reply #106 on: June 27, 2013, 01:21:57 PM »
Maybe you already noticed that latest builds already include an estimation of how many frames you are going to get with current settings.

That estimation is the ideal mathematical model: if the buffer filling speed is capture_speed - write_speed (say in bytes per second), and we know the memory buffer size, you are going to get X frames, where x = buffer_size / (capture_speed - write_speed). The exact formula is in predict_frames in raw_rec.c.

Of course, when write_speed >= capture_speed, you have continuous recording.

I think it's clear that you are not going to get more frames unless (a) you increase the write speed, or (b) you increase the memory buffer.

In my experiments, the numbers predicted with this model were pretty close to reality, which means the buffering strategy is close to optimal. I'd like you to check this hypothesis - e.g. run some test and check whether predicted and actual frame numbers are close.

If you notice that predicted numbers are way higher than actual numbers, this means ML is overestimating the card speed, so it's trying to use buffers that are too large. I need to know whether this happens or not, so I can fine-tune the prediction algorithm.



Here's the latest binary (for all cameras). I've added the dialog slowdown tricks from 1% (enabled for 5D3/5D2/50D only for now, feel free to find the address for the other cameras), so you can expect a little speed boost if you enable "small hacks" in menu.

raw_rec.mo

Bioskop.Inc

  • Member
  • ***
  • Posts: 109
Re: New writing strategy - variable buffering
« Reply #107 on: June 27, 2013, 03:08:51 PM »
Seems accurate on 60D, maybe 1 frame out (but that's down to there always being a skipped frame @ max res) & it changes its prediction as recording goes along.
I only tested it on full resolution, so 1728x....

1%

  • Developer
  • Hero Member
  • *****
  • Posts: 5936
  • 600D/6D/50D/EOSM/7D
Re: New writing strategy - variable buffering
« Reply #108 on: June 27, 2013, 04:09:25 PM »
50D can over/under estimate in the initial recordings.

How did it work on 5d3? Esp with timer changed before LV open?

KSphoto

  • New to the forum
  • *
  • Posts: 24
Re: New writing strategy - variable buffering
« Reply #109 on: June 27, 2013, 05:52:09 PM »
I'm getting close numbers with my 5D2 Lexar pro 600/32 card @ 1880x1058.

actual/predicted
259/263     GD on
244/244     GD off
344/352     GD on/Hack on
345/370     GD off/Hack on
5DC, 5D2, 5D3, EOS M, too many lenses.

Redrocks

  • Member
  • ***
  • Posts: 151
  • 5D2 Windows
Re: New writing strategy - variable buffering
« Reply #110 on: June 27, 2013, 06:50:02 PM »
1872 x 1250, 5d2, Komputerbay 64GB, small hacks on:

Getting 9/10 frames less than expected consistently, although the number of frames captured has increased from 216 - 229 with the first variable buffer module on the 22nd to 260 - 282 with this module.

xNiNELiVES

  • Senior
  • ****
  • Posts: 461
  • 5D II, 17-35mm F2.8L, 50mm F1.4, 70-200 F4L
Re: New writing strategy - variable buffering
« Reply #111 on: August 03, 2013, 12:13:12 AM »
Has variable buffering been optimized to it's fullest potential yet?