Squeezing the last bit of performance out of MLV Lite (for testers)

[a1ex]

Result from last night (1920x1080, 30 fps, forgot global draw on at default settings):

Code: [Select]

Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
728,1163,1073,1172,1059,983,682,639
458,1091,1132,1145,1135,1074,845,740
328,1044,1116,1045,1136,1137,944,1017
1011,1134,1048,1153,819,1162,1139,747
899,1187,1162,1122,1172,1179,931,652
827,1136,1127,1000,1094,1093,1124,972
660,1064,1050,1062,1054,1044,931,891

Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
853,1067,1056,1045,1073,1063,1090,778
445,1060,1085,1066,1072,874,907,976
1043,1069,1037,1066,1047,1092,926,825
876,1087,852,963,1050,960,1083,1073
1036,1076,1037,1004,878,1084,874,986
1044,1088,1072,972,1052,939,842,1049
954,978,1047,967,1020,1019,1019,945
527,987,909,991,997,832
699,915,1007,982,909,883,910,949

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
689,872,864,774,776,794,646,793
786,810,873,730,771,797,781,883

Three problems:
- the MLV version appears slower (or maybe my script is not handling the MLVs properly; investigating)
- the random number generator doesn't seem very good (it picked MLV only two times)
- the script stopped recording for some reason after about 3 hours (I suspect overheating)

[Frank7D]

A: 32M-RAW.MO
Count = 56
Mean frames = 992

B: UNIFIED.MO
Count = 70
Mean frames = 972

C: 32M-MLV.MO
Count = 16
Mean frames = 790

A vs. B
T -Stat: 0.70
Critical T: 1.66

A vs. C
T -Stat: 4.11
Critical T: 1.70

B vs. C
T -Stat: 5.80
Critical T: 1.71

So A and B both beat C (32M-RAW.MO and UNIFIED.MO beat 32M-MLV.MO)

[a1ex]

Partial results after some small tweaks to the script:

rawbench.lua (history)

5D3, 1920x1080, 30 fps, global draw off, builds from reply #42 vs current unified:

Code: [Select]

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 08:26:24
===============================================================================

[08:26:30] Recording test clip 1 of 8...
[08:27:10] Recording stopped.
[08:27:30] Recording test clip 2 of 8...
[08:28:21] Recording stopped.
[08:28:41] Recording test clip 3 of 8...
[08:29:32] Recording stopped.
[08:29:52] Recording test clip 4 of 8...
[08:30:43] Recording stopped.
[08:31:03] Recording test clip 5 of 8...
[08:31:53] Recording stopped.
[08:32:13] Recording test clip 6 of 8...
[08:32:59] Recording stopped.
[08:33:19] Recording test clip 7 of 8...
[08:34:00] Recording stopped.
[08:34:20] Recording test clip 8 of 8...
[08:34:36] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 08:36:28
===============================================================================

A:/DCIM/100EOS5D/M20-0826.RAW: 1065
A:/DCIM/100EOS5D/M20-0827.R00: 1360
A:/DCIM/100EOS5D/M20-0828.R00: 1273
A:/DCIM/100EOS5D/M20-0829.R00: 1260
A:/DCIM/100EOS5D/M20-0831.R00: 1352
A:/DCIM/100EOS5D/M20-0832.R00: 1216
A:/DCIM/100EOS5D/M20-0833.RAW: 1091
A:/DCIM/100EOS5D/M20-0834.RAW: 310
Recorded frames : 1065,1360,1273,1260,1352,1216,1091,310
Quartile summary: 1238 frames (1078...1312.5)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 08:37:57
===============================================================================

[08:38:02] Recording test clip 1 of 8...
[08:38:32] Recording stopped.
[08:38:52] Recording test clip 2 of 8...
[08:39:43] Recording stopped.
[08:40:03] Recording test clip 3 of 8...
[08:40:54] Recording stopped.
[08:41:14] Recording test clip 4 of 8...
[08:41:50] Recording stopped.
[08:42:10] Recording test clip 5 of 8...
[08:42:56] Recording stopped.
[08:43:16] Recording test clip 6 of 8...
[08:44:01] Recording stopped.
[08:44:21] Recording test clip 7 of 8...
[08:45:02] Recording stopped.
[08:45:22] Recording test clip 8 of 8...
[08:45:53] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 08:47:45
===============================================================================

A:/DCIM/100EOS5D/M20-0838.RAW: 657
A:/DCIM/100EOS5D/M20-0839.R00: 1268
A:/DCIM/100EOS5D/M20-0840.R00: 1261
A:/DCIM/100EOS5D/M20-0841.RAW: 817
A:/DCIM/100EOS5D/M20-0842.R00: 1186
A:/DCIM/100EOS5D/M20-0843.RAW: 1160
A:/DCIM/100EOS5D/M20-0844.RAW: 989
A:/DCIM/100EOS5D/M20-0845.RAW: 745
Recorded frames : 657,1268,1261,817,1186,1160,989,745
Quartile summary: 1074.5 frames (781...1223.5)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 08:49:14
===============================================================================

[08:49:19] Recording test clip 1 of 8...
[08:49:55] Recording stopped.
[08:50:15] Recording test clip 2 of 8...
[08:50:56] Recording stopped.
[08:51:16] Recording test clip 3 of 8...
[08:51:52] Recording stopped.
[08:52:12] Recording test clip 4 of 8...
[08:52:47] Recording stopped.
[08:53:07] Recording test clip 5 of 8...
[08:53:43] Recording stopped.
[08:54:03] Recording test clip 6 of 8...
[08:54:44] Recording stopped.
[08:55:04] Recording test clip 7 of 8...
[08:55:45] Recording stopped.
[08:56:05] Recording test clip 8 of 8...
[08:56:35] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 08:58:27
===============================================================================

A:/DCIM/100EOS5D/M20-0849.MLV: 916
A:/DCIM/100EOS5D/M20-0850.MLV: 1046
A:/DCIM/100EOS5D/M20-0851.MLV: 812
A:/DCIM/100EOS5D/M20-0852.MLV: 802
A:/DCIM/100EOS5D/M20-0853.MLV: 878
A:/DCIM/100EOS5D/M20-0854.MLV: 1028
A:/DCIM/100EOS5D/M20-0855.MLV: 1055
A:/DCIM/100EOS5D/M20-0856.MLV: 784
Recorded frames : 916,1046,812,802,878,1028,1055,784
Quartile summary: 897 frames (807...1037)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 09:00:11
===============================================================================

[09:00:16] Recording test clip 1 of 8...
[09:00:57] Recording stopped.
[09:01:17] Recording test clip 2 of 8...
[09:01:57] Recording stopped.
[09:02:17] Recording test clip 3 of 8...
[09:02:58] Recording stopped.
[09:03:18] Recording test clip 4 of 8...
[09:04:09] Recording stopped.
[09:04:29] Recording test clip 5 of 8...
[09:05:20] Recording stopped.
[09:05:40] Recording test clip 6 of 8...
[09:06:30] Recording stopped.
[09:06:50] Recording test clip 7 of 8...
[09:07:26] Recording stopped.
[09:07:46] Recording test clip 8 of 8...
[09:08:17] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 09:10:09
===============================================================================

A:/DCIM/100EOS5D/M20-0900.RAW: 1087
A:/DCIM/100EOS5D/M20-0901.RAW: 1030
A:/DCIM/100EOS5D/M20-0902.RAW: 1069
A:/DCIM/100EOS5D/M20-0903.R00: 1317
A:/DCIM/100EOS5D/M20-0904.R00: 1260
A:/DCIM/100EOS5D/M20-0905.R00: 1313
A:/DCIM/100EOS5D/M20-0906.RAW: 905
A:/DCIM/100EOS5D/M20-0907.RAW: 682
Recorded frames : 1087,1030,1069,1317,1260,1313,905,682
Quartile summary: 1078 frames (967.5...1286.5)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 09:11:38
===============================================================================

[09:11:43] Recording test clip 1 of 8...
[09:12:04] Recording stopped.
[09:12:24] Recording test clip 2 of 8...
[09:13:09] Recording stopped.
[09:13:29] Recording test clip 3 of 8...
[09:14:20] Recording stopped.
[09:14:40] Recording test clip 4 of 8...
[09:15:31] Recording stopped.
[09:15:51] Recording test clip 5 of 8...
[09:16:27] Recording stopped.
[09:16:47] Recording test clip 6 of 8...
[09:17:33] Recording stopped.
[09:17:53] Recording test clip 7 of 8...
[09:18:38] Recording stopped.
[09:18:58] Recording test clip 8 of 8...
[09:19:39] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 09:21:31
===============================================================================

A:/DCIM/100EOS5D/M20-0911.RAW: 476
A:/DCIM/100EOS5D/M20-0912.R00: 1188
A:/DCIM/100EOS5D/M20-0913.R00: 1340
A:/DCIM/100EOS5D/M20-0914.R00: 1355
A:/DCIM/100EOS5D/M20-0915.RAW: 805
A:/DCIM/100EOS5D/M20-0916.R00: 1205
A:/DCIM/100EOS5D/M20-0917.RAW: 1095
A:/DCIM/100EOS5D/M20-0919.RAW: 1071
Recorded frames : 476,1188,1340,1355,805,1205,1095,1071
Quartile summary: 1141.5 frames (938...1272.5)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 09:23:00
===============================================================================

[09:23:05] Recording test clip 1 of 8...
[09:23:26] Recording stopped.
[09:23:46] Recording test clip 2 of 8...
[09:24:27] Recording stopped.
[09:24:47] Recording test clip 3 of 8...
[09:25:23] Recording stopped.
[09:25:43] Recording test clip 4 of 8...
[09:26:09] Recording stopped.
[09:26:29] Recording test clip 5 of 8...
[09:27:09] Recording stopped.
[09:27:29] Recording test clip 6 of 8...
[09:28:00] Recording stopped.
[09:28:20] Recording test clip 7 of 8...
[09:29:01] Recording stopped.
[09:29:21] Recording test clip 8 of 8...
[09:30:02] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 09:31:54
===============================================================================

A:/DCIM/100EOS5D/M20-0923.MLV: 481
A:/DCIM/100EOS5D/M20-0924.MLV: 1036
A:/DCIM/100EOS5D/M20-0925.MLV: 936
A:/DCIM/100EOS5D/M20-0926.MLV: 541
A:/DCIM/100EOS5D/M20-0927.MLV: 979
A:/DCIM/100EOS5D/M20-0928.MLV: 685
A:/DCIM/100EOS5D/M20-0929.MLV: 954
A:/DCIM/100EOS5D/M20-0930.MLV: 952
Recorded frames : 481,1036,936,541,979,685,954,952
Quartile summary: 944 frames (613...966.5)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 09:33:36
===============================================================================

[09:33:41] Recording test clip 1 of 8...
[09:34:17] Recording stopped.
[09:34:37] Recording test clip 2 of 8...
[09:35:18] Recording stopped.
[09:35:38] Recording test clip 3 of 8...
[09:36:13] Recording stopped.
[09:36:34] Recording test clip 4 of 8...
[09:37:09] Recording stopped.
[09:37:29] Recording test clip 5 of 8...
[09:38:10] Recording stopped.
[09:38:30] Recording test clip 6 of 8...
[09:39:11] Recording stopped.
[09:39:31] Recording test clip 7 of 8...
[09:40:12] Recording stopped.
[09:40:32] Recording test clip 8 of 8...
[09:41:07] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 09:43:00
===============================================================================

A:/DCIM/100EOS5D/M20-0933.MLV: 908
A:/DCIM/100EOS5D/M20-0934.MLV: 1034
A:/DCIM/100EOS5D/M20-0935.MLV: 798
A:/DCIM/100EOS5D/M20-0936.MLV: 938
A:/DCIM/100EOS5D/M20-0937.MLV: 954
A:/DCIM/100EOS5D/M20-0938.MLV: 1034
A:/DCIM/100EOS5D/M20-0939.MLV: 952
A:/DCIM/100EOS5D/M20-0940.MLV: 898
Recorded frames : 908,1034,798,938,954,1034,952,898
Quartile summary: 945 frames (903...994)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO

So, unless there's a problem with the script analyzing the MLVs (which I'll double-check shortly), it looks like something is really slowing it down. Figure out what...

[DeafEyeJedi]

Still trying to get the script to run on 5D3 but here's what I was able to accomplish from the past few nights of practice runs.

5D3, 1920x1080p, 30p global draw ON with builds from reply #42 vs the current nightly (also did 60p @ 1728x584p just for kicks).

Unified (nightly):
30p = 31s
60p = 10s

e78c7b8:
30p = 35s
60p = 11s

30bb06a:
30p = 27-28s (probably due to overheat? will test some more)
60p = 8-10s

I also noticed you added the rawbench.lua file on your recent post ... downloaded and will place it into script folder within ML and hopefully it'll run it overnight without overheating it? Ha.

*edit*

I just started to run the script on 5D3 or at least thought so but I keep getting this error saying "unexpected symbol near '<\5>' ... any idea what I could be doing wrong?

Thanks, @a1ex!

[a1ex]

To run the script, compile lua_fix and run the tests on top of that branch.

Under ML/MODULES/RAW_REC you can place binaries from other branches, so the script will pick them for testing.

[a1ex]

Another test: I've aligned the MLV header at 512 bytes.

5D3, 1920x1080, 30 fps, global draw on. 32M-MLV.MO is the one from reply #42 (30bb06a), and HDR512.MO is the one with header aligned at 512 bytes (b658401).

Filtered log, with grep -E "(Recorded|version)":

Code: [Select]

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 632,475,772,726,659,702,642,754
Raw_rec version : ML/MODULES/RAW_REC/HDR512.MO
Recorded frames : 793,1019,976,944,1114,1017,714,577
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 688,724,559,786,863,786,785,867
Raw_rec version : ML/MODULES/RAW_REC/HDR512.MO
Recorded frames : 1032,1053,927,1139,939,1055,754,659
Raw_rec version : ML/MODULES/RAW_REC/HDR512.MO
Recorded frames : 689,890,921,802,1101,1049,918,1133
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 422,721,762,778,879,869,706,868
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 797,780,847,679,577,847,768,799
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 629,793,770,754,751,845,783,835

Full log:

Code: [Select]

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 12:15:20
===============================================================================

[12:15:25] Recording test clip 1 of 8...
[12:15:51] Recording stopped.
[12:16:11] Recording test clip 2 of 8...
[12:16:32] Recording stopped.
[12:16:52] Recording test clip 3 of 8...
[12:17:23] Recording stopped.
[12:17:43] Recording test clip 4 of 8...
[12:18:14] Recording stopped.
[12:18:34] Recording test clip 5 of 8...
[12:19:05] Recording stopped.
[12:19:25] Recording test clip 6 of 8...
[12:19:56] Recording stopped.
[12:20:16] Recording test clip 7 of 8...
[12:20:42] Recording stopped.
[12:21:02] Recording test clip 8 of 8...
[12:21:32] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 12:23:29
===============================================================================

A:/DCIM/100EOS5D/M20-1215.MLV:  632 frames, 2.136047 GB
A:/DCIM/100EOS5D/M20-1216.MLV:  475 frames, 1.605415 GB
A:/DCIM/100EOS5D/M20-1217.MLV:  772 frames, 2.609222 GB
A:/DCIM/100EOS5D/M20-1218.MLV:  726 frames, 2.453751 GB
A:/DCIM/100EOS5D/M20-1219.MLV:  659 frames, 2.227303 GB
A:/DCIM/100EOS5D/M20-1220.MLV:  702 frames, 2.372635 GB
A:/DCIM/100EOS5D/M20-1221.MLV:  642 frames, 2.169846 GB
A:/DCIM/100EOS5D/M20-1222.MLV:  754 frames, 2.548386 GB
Recorded frames : 632,475,772,726,659,702,642,754
Quartile summary: 680.5 frames (637...740)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/HDR512.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 12:25:14
===============================================================================

[12:25:19] Recording test clip 1 of 8...
[12:25:50] Recording stopped.
[12:26:10] Recording test clip 2 of 8...
[12:26:51] Recording stopped.
[12:27:11] Recording test clip 3 of 8...
[12:27:52] Recording stopped.
[12:28:12] Recording test clip 4 of 8...
[12:28:48] Recording stopped.
[12:29:08] Recording test clip 5 of 8...
[12:29:53] Recording stopped.
[12:30:14] Recording test clip 6 of 8...
[12:30:54] Recording stopped.
[12:31:14] Recording test clip 7 of 8...
[12:31:45] Recording stopped.
[12:32:05] Recording test clip 8 of 8...
[12:32:31] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 12:34:28
===============================================================================

A:/DCIM/100EOS5D/M20-1225.MLV:  793 frames, 2.680199 GB
A:/DCIM/100EOS5D/M20-1226.MLV: 1019 frames, 3.444039 GB
A:/DCIM/100EOS5D/M20-1227.MLV:  976 frames, 3.298707 GB
A:/DCIM/100EOS5D/M20-1228.MLV:  944 frames, 3.190552 GB
A:/DCIM/100EOS5D/M20-1229.MLV: 1114 frames, 3.765122 GB
A:/DCIM/100EOS5D/M20-1230.MLV: 1017 frames, 3.437279 GB
A:/DCIM/100EOS5D/M20-1231.MLV:  714 frames, 2.413193 GB
A:/DCIM/100EOS5D/M20-1232.MLV:  577 frames, 1.950158 GB
Recorded frames : 793,1019,976,944,1114,1017,714,577
Quartile summary: 960 frames (753.5...1018)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 12:36:12
===============================================================================

[12:36:17] Recording test clip 1 of 8...
[12:36:48] Recording stopped.
[12:37:08] Recording test clip 2 of 8...
[12:37:39] Recording stopped.
[12:37:59] Recording test clip 3 of 8...
[12:38:25] Recording stopped.
[12:38:45] Recording test clip 4 of 8...
[12:39:16] Recording stopped.
[12:39:36] Recording test clip 5 of 8...
[12:40:11] Recording stopped.
[12:40:32] Recording test clip 6 of 8...
[12:41:02] Recording stopped.
[12:41:23] Recording test clip 7 of 8...
[12:41:54] Recording stopped.
[12:42:14] Recording test clip 8 of 8...
[12:42:49] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 12:44:46
===============================================================================

A:/DCIM/100EOS5D/M20-1236.MLV:  688 frames, 2.325317 GB
A:/DCIM/100EOS5D/M20-1237.MLV:  724 frames, 2.446991 GB
A:/DCIM/100EOS5D/M20-1238.MLV:  559 frames, 1.88932 GB
A:/DCIM/100EOS5D/M20-1239.MLV:  786 frames, 2.65654 GB
A:/DCIM/100EOS5D/M20-1240.MLV:  863 frames, 2.916786 GB
A:/DCIM/100EOS5D/M20-1241.MLV:  786 frames, 2.65654 GB
A:/DCIM/100EOS5D/M20-1242.MLV:  785 frames, 2.65316 GB
A:/DCIM/100EOS5D/M20-1243.MLV:  867 frames, 2.930305 GB
Recorded frames : 688,724,559,786,863,786,785,867
Quartile summary: 785.5 frames (706...824.5)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/HDR512.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 12:46:32
===============================================================================

[12:46:38] Recording test clip 1 of 8...
[12:47:19] Recording stopped.
[12:47:39] Recording test clip 2 of 8...
[12:48:19] Recording stopped.
[12:48:40] Recording test clip 3 of 8...
[12:49:15] Recording stopped.
[12:49:35] Recording test clip 4 of 8...
[12:50:21] Recording stopped.
[12:50:41] Recording test clip 5 of 8...
[12:51:17] Recording stopped.
[12:51:37] Recording test clip 6 of 8...
[12:52:18] Recording stopped.
[12:52:38] Recording test clip 7 of 8...
[12:53:09] Recording stopped.
[12:53:29] Recording test clip 8 of 8...
[12:54:00] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 12:55:56
===============================================================================

A:/DCIM/100EOS5D/M20-1246.MLV: 1032 frames, 3.487977 GB
A:/DCIM/100EOS5D/M20-1247.MLV: 1053 frames, 3.558953 GB
A:/DCIM/100EOS5D/M20-1248.MLV:  927 frames, 3.133095 GB
A:/DCIM/100EOS5D/M20-1249.MLV: 1139 frames, 3.849617 GB
A:/DCIM/100EOS5D/M20-1250.MLV:  939 frames, 3.173653 GB
A:/DCIM/100EOS5D/M20-1251.MLV: 1055 frames, 3.565712 GB
A:/DCIM/100EOS5D/M20-1252.MLV:  754 frames, 2.548386 GB
A:/DCIM/100EOS5D/M20-1253.MLV:  659 frames, 2.227303 GB
Recorded frames : 1032,1053,927,1139,939,1055,754,659
Quartile summary: 985.5 frames (840.5...1054)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/HDR512.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 12:57:41
===============================================================================

[12:57:46] Recording test clip 1 of 8...
[12:58:17] Recording stopped.
[12:58:37] Recording test clip 2 of 8...
[12:59:13] Recording stopped.
[12:59:33] Recording test clip 3 of 8...
[13:00:09] Recording stopped.
[13:00:29] Recording test clip 4 of 8...
[13:01:05] Recording stopped.
[13:01:25] Recording test clip 5 of 8...
[13:02:10] Recording stopped.
[13:02:30] Recording test clip 6 of 8...
[13:03:11] Recording stopped.
[13:03:31] Recording test clip 7 of 8...
[13:04:07] Recording stopped.
[13:04:27] Recording test clip 8 of 8...
[13:05:13] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 13:07:10
===============================================================================

A:/DCIM/100EOS5D/M20-1257.MLV:  689 frames, 2.328698 GB
A:/DCIM/100EOS5D/M20-1258.MLV:  890 frames, 3.008042 GB
A:/DCIM/100EOS5D/M20-1259.MLV:  921 frames, 3.112816 GB
A:/DCIM/100EOS5D/M20-1300.MLV:  802 frames, 2.710618 GB
A:/DCIM/100EOS5D/M20-1301.MLV: 1101 frames, 3.721184 GB
A:/DCIM/100EOS5D/M20-1302.MLV: 1049 frames, 3.545433 GB
A:/DCIM/100EOS5D/M20-1303.MLV:  918 frames, 3.102677 GB
A:/DCIM/100EOS5D/M20-1304.MLV: 1133 frames, 3.829339 GB
Recorded frames : 689,890,921,802,1101,1049,918,1133
Quartile summary: 919.5 frames (846...1075)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 13:08:54
===============================================================================

[13:08:59] Recording test clip 1 of 8...
[13:09:20] Recording stopped.
[13:09:40] Recording test clip 2 of 8...
[13:10:11] Recording stopped.
[13:10:31] Recording test clip 3 of 8...
[13:11:02] Recording stopped.
[13:11:22] Recording test clip 4 of 8...
[13:11:53] Recording stopped.
[13:12:13] Recording test clip 5 of 8...
[13:12:49] Recording stopped.
[13:13:09] Recording test clip 6 of 8...
[13:13:45] Recording stopped.
[13:14:05] Recording test clip 7 of 8...
[13:14:36] Recording stopped.
[13:14:56] Recording test clip 8 of 8...
[13:15:32] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 13:17:28
===============================================================================

A:/DCIM/100EOS5D/M20-1309.MLV:  422 frames, 1.426285 GB
A:/DCIM/100EOS5D/M20-1310.MLV:  721 frames, 2.436851 GB
A:/DCIM/100EOS5D/M20-1311.MLV:  762 frames, 2.575424 GB
A:/DCIM/100EOS5D/M20-1312.MLV:  778 frames, 2.629501 GB
A:/DCIM/100EOS5D/M20-1313.MLV:  879 frames, 2.970863 GB
A:/DCIM/100EOS5D/M20-1314.MLV:  869 frames, 2.937065 GB
A:/DCIM/100EOS5D/M20-1315.MLV:  706 frames, 2.386154 GB
A:/DCIM/100EOS5D/M20-1316.MLV:  868 frames, 2.933685 GB
Recorded frames : 422,721,762,778,879,869,706,868
Quartile summary: 770 frames (713.5...868.5)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 13:19:15
===============================================================================

[13:19:20] Recording test clip 1 of 8...
[13:19:51] Recording stopped.
[13:20:11] Recording test clip 2 of 8...
[13:20:42] Recording stopped.
[13:21:02] Recording test clip 3 of 8...
[13:21:38] Recording stopped.
[13:21:58] Recording test clip 4 of 8...
[13:22:29] Recording stopped.
[13:22:49] Recording test clip 5 of 8...
[13:23:15] Recording stopped.
[13:23:35] Recording test clip 6 of 8...
[13:24:11] Recording stopped.
[13:24:31] Recording test clip 7 of 8...
[13:25:02] Recording stopped.
[13:25:22] Recording test clip 8 of 8...
[13:25:53] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 13:27:49
===============================================================================

A:/DCIM/100EOS5D/M20-1319.MLV:  797 frames, 2.693718 GB
A:/DCIM/100EOS5D/M20-1320.MLV:  780 frames, 2.636261 GB
A:/DCIM/100EOS5D/M20-1321.MLV:  847 frames, 2.862709 GB
A:/DCIM/100EOS5D/M20-1322.MLV:  679 frames, 2.294899 GB
A:/DCIM/100EOS5D/M20-1323.MLV:  577 frames, 1.950157 GB
A:/DCIM/100EOS5D/M20-1324.MLV:  847 frames, 2.862709 GB
A:/DCIM/100EOS5D/M20-1325.MLV:  768 frames, 2.595703 GB
A:/DCIM/100EOS5D/M20-1326.MLV:  799 frames, 2.700478 GB
Recorded frames : 797,780,847,679,577,847,768,799
Quartile summary: 788.5 frames (723.5...823)

For next experiment
===================

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 13:29:36
===============================================================================

[13:29:41] Recording test clip 1 of 8...
[13:30:07] Recording stopped.
[13:30:27] Recording test clip 2 of 8...
[13:30:58] Recording stopped.
[13:31:18] Recording test clip 3 of 8...
[13:31:49] Recording stopped.
[13:32:09] Recording test clip 4 of 8...
[13:32:40] Recording stopped.
[13:33:00] Recording test clip 5 of 8...
[13:33:31] Recording stopped.
[13:33:51] Recording test clip 6 of 8...
[13:34:27] Recording stopped.
[13:34:47] Recording test clip 7 of 8...
[13:35:18] Recording stopped.
[13:35:38] Recording test clip 8 of 8...
[13:36:14] Recording stopped.

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-20 13:38:10
===============================================================================

A:/DCIM/100EOS5D/M20-1329.MLV:  629 frames, 2.125908 GB
A:/DCIM/100EOS5D/M20-1330.MLV:  793 frames, 2.680199 GB
A:/DCIM/100EOS5D/M20-1331.MLV:  770 frames, 2.602463 GB
A:/DCIM/100EOS5D/M20-1332.MLV:  754 frames, 2.548386 GB
A:/DCIM/100EOS5D/M20-1333.MLV:  751 frames, 2.538246 GB
A:/DCIM/100EOS5D/M20-1334.MLV:  845 frames, 2.855949 GB
A:/DCIM/100EOS5D/M20-1335.MLV:  783 frames, 2.6464 GB
A:/DCIM/100EOS5D/M20-1336.MLV:  835 frames, 2.822151 GB
Recorded frames : 629,793,770,754,751,845,783,835
Quartile summary: 776.5 frames (752.5...814)


[Frank7D]

For Reply #55:

A: 32M-MLV.MO
Count = 40
Mean frames = 742

B: HDR512.MO
Count = 24
Mean frames = 926

A vs. B
T -Stat: 5.54
Critical T: 1.68

B beat A (HDR512.MO beat 32M-MLV.MO)

[a1ex]

I've merged lua_fix, so you should be able to run the automatic tests with the next nightly.

How to use:

* highly recommended: a power adapter, so you can leave the camera overnight running the test
* set up your raw recording preferences
* review rawbench.lua (don't run code from strangers without understanding what it does)
* save rawbench.lua into ML/SCRIPTS/ on your card
* save stats.lua into ML/SCRIPTS/LIB/
* create ML/MODULES/RAW_REC and put there any raw_rec.mo files you want to test (any recent versions would work), either old raw or mlv_lite, renaming them as needed (8.3 filenames)
* mlv_rec.mo binaries might work as well (not tested)
* start the camera in movie mode and follow the instructions on the script console
* when you think it's enough, stop the camera and look at ML/LOGS/RAWBENCH.LOG


To test old versions that can't run on this code base (such as reddeercity's "gold standard" raw_rec, which doesn't run on latest core), you can use this trick:
* place the old ML build on the card
* record a few clips manually
* place the latest nightly on the card, including this script
* start the camera outside LiveView; the script will analyze the clips (count the number of frames), but it will not record anything
* check ML/LOGS/RAWBENCH.LOG
* format the card, keep ML (it will also keep the log)

As you can see, the analysis happens on camera; you don't have to run anything on the PC.


But if you don't mind processing the files on the PC, here's an alternative workflow for manual testing:
* place the old ML build on the card
* record a few clips manually
* place the card in your card reader
* run these scripts on your DCIM directory to analyze the videos (Linux/Mac; on Windows it might work with Cygwin or msys, not tested)
* copy the output from the console


You'll probably want to test the latest MLV Lite (raw_rec.mo) against the "classic" raw_rec from reply #42, and against the one from unified.

So, the questions that should be answered by testing are basically the same:

- are the speeds identical in latest MLV Lite vs latest "classic" raw_rec?
- how do they compare (speed-wise) with the raw_rec from the nightly?
- any bugs in mlv_lite that are not present in the "classic" raw_rec? (I've found one with this script, and hopefully fixed it, but... who knows)

[a1ex]

New data set:

Code: [Select]

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 671,1045,995,725,771,704,772,564
Raw_rec version : ML/MODULES/RAW_REC/HDR512G.MO
Recorded frames : 710,1253,1455,1326,1430,898,691,536
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 451,1054,959,1045,1038,755,666,798
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
Recorded frames : 832,1315,1316,1308,1178,1260,708,528
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 1036,1049,983,1129,1114,1175,845,1066
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 806,997,1206,1368,1346,866,1066,731
Raw_rec version : ML/MODULES/RAW_REC/HDR512G.MO
Recorded frames : 924,1259,1389,1080,1094,961,874,899
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 560,1048,1036,719,478,742,1067,880
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 1042,1353,1193,1342,1353,1077,862,565
Raw_rec version : ML/MODULES/RAW_REC/HDR512G.MO
Recorded frames : 818,1096,637,1200,1403,1460,1384,542
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 458,694,862,1282,1374,1224,735,624
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 660,1046,1021,1045,1050,888,771,952
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 586,1361,1346,1360,976,1293,1218,603
Raw_rec version : ML/MODULES/RAW_REC/HDR512G.MO
Recorded frames : 630,1257,849,884,956,1260,1255,850
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 747,577,753,1043,1064,954,1040,929
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 805,695,952,1040,1034,1024,1037,962
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 1039,1286,1357,1359,1350,1274,770,455
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 356,1060,1211,1173,1284,967,968,915
Raw_rec version : ML/MODULES/RAW_REC/HDR512G.MO
Recorded frames : 940,1258,1077,1055,958,1427,1260,640
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
Recorded frames : 710,964,1004,928,1310,1151,799,1029
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 1126,679,1337,1351,1330,1343,734,881
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 305,1027,952,1031,798,594,869,637
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 1318,1335,1343,1272,1278,722,871,449
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 740,763,756,660,534,774,1042,772
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 746,921,1037,1036,1031,1030,929,734
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 817,1280,1337,1364,1257,1342,728,536
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
Recorded frames : 754,1297,1318,1046,833,1057,841,842
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
Recorded frames : 1013,1310,951,944,1329,1165,763,1070
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
Recorded frames : 412,1009,608,817,1192,1134,844,824
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
Recorded frames : 1056,1121,1298,1311,1315,852,1063,431
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
Recorded frames : 892,1260,1311,1038,699,918,1031,890
Raw_rec version : ML/MODULES/RAW_REC/HDR512G.MO
Recorded frames : 819,1256,1424,607,1211,1394,1240,633
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 683,1336,986,726,1110,1335,1328,743
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 1186,1086,965,1355,1332,1327,1078,569
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 300,956,726,866,867,947,563,800
Raw_rec version : ML/MODULES/RAW_REC/HDR512G.MO
Recorded frames : 1031,1269,1395,1399,1383,848,505,506
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 1026,1356,1255,1358,848,1193,642,736
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 323,1032,1030,1028,665,815,635,860
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
Recorded frames : 1244,1307,1041,1295,1260,1264,821,378
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
Recorded frames : 439,1064,1009,1063,1038,1260,1232,608
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
Recorded frames : 754,562,1250,1296,1311,1296,1132,827
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 333,778,1048,1029,1037,881,752,1036
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO
Recorded frames : 1036,1346,1348,1273,1285,1278,875,408
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 340,868,546,604,763,688,758,947

Good news: the script did no longer crash, and there were no errors in the log

I also wrote a quick analysis script: analyze-dataset.py

Results:

Code: [Select]

Quartile stats: median (Q1 ... Q3)
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO:  863 frames (722 ... 1030)
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO: 1040 frames (833 ... 1260)
Raw_rec version : ML/MODULES/RAW_REC/HDR512G.MO: 1079 frames (849 ... 1265)
Raw_rec version : ML/MODULES/RAW_REC/32M-RAW.MO: 1120 frames (855 ... 1331)

P =

           1  1.7114e-07  8.1831e-08  5.2004e-11
  1.7114e-07           1     0.42529     0.21991
  8.1831e-08     0.42529           1     0.81414
  5.2004e-11     0.21991     0.81414           1

T =

           0     -5.4388     -5.6262     -6.9187
      5.4388           0    -0.79972     -1.2308
      5.6262     0.79972           0    -0.23547
      6.9187      1.2308     0.23547           0

The versions are:
hdr512g: 2ab79de
32m-mlv/raw: binaries from reply #42 (30bb06a/e78c7b8)
unified: 4eacd1f

[Ottoga]

@A1ex

Having mixed success with the rawbech.lua script. Feedback can be found here: https://www.dropbox.com/s/q5s1whs1vsqwph7/Rawbench_Testing.7z?dl=0

[a1ex]

stdev too large

This is from the raw backend trying to compute the black level (it may also help diagnosing pink frames); you can ignore it for this test.

Looking at the log files generated, I can't tell which raw module was executing

This happens right after formatting the card, before restarting. I don't see it in the log file though, but I pushed a possible fix.

I also didn’t see the rawbench.lua script under any menu

The menu code is commented out; the script now runs unattended.

However, assuming that it is working correctly, for this script to provide you with any meaningful feedback I think that it needs to limit the size of the size of the clips that it records. To say "2 minutes" before stopping and moving to the next iteration.

The script assumes you have picked recording settings that do not result in very long clips. But, indeed, I can see why a timeout can be useful, so I've hardcoded a 1-minute timeout.

I think that the rawbench or stats scripts are not correctly handling instances when there is just a single raw clip in the DCIM folder on the card (even if it is a multi chunk one).

True, but I don't see it as a problem, since you need many test clips to get a relevant result. I just delete those lines from the logs.

Thanks for the detailed feedback.

[Ottoga]

@A1ex

The script assumes you have picked recording settings that do not result in very long clips. But, indeed, I can see why a timeout can be useful, so I've hardcoded a 1-minute timeout.

At the minute mark the error in the linked screenshot was displayed and the camera continued recording.

Screenshot:  https://www.dropbox.com/s/3d119euqnr2dz0i/20160421_184655.jpg?dl=0

Sorry, its a bit blurry

Which raw_rec module is running?
Quote

Looking at the log files generated, I can't tell which raw module was executing



This happens right after formatting the card, before restarting. I don't see it in the log file though, but I pushed a possible fix.

Nothing in the log file.

[Edit] The log Contents

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-21 19:37:58
===============================================================================

[19:38:03] Recording test clip 1 of 8...

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-21 19:42:31
===============================================================================

A:/DCIM/100EOS7D/M21-1938.M02: 4654 frames, 12.745171 GB
ML/SCRIPTS/LIB/stats.lua:29: attempt to perform arithmetic on a nil value (field '?')
stack traceback:
   ML/SCRIPTS/LIB/stats.lua:29: in function 'globals.getMedian'
   ML/SCRIPTS/LIB/stats.lua:69: in function 'stats.quartiles'
   ML/SCRIPTS/RAWBENCH.LUA:216: in function 'globals.check_files'
   [C]: in function 'globals.xpcall'
   ML/SCRIPTS/RAWBENCH.LUA:322: in function 'globals.run_test'

===============================================================================
ML/SCRIPTS/RAWBENCH.LUA - 2016-4-21 19:43:59
===============================================================================

[19:44:04] Recording test clip 1 of 8...

[a1ex]

Stopping at 1 minute should be OK now.

The other... I'm not sure where to look. Do you see the raw_rec version messages on the screen (including "For next experiment"), right after formatting the card, but before restart?

[Ottoga]

Can't look at this again for a couple of hours.  Will test it again tonight though before hitting the sack.

[Ottoga]

@A1ex

Stopping at 1 minute should be OK now.

The other... I'm not sure where to look. Do you see the raw_rec version messages on the screen (including "For next experiment"), right after formatting the card, but before restart?

The 1 minute restart is working fine now thanks. I've also tried it with settings that won't record continuously and it is looping through that nicely as well.

Also, I am seeing the raw_rec version messages on screen. They don't stay for long, but they are there.

Thanks for your time and effort to get this script working for. I'll put a recharged battery in tomorrow and let it run until it dies. It should give a few cycles of good info for you.

[a1ex]

Another test worth trying is api_test.lua (it also tests file I/O, among other things). Might give some hints.

[Frank7D]

For reply #58:

A: 32M-MLV.MO
Count = 104
Mean frames = 829

B: HDR512G.MO
Count = 56
Mean frames = 1050

C: UNIFIED.MO
Count = 80
Mean frames = 1012

D: 32M-RAW.MO
Count = 112
Mean frames = 1061

A vs. B
T -Stat: 5.63
Critical T: 1.66

A vs. C
T -Stat: 5.44
Critical T: 1.66

A vs. D
T -Stat: 6.92
Critical T: 1.65

B vs. C
T -Stat: 0.80
Critical T: 1.66

B vs. D
T -Stat: 0.24
Critical T: 1.66

C vs. D
T -Stat: 1.23
Critical T: 1.65

B, C and D are tied (means are not significantly different).
B, C and D beat A (32M-MLV.MO)

EDITED to correct some numbers (the spreadsheet is a work in progress) but it didn't change the bottom line.

[Ottoga]

@A1ex

Here are the test results after running rawbench.lua. I ran two series of tests, one with global draw on and the other with it off. I transposed the log file details into a spreadsheet in a format that is bit more readable for me. I'm not sure if all the required log entries have been written into the log or not. As such, the original log files are supplied as well.

I also ran the api_test.lua through a couple of cycles as you suggested and have included the log file.

Test results can be found here:  https://www.dropbox.com/s/ohr3qlav4ozh810/Rawbench_Testing_7D.7z?dl=0

Let me know if you want me to try anything else.

[Frank7D]

For Reply #67

Code: [Select]

GDraw ON

A: raw_rec.mo
Count = 15
Mean frames = 1061

B: raw_lite.mo
Count = 10
Mean frames = 1118

C: 30bb06a.mo
Count = 20
Mean frames = 939

D: e78c7b8.mo
Count = 5
Mean frames = 926

E: Don't know
Count = 5
Mean frames = 920

A vs. B
T -Stat: 0.88
Critical T: 1.73

A vs. C
T -Stat: 1.55
Critical T: 1.70

A vs. D
T -Stat: 1.10
Critical T: 1.89

A vs. E
T -Stat: 1.21
Critical T: 1.89

B vs. C
T -Stat: 2.27
Critical T: 1.75

B vs. D
T -Stat: 1.90
Critical T: 1.83

B vs. E
T -Stat: 2.24
Critical T: 1.83

C vs. D
T -Stat: 0.10
Critical T: 1.94

C vs. E
T -Stat: 0.15
Critical T: 1.94

D vs. E
T -Stat: 0.03
Critical T: 1.89

B beat C, D and E

GDraw OFF

A: raw_rec.mo
Count = 15
Mean frames = 1826

B: raw_lite.mo
Count = 15
Mean frames = 1720

C: 30bb06a.mo
Count = 15
Mean frames = 1848

D: e78c7b8.mo
Count = 10
Mean frames = 1576

A vs. B
T -Stat: 0.68
Critical T: 1.70

A vs. C
T -Stat: 0.16
Critical T: 1.70

A vs. D
T -Stat: 1.14
Critical T: 1.72

B vs. C
T -Stat: 0.97
Critical T: 1.70

B vs. D
T -Stat: 0.68
Critical T: 1.72

C vs. D
T -Stat: 1.41
Critical T: 1.72

No winners
Bottom line:

Global Draw ON:
B beat C, D and E

Global Draw OFF:
No winners

If we're going to do much of this, would it be possible to get the data in a predictable format?
Perhaps something like a1ex provided in Reply #58, i.e.:

Code: [Select]

Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 671,1045,995,725,771,704,772,564
Raw_rec version : ML/MODULES/RAW_REC/HDR512G.MO
Recorded frames : 710,1253,1455,1326,1430,898,691,536
Raw_rec version : ML/MODULES/RAW_REC/32M-MLV.MO
Recorded frames : 451,1054,959,1045,1038,755,666,798
Raw_rec version : ML/MODULES/RAW_REC/UNIFIED.MO
Recorded frames : 832,1315,1316,1308,1178,1260,708,528

[Ottoga]

@Frank7D

Happy to provide the feedback in what ever the preferred format is.

I don't know what the following are/represent or how they are calculated. Do you have a sample spreadsheet that I could follow?

T -Stat: 0.97
Critical T: 1.70

The raw_rec version info has been manually captured while the script was running and added after the event as the logging is not capturing this information. I can add the path if it's important even though the script looks for them in a specific location.

[Frank7D]

Ottoga,

Here are the formulas I am using. Note that they do not show all the math since I am using some Excel functions ("DEVSQ" etc.).

Code: [Select]

Sum of the Squared Deviations
x=DEVSQ(Data Set A)+DEVSQ(Data Set B)

Pooled Sample Variance
y=x/([Count of A]-1+[Count of B]-1)

Standard Error of Difference in Means
z=SQRT(y*(1/[Count of A]+1/[Count of B]))

Our T -Stat
=([Mean of A]-[Mean of B])/z

U (used in DF calculation)
U=[Mean of B]^2/[Mean of A]^2

Degrees of Freedom
V=(1/[Count of A]+U/[Count of B])^2/(1/([Count of A]^2*([Count of A]-1))+U^2/([Count of B]^2*([Count of B]-1)))

Critical T Value @ 0.95 (95% confidence)
=TINV(0.05*2,V)

If the T -Stat is larger than the Critical T Value, then the difference in the means of the two data sets is significant.

If someone wants the spreadsheet, I can upload it.

As far as the data format, it doesn't matter much to me except that it would be nice if it were consistent.

EDIT: I should probably give the site where I got these formulas:
http://policeanalyst.com/performing-a-statistical-t-test-in-excel/

[Ottoga]

@Frank7D

Interesting article. Also, thanks for your analysis. I've updated my spreadsheet as well for future use.

[a1ex]

I think I know why the raw_rec version didn't appear in the log file. When formatting the card, I should have closed the log file and reopened it afterwards...

Maybe the reason it worked fine on 5D3 was... a Canon bug?!

[Ottoga]

@a1lex

I'll try it again tonight and let you know if it is now logging correctly.

btw: When the script says finish, have a little sleep.... My camera appears to go into preview mode of a clip (don't know which one).  After the sleep period the script resumes and continues as per normal. As a result though, the camera never really gets the intended break and chance to cool a little.

Here is a link to a video captured with my phone:  https://www.dropbox.com/s/880kd2t74rddk21/20160427_193655.mp4?dl=0

[a1ex]

Got it - I ran the script without mlv_play. So, when it went to preview mode on my camera, there was no activity.