Messages

@a1ex - thanks for the reply. I'm back stacked with work for a few days but when I get a chance will look into these tests.

Just had a quick play with magiclantern-crop_rec_4k.2018Mar10.650D104 - on my 650D.

Looks great - it's amazing what you've managed to squeeze out of this camera.

One tiny caveat, the crop rec res in 720p & 1080p is 1920x1076. Is there a reason why we couldn't get 1080 vertical? dforts earlier builds managed a full 1920x1080.(i.e. crop_rec_4k.2018Mar05.650D104)

Anyway - major progress - haven't had time to look at anything else much - FYI the non crop rec res remains at 1736x976 as per your last builds. (i.e. crop_rec_4k.2018Mar05.650D104)

Compressed recording looks fine btw.

So,
Here are two more DNG test files.
This time silent & crop-rec on 650D - using - dforts: crop_rec_4k.2018Mar05.650D104

650D 720pCropRec SimpleDNG 2520x1082 Uncompressed

650D 720pCropRec SimpleDNG 2520x1082 LossLess

They look fine.

Will be available for a week from this site - PM me if you need them after that:

https://wetransfer.com/downloads/89d0be71f9cc9c17a5b9498bc4c817e620180309163835/12ad94a9cb23c2f738f6e6525bcbf39220180309163835/256e0b

Let me know if there's anything else needed.

@dfort - thanks of course you are correct!


Some recent 650D sample files as requested using - dforts: crop_rec_4k.2018Mar05.650D104

1. Silent Simple

650D 1080p simple DNG NoMlvLite Or CropRec loaded.dng
650D 1080p simple LDNG NoMlvLite Or CropRec loaded.dng
These will be on the following link for about 5 days. If anyone needs them after that i.e. after 14/3/2018 PM me.
https://wetransfer.com/downloads/f0fb7507f2ce14e1941d5d5ef1b7d0af20180308235844/c092bfd652910657811675c5ac7703b920180308235844/edf0cd

The resolution has gone from 1736x976 > 1736x1158 - not sure why. I rechecked and it went back to 976 - some knob-related confusion no doubt.
Anyway they look fine. Tried to capture a strong highlight.

2. FRSP

Also here are two FRSP dngs one lossless the other compressed. They look fine. Res 5208x3476
These will be on the following link for about 5 days. If anyone needs them after that i.e. after 14/3/2018 PM me.
https://wetransfer.com/downloads/190daac0b1f6b3aa02bf40daa4a7db0620180309043604/cda99f2fcc65521ebc0d19f3a189025820180309043604/484ae7

FRSP5208x3476_650D_UNC.dng
FRSP5208x3476_650D_LLESS.dng

As before build is crop_rec_4k.2018Mar05.650D104

3. MLVS lossless and uncompressed.

Finally 4 short mlvs. 500MB approx.

650D1080pCropRec1920x1080un14bit.mlv
650D1080pCropRec1920x1080LLess14bit.mlv
650D1080p1736x976un14bit.mlv
650D1080p1736x976LLess14bit.mlv

They look fine - titles explain it all. tried to make these as small as possible.

These will be on the following link for about 5 days. If anyone needs them after that i.e. after 14/3/2018 PM me.
https://wetransfer.com/downloads/bcd8179dd72c3edece83509355491ed820180308235937/fa2f890c986b266b9b1df9862c51d89e20180308235937/764962

Thanks again for everyones amazing work.

@a1ex - if there's any additional testing you'd like me do, let me know. I'm afraid I can't do any coding or scripting though.

Using a fast ExFat SD card and dfort crop_rec_4k.2018Mar05.650D104.

All working as far as my limited testing goes.

Recording durations - timings show when the camera stopped recording except on three occasions when I manually stopped after one minute. No sound.

This new build shows:
1080p (1736x976) mode with slightly longer recording times,
while the 1920x1080 crop mode is a mixed bag, small improvements and one substantial worsening (1920x1080 crop mode 10bit lossless.)

1080p (1736x976) 24fps - 14bit uncompressed 115 frames - marginal change
1080p (1736x976) 24fps - 12bit uncompressed 167 frames - marginal change
1080p (1736x976) 24fps - 10bit uncompressed 330 frames - marginal change
1080p (1736x976) 24fps - 14bit lossless At least 1 minute - I manually stopped the recording. - BETTER!
1080p (1736x976) 24fps - 12bit lossless At least 1 minute - I manually stopped the recording. similar
1080p (1736x976) 24fps - 10bit lossless At least 1 minute - I manually stopped the recording. similar
1080p CrpRec (1920x1080) 24fps - 14bit uncompressed -    72 frames slight improvement
1080p CrpRec (1920x1080) 24fps - 12bit uncompressed -    104 frames - same
1080p CrpRec (1920x1080) 24fps - 10bit uncompressed -    163 frames - improvement
1080p CrpRec (1920x1080) 24fps - 14bit lossless -    180 frames - slightly worse
1080p CrpRec (1920x1080) 24fps - 12bit lossless -    380 frames - slightly worse
1080p CrpRec (1920x1080) 24fps - 10bit lossless -    915 frames - worse (from 1333 frames).

I'll do the sample images tomorrow.

@a1ex - what test did you want me to do? dfort messaged me but I cant follow what test you are looking for. Thanks.

[1080p CrpRec (1920x1080) 24fps - 10bit lossless - 1333 frames - that's 55 seconds! (2.35Gb)]

Latest on Canon 650D
Using a fast ExFat SD card and dfort and a1ex's (plus everyone else involved) recent crop_rec_4k.2018Mar04.650D104

Recording Durations - timings show when the camera stopped recording except on two occasions when I stopped after one minute. No sound.

1080p (1736x976) 24fps - 14bit uncompressed 116 frames
1080p (1736x976) 24fps - 12bit uncompressed 159 frames
1080p (1736x976) 24fps - 10bit uncompressed 320 frames
1080p (1736x976) 24fps - 14bit lossless 1062 frames - 44 seconds.
1080p (1736x976) 24fps - 12bit lossless - At least 1 minute - I manually stopped the recording.
1080p (1736x976) 24fps - 10bit lossless - At least 1 minute - I manually stopped the recording.
1080p CrpRec (1920x1080) 24fps - 14bit uncompressed - 65 frames
1080p CrpRec (1920x1080) 24fps - 12bit uncompressed - 104 frames
1080p CrpRec (1920x1080) 24fps - 10bit uncompressed - 163 frames
1080p CrpRec (1920x1080) 24fps - 14bit lossless - 182 frames
1080p CrpRec (1920x1080) 24fps - 12bit lossless - 433 frames - 18 seconds.
1080p CrpRec (1920x1080) 24fps - 10bit lossless - 1333 frames - that's 55 seconds! (2.35Gb)
1080p CrpRec (2520x1080) 24fps - 10bit lossless - 193 frames

Same exposure levels - no pink frames on extreme highlights.

Silent DNG and Silent L-DNG both working - no colour shifts.

This in conjunction the excellent MLVApp0.14 (so no visible focus pixels) represents something of a landmark for my lowly 650D which can now record nearly a minute of continuous 1080p 24fps 10bit Raw. Happy days.

truly the ML guys rwk

@ dfort & a1ex: I can confirm I had idiotmode activated.

with auto iso off(!)

Again using dfort's recent crop_rec_4k.2018Mar04.650D104. 650D in 1080p:
14, 12, 10, 14LL, 12LL, 10LL - all working - no differences in exposure level & no pink highlights.

Fabulous. Thank you so much - this is amazing!!

New test 650D using dfort's  crop_rec_4k.2018Mar04.650D104.

A significant advance in the loss less capabilities using this camera / build combination.

1080p & crop mode (1920x1080) tested so far.

14bit lossless, 12bit lossless, 11bit lossless - all working much better - the blocking cadence has gone - brilliant.

However there is still this behaviour that the brightness of the realtime live-view image decreases as I step from 14 > 12 > 11bit lossless. This doesn't happen without lossless compression engaged in the mlv-lite menu. This characteristic also appears to be encoded into the compressed mlv recordings so they actually appear to get brighter and brighter as the compression ratio increases.
The other issue here is that the highlights become pinker and pinker and even using the highlight reconstruction of the mlv014 app it won't go away.

Also
silent DNG compressed / uncompressed working - except there is a slight greenish cast to the compressed version. So something is going on with the lossless on the 650D.

14bit lossless - slight pinkish cast to highlights
12bit lossless - darker live view / brighter mlv / pinker highlights
11bit lossless - even darker live view / even brighter mlv / even pinker highlights

silent - faint green cast to lossless DNG

Will do some more tests later.

happy to help

A1ex - I've done 4 tests - the first a repeat of the 1736x976 but with different compression settings. It's worth noting that the live view appears to darken as I step through from 14LL 12LL 11-8LL - not sure if this is normal behaviour but it seems that the settings in the mlv-lite and what the live_view shows may be connected.

All tests on a 650D with dfort's:crop_rec_4k.2018Feb25.650D104

I've also overlaid some screen measurements of the compression artefacts which significantly remain geometrically the same with differing compression levels but do change with changing overall image geometry as the next posts demonstrate - which is what you suggested above. Again I've no idea if any of this is helpful. The original image is a grey defocused swatch which makes it much easier to see what the compression is doing to the 650D image.

Thanks again. I've no idea how you get to think about this in the way that you do - I suppose you somehow see the numerical relations.





1736x976 1080p mlvlite





1736x584 720p mlvlite





1920x1080 720p croprec 3x3 z5




online print screen


2520x1080 720p croprec 3x3 z5

Hopefully these tests with different 650D mlv capture geometry will reveal some more of the numerical relations that you managed to identify in the first 1736x976 test.

I have the 16bit tiffs if required.

A1ex - amazing! - the cadence of your block gradient calculated image is an EXACT match to the compression artefacts in my 14lossless 1736x976 image (650D) that I posted yesterday.

It's interesting that the correlations are so exact.





free image hosting no sign up

These images are using dfort's - crop_rec_4k.2018Feb25.650D104.zip filming a gradient offscreen - using 14bit lossless & 14bit uncompressed - locked off camera - frames exported from MLVAPP0.14 as tiff - then averaged to a single frame each - overlaid with difference - I've uploaded these here at jpgs. There is a 73 pixel descending pattern in the lossless version with a horizontal repeat at about 144 pixels which might be double 73(ish). There is a 6 pixel micro pattern there too - not sure if any of this of help - there seems to be some kind of regularity in the pattern - I seem to recall that jpeg works off 7 pixel quadrants but I might be hopelessly out of my depth. Anyway if it helps...

4 images from a 650D switching LV on off, photo, 720p, 1080p, 1x5, 5x5.
If you need me to identify which is which let me know.
Using dforts allocate-raw-lv-buffer.2018Feb28.650D104.zip

Hope this helps.

dfort - I think that this build seems to work now - I've posted some images - hopefully they're what is needed. If not let me know and I will redo.

dfort, I had another go with your "allocate-raw-lv-buffer.2018Feb25.650D104.zip" and it would appear that the problem may lie with the crop.rec module. If I switch on the console and then load mlv-lite and reboot - everything initially looks okay. But when I then load the crop-rec module - both modules appear with a 'Er' sign after rebooting. Similarly if I load the crop-rec first - I get a straight croprec 'Er' on rebooting.

Not sure if this is of any help.

Did you try starting the camera with mlv_lite active? You can also start with mlv_lite off then turn it on and it should update:

unfortunately trying repeatedly with the allocate-raw-lv-buffer.2018Feb25.650D104 build, its not possible to activate mlv_lite or crop_rec

Some console screens from 650D using dforts - allocate-raw-lv-buffer.2018Feb25.650D104.zip

I tried changing photo/video/lv/crop/720p/1080p modes but the numbers seem to stay the same.

@saulbass - Thanks for testing. The 650D seems to be in the same state that is was when it was merged into the crop_rec_4k branch. Let's check and see if any of the commits made since Feb. 4 made any difference--probably not but still worth checking. In addition, the 650D hasn't been run through this test that might help. I put up the test builds on my downloads page.

I'd like to see your test MLV's but I'm no expert on how to figure out what went wrong or how to fix them. Best if you could upload them to a site and share with everyone instead of doing via PM with just me. Besides, I got called in to work this week so I won't have much time to look at it.

dfort - thanks for the reply.
I have tried the two builds that you posted:
crop_rec_4k.2018Feb25.650D104.zip
crop_rec_4k_mlv_lite_snd.2018Feb25.650D104.zip
but the errors still appear in the lossless compressed versions.

I also had a go with the console tests you described using allocate-raw-lv-buffer.2018Feb25.650D104.zip
and posted these on the https://www.magiclantern.fm/forum/index.php?topic=5601.1600
but not entirely certain I know what I'm doing here.

There's been lots of changes on the crop_rec_4k and crop_rec_4k_mlv_lite_snd branches.

The 650D is in these branches but there are no builds available on the Experiments downloads page because there are some issues with lossless compression that still need to be worked out. If you want a preview of coming attractions I put up a test build on my downloads page. Who knows, maybe we got lucky?

Maybe not so lucky with the lossless compression on the 650D. Otherwise looking good.

Using dforts experimental - 'crop_rec_4k_mlv_lite_snd.2018Feb04.650D104' from 3 weeks ago.

Results based on the fastest SD card that I can find a 32GB SanDisk Extreme Pro 95Mb/s formatted using ExFAT.
Testing with the dforts Feb 4th build: mlv_lite crop_rec modules only and setting the canon menu to 1080p.

With this build I can get a firm 165 frames at 1920x1080 10bit uncompressed. The picture via MLV App v0.14 looks great. Fantastic!

If I switch to 1920x1080 11-8bit lossless I can still only get a few seconds but the picture has become flawed - presumably there are still problems with the lossless codec on the 650D.

I've tried every combination of 10,12,14 lossless vs 10,12,14 uncompressed and all the compressed lossless clips have the same alternate scanline problem as above.

In other settings I can get 1736x976 10bit - I can get perfect pictures again for slightly longer: 331 frames but at a lower resolution.

At this resolution if I switch to 10bit lossless I can record continuously but with similar critical compression issues as above.

If dfort thinks it helpful please PM me and I can send some mlv samples.

Very happy with what's there, the MLV App v.0.14 seems to have defeated the focus pixel problem at all these resolutions.

Thanks guys.

Attached is sample of lossless problems.

Just checked dforts latest build - crop_rec_4k.2017Oct15.650D104 and have to say am absolutely stoked with the results - without crop-rec I am getting 1728x584 12 bit raw on my 650D - I think continuous but most importantly the image is clean - no focus pixels which is HUGE for us lowly 650D ML types! Really impressive. a1ex I'll try your test's with the latest build - for some reason the raw_diag didnt seem to work with the earlier build - (possibly operator error . Also the other good news is that with this latest build I can just use the generic Footage 0.31 to view/convert files on the mac - so - really this is substantial step forward for users of this humble camera. No doubt we can squeeze more resolution with the crop-rec 3x3 thing.

@a1ex - the banding artifacts that you can see in the images posted previously above do not appear in mlv_lites preview - no matter which compression format is selected.
What does happen is if I select the 3x3 zoom, my white highlights turn pink - not
sure why this happens I tried adjusting my iso and aperture but the pink highlights remain. This is common to 14.12.10 bit image formats. If I selected the next zoom ratio the pink highlights go away again and stay gone if I return to non zoom mode.

I haven't had time yet to do the specific tests you've asked for but I did some testing tonight filming a very bright light
in my shot and the banding that you see in my previous post above is still there - in all the compression modes.

Also and I'm using the fastest san disc I can for the 650D my record times are very short when using Dforts 650D test build. i.e. 2 seconds. Im not sure if this is a feature of his build or again something else is happening...

What is interesting and positive is that the dreaded focus pixels seem to have gone - at least that seems to be the case when viewing the mlvs. So that's really positive. Also I should point out that I think this is the case if I use Dforts 650D test build and dont use the zoom - just set the camera to 720p and mlv lite gives me 1728x584 which without focus pixels is very useful but I need to check this again.

I'll try and do the tests you've requested over the weekend.

Danne - thanks for the pointer - really useful to know - cheers!

Hi Bouncyball - I have a 300mb mlv from a 650D running dforts latest crop rec experiment crop_rec_4k.2017Oct11.650D104.zip - it's 2512x1072. 14bit lossless compressed.
How should I pass it over to you - Hightail?

I used dforts instructions from yesterday and managed to get the 650D set to 2512 x 1072. The horizontal bars are still there however... The camera managed about 4 seconds. I tried different image quality / compression settings but they didn't make much difference.  I can't see any focus pixels. I tried using Danne's Switch App - looks good I used it for the first image. I attached two stills of what I got. I have a 300Mb MLV if needed but not sure how to upload or transfer it. I could use Hightail but will need an address to send it. As you can see I'm not quite sure what I am doing...

Here is the 1280x720 file.

FWIW MLV.App.v0.8.alpha.OSX.zip managed to convert both the mlv files into Prores - so the actual mlv file is well formed - just the pixel structure - how this relates to the focus pixel issue I'm not sure.

So close - the data rate reduction will make a big difference with the 650D