Messages

Also tried 1824 x different resolutions:

1824 x 986 = corrupted frame
1824 x 1026 = corrupted frame
1824 x 1094 = corrupted frame

[1504 x 1002 = Corrupted as the 1824 x 1026 in the post above]

Tested some different resolutions

1776 x 1000 = normal
1504 x 1002 = Corrupted as the 1824 x 1026 in the post above
1680 x 1008 = normal
1520 x 1012 = (almost) normal - looks a bit like the chroma isn't aligned
1360 x 1020 = (almost) normal - looks a bit like the chroma isn't aligned
1536 x 1024 = gives MLV's which can't be processed to dng's, normal file size  (tried multiple files, record a clip twice)
1552 x 1032 = normal

Corrected jpegs and original dng's are here:
https://drive.google.com/drive/folders/0B1BxGc3dfMDaTkpKQ0FpSFMyVmc?usp=sharing

[always]

Did some more testing with the mlv_lite and lossless 14bit.
All showed frames are corrected for black levels

In full width sensor mode, it always gives half corrupted frames (resolution 1824x1026):


But 1808 x 1016 resolution is normal:


And 1792 x 1008 has none corrupted frames but, looks like I can see a little pattern showing up:


Original dng's for download:
https://drive.google.com/drive/folders/0B1BxGc3dfMDabVdqUEdUUGMzckk?usp=sharing

This defect is likely predictable and can be easily worked around in post. Let's check the actual black levels (if you didn't do the math already):

Code Select Expand


octave:7> for i = 1:2, for j = 1:2,
>           obch = b(i:2:end,j:2:50);
>           disp(median(obch(:)))
>         end, end
1919
1791
1536
1024


Dial these values in darktable (the 4 black level sliders) and the image is back to normal

Homework for you: hardcode the above black levels in the DNG with exiftool / exiv2 / whatever you prefer.

I've managed to get the lossless 14 bit files right in RawTherapee, look at the raw black point sliders at the right upper corner.
These work with values which are extracted from the baked in black level.
Now look at those numbers, -128, -512, -256 and -1024
Just what Alex explains in his post each offset differs by the power of two.



This trick doesn't work with 12 bit or 11 bit lossless files, there's still a little pattern to be seen, probably due to some dividing data and rounding numbers during recording.

@Alex and Dfort

Null-pointer test works, no crash and a crash log:

https://drive.google.com/drive/folders/0B1BxGc3dfMDaYmQ4TnRQZ2M4TkU?usp=sharing

@Dfort

The selftest module you compiled won't link with the new-dryos-task-hooks.2017Oct20.6D116 build you made 

Committed another test I'd like you to run: selftest.mo - null pointer test. It's what caused trouble on 100D after merging this backend.

@Alex, can't find the null pointer test in the Selftest menu in the 'New DryOS task hooks' build Dfort made
I find the commit:
https://bitbucket.org/hudson/magic-lantern/commits/84dd3fdfe3c69ce111b7b8729739595e36bf3aa0?at=unified
Can't compile a build myself yet. Anybody else is willing to compile ?

Also note the lua_fix build (Experiments page) has a few additional selftest's - can you also run the stubs API from there, and also api_test.lua?

Have run the stubs api test, didn't know what to do with it, made a screenshot, seems to me like the test fails, it's hanging, doing nothing ?


Have run the api_test.lua, here's the log file:
https://drive.google.com/drive/folders/0B1BxGc3dfMDaWmhoMDEzbmpGRDg?usp=sharing

Posted new-dryos-task-hooks and dm-spy-experiments 6D.116 builds for testers. The dm-spy-experiments was built with CONFIG_DEBUG_INTERCEPT_STARTUP and I added some modules that might help move this along. Is the mpu_dump module something that should be run or is just the startup log needed?

Note to testers, search for instructions because this has probably been done before on other cameras.

As always, the builds are on my Bitbucket downloads page and may be changed or removed without notice.

Am I right in thinking we need to have lossless full res silent pictures working on the 6d before the dm-spy-experiments are of any use ?

That way we can compare Normal CR2 file taken by canon to a lossless full res silent picture

[EDIT:PROBABLY DID USE WRONG BUILD HERE FIRST, FIRST TESTRESULTS ARE USELESS, WILL DO TESTS AGAIN WITH THE RIGHT BUILD.]

EDIT:PROBABLY DID USE WRONG BUILD HERE FIRST, FIRST TESTRESULTS ARE USELESS, WILL DO TESTS AGAIN WITH THE RIGHT BUILD.
UPDATED, screenshots shown here are done with the right build 'new-dryos-task-hooks.2017Oct20.6D116'

@alex
When camera is idle, cpu usage floats around 3,5 %

Show tasks:

Sounds like you need to do a hot/bad pixel remapping on your camera.
Google for it:
hot pixel remapping canon dslr

I believe it is done by taking the lens off and putting the body cap on your camera and then select 'clean now' in the sensor cleaning menu.

@Alex
Got two screenshots from Free Memory Dialog.
https://drive.google.com/drive/folders/0B1BxGc3dfMDaTHUwR2kzNjNsbms?usp=sharing

One is from the nightly build for 6d, 4 okt 2017 and one is from a build according to this pull request:
https://bitbucket.org/hudson/magic-lantern/pull-requests/672/dryos-task-hooks-for-newer-cameras-6d-70d/diff

[If things needs to be tested for the 6d, I'm happy willing to help. But I prefer a compiled build which is ready for use.]

Another problem is that for most users, compiling ML yourself, is the biggest hurdle.

I tried it last year on OSX. For testing purposes for 10/12 bit.
Followed a tutorial on the forum and after copy and paste some stuff in terminal app, I got ML source code on my computer.
After that I could made small changes to the code and compile ML for testing purposes.
But the compiling step was also difficult, never really understood how to use it properly.
Most off the times I ended up with builds which didn't work for the 6d, or the modules wouldn't work with the ML version.
Lot's of errors are reported during the compiling, which seems normal I learned.
But if you're doing stuff you don't fully understand, it scares the shit out of you
At last I ended up compiling only the module I altered and tested it in a ML build from someone else, which worked with the module I compiled.
So the whole compiling thing didn't feel comfortable to me.

I guess it seems really straightforward and simple if your used to programming or really savvy with computers.
But for most of us, using the terminal app (or for windows user, using the command prompt) is already a big deal.
It feels like besides risking to brick our cameras with ML, were also risking to brick our computers

The last week I could do tests with the help of Dfort, he compiled the builds needed for testing and shared it with PM.
Works for me, works for Dfort and the end result for ML community is the same.

If things needs to be tested for the 6d, I'm happy willing to help. But I prefer a compiled build which is ready for use.


Also wanna help with the pull request Alex mentioned:
http://bitbucket.org/hudson/magic-lantern/pull-requests/672/dryos-task-hooks-for-newer-cameras-6d-70d/diff
If someone can provide me two builds, or maybe one build with the alterations only is enough, I'll be able to post two screenshots of the Free Memory dialog

Digging through lossless.c because Alex said the problem is there.
Can't find anything about the black levels.
http://bitbucket.org/daniel_fort/magic-lantern/src/0c3ef797ce995b51e86184049350022d0ee97324/modules/silent/lossless.c?at=crop_rec_4k_6D.116&fileviewer=file-view-default

However, lossless.c also uses raw.h, where black levels are written into the file, only thing is, the black levels seems to be autodetected 
http://bitbucket.org/daniel_fort/magic-lantern/src/0c3ef797ce995b51e86184049350022d0ee97324/src/raw.h?at=crop_rec_4k_6D.116&fileviewer=file-view-default

Tried Dfort latest build, but can't get any lossless full res silent pics with it.
Normal full res silent pics works.

Can other people with 6d confirm ?

Tried the build, but I can't take full res silent pics with it, at least not lossless.
Normal dng full res works, but if lossless is activated, no image is saved on the card 

Ok, I believe I have done step 2.

Got two pgm files from the number 2 images I posted, did md5sum check and got two different checksums, as expected...
18c6f801394a14aa87bc5159f9c73c74
VS
dc8cc8bb8eda68bd63551228b435de1f

So next step 3...

[Test 1]

Test 1
Result
- Image from pic_view has the same defects as with dcraw/mlv_dump? The problem is in the encoder configuration (lossless.c).

Test 2
Made 2 pairs of silent pics with a custom silent.mo (with some help from Dfort  )
https://drive.google.com/drive/folders/0B1BxGc3dfMDaaDhrcTVCNkJ4a2s?usp=sharing

There's also a 5th picture in this directory, it's a silent dng, but it shows nothing, but it has the full file size? This happened before with the silent module which can handle lossless.

Step 1 is done
Can other people help with step 2, 3, 4 and 5 (see post from Alex above)

Wait wait, could be that it needs a little finetuning.
The frames I get with lossless recording are a little weird, first I thought it was me who was doing things wrong with converting the mlv's to dng's with some older mlv_dump versions.
But it turns out that the silent picture module, which can create lossless dng's, also gives weird frames.

Here's an example of a normal silent picture and a lossless silent picture
http://drive.google.com/drive/folders/0B1BxGc3dfMDaNGZtLWwtYVd5WXc?usp=sharing

[mlv_lite module]

Lossless recording in the above build, use the mlv_lite module not the standard mlv_rec.mo module

Here's a chart with ML capabalities  for different cams.
https://docs.google.com/spreadsheets/d/16cgnRivbUv7nA9PUlCLmLdir3gXdIN3pqzCNAAybepc/edit#gid=5

In my opinion the 50D is way worse then the 6d you're using now.
50D is crop sensor with max video resolution of 1584 pixels wide, so not nearly 1920 pixels for Full HD.
And no audio options.

At the moment, with your 6d, you have continuously raw recording in 10bit, 24fps at 1824 × 730 (1: 2.50 aspect ratio).
Only things that could be better are
-higher resolution
-higher Bit rate (12 or 14 bit)
-aspect ratio (instead of 1:2.50 -> 16:9)

For the above you need higher writing speeds, so a camera with compact flash.
In that case, the choice is limited to 5d2, 5d3 or 7d.  (50d and 7d don't have higher resolutions in full sensor mode then the 6d)

Maybe go for a 5d2 or 5d3 both for video and stills and sell the 6d ?

Or be happy with the 6d and master the limitations for raw video.
For shots shorter then 30 seconds, you could use 12 bit.
Or for 16:9 aspect ratio use 1824 x 972 in 10 bit 24fps and have about 15 to 20 seconds recording time.

While changing memory cards a few times for testing the builds of dfort, I got two times that the red led led light started burning and stayed on while the camera didn't react to on/off button, battery pull solved it. Never seen this before, should I be worried that the error from a few post earlier has to do with this behaviour ?

Wanted to try the crop_rec module from the 6D-crop_rec.zip.
But I can't get it to work with the latest nightly build for 6d ?
It won't load/link the modules at camera startup.

@dfort You got my attention 

Tested the highly experimental build: crop_rec_4k.2017Oct15.6D116
Used the mlv_rec module and the crop_rec module (So not MLV_Lite)
And I got this error on screen and I believe error 70 on the top display

ML ASSERT:
SRM_BUFFER_SIZE == raw_buffer_size
at ../../src/exmem.c:380 (srm_malloc_cbr), task RscMgr
lv:1 mode:3

RscMgr stack: 174e70 [174f88-173f88]
0xUNKNOWN  @ ea80:174f80
0xUNKNOWN  @ 39dbc:174f58
0x00039AB4 @ ff0ef3d0:174f20
0xUNKNOWN  @ 39ae4:174f10
0xUNKNOWN  @ 39b6c:174ef0
0xFF202134 @ ff204488:174ed8
0xUNKNOWN  @ ff202304:174eb0
0x0044C930 @ 45b8ec:174ea0
0x0044C468 @ 44c990:174e70

Magic Lantern version : Nightly.2017Oct15.6D116
Mercurial changeset   : 5a67a9cd8aa9 (crop_rec_4k_6D_wip) tip
Built on 2017-10-16 01:49:21 UTC by rosiefort@RosieFoComputer.
Free Memory  : 304K + 2315K

I'm not using the latest experimental build, so can't reproduce what you're saying.

Are you in M mode on the mode dial ?

And I'm wondering, what does your display on top of your 6d say for shutter time ?
Could it be just a wrong number in the ML interface ? I'll always trust on the shutter time which is displayed on the top display of the Canon 6d.

Nope

http://www.magiclantern.fm/forum/index.php?topic=3928.msg189097#msg189097