Canon 1100D / T3

Started by a1ex, June 14, 2012, 04:50:54 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Danne

According to your question:
QuoteCan you try this on a MLV with more than 2 frames?
It´s about 60mb in size so I assume it´s more than two shitty frames in there:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M29-1336.MLV
Enjoy your wine...

a1ex

Hm, other than frame #2 (corrupted) and possibly #3 (broken metadata), all other frames are clean.

Can you run the stubs test (selftest.mo) from both the 10/12-bit build and the lua_fix build? Also EDMAC screenshots and other related stuff from selftest.mo and edmac.mo.

Danne


starbase64

Quote from: a1ex on January 29, 2018, 11:03:50 AM
There is an experimental raw video build, but other than a few snapshots from Danne, I have no idea how it works in practice => please report back. The silent picture module requires the LiveView raw backend enabled, so try it from the 10/12-bit raw video build.

If it works fine or if I can solve it easily based on user feedback, I'll consider integrating this feature into mainline.

this works fine, but display goes off and not on again.

possible to combine silent pic with advanced bracketing and intervalometer?


dfort

Quote from: Danne on January 29, 2018, 04:28:18 PM
Stubs API test starts and run but after a while falls into an error.

Interesting, I just ran the Stubs API test on the EOSM2 thinking it might show what I need to work on and it failed on LoadCalendarFromRTC, same as your 1100D test. Ran it on the EOSM and it also failed at the same spot. All tests passed on the 700D.

a1ex

The above log shows successful result on LoadCalendarFromRTC; the problem is at the next test ;)

Got this in QEMU:

       m0 = MALLOC_FREE_MEMORY => 0x43148
[Pass] p = (void*)_malloc(50*1024) => 0xc1b40
[Pass] CACHEABLE(p) => 0xc1b40
       m1 = MALLOC_FREE_MEMORY => 0x36938
       _free(p)
       m2 = MALLOC_FREE_MEMORY => 0x43148
[Pass] ABS((m0-m1) - 50*1024) => 0x10
[Pass] ABS(m0-m2) => 0x0
       m0 = GetFreeMemForAllocateMemory() => 0x6f24c
ASSERT : Memory\Memory.c, Task = run_test, Line 188
  1862:  7111.168 [MEM] NG AllocateMemory 262144
  1863:  7111.168 [MEM] Total = 0x8b0000, Free = 0x6f1ec, MaxReg = 0x239d4
  1864:  7111.936 [MEM] Num Alloc = 3296, Num Free = 293


The error is actually on the next line - it's unable to allocate 256K of memory...

a1ex

The lua_fix and 10/12-bit experimental builds also have the latest changes for 1100D; any luck with these?

The silent picture module should work too.

Danne

Tested 10/12-bit experimental
Silent DNG works and MLV has the correct size although giving malloc errors, check pics below. As I managed to stop the file before the camera buffer ran out it´s also possible to transcode the 10bit file with mlv_dump and the files looks really nice.

Stubs api test is working, ends with following message:
"test complete 11756 passed. 3 failed"
STUBTEST.LOG
https://drive.google.com/open?id=1AftiepRPtHyTs2iKOzcauhGLGH94uUUN
Some other log:
https://drive.google.com/open?id=1X88I13ruQ_jKapWnR1R0ijZI9fE5KFFe

Silent DNG:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/90630000.DNG

MLV:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M30-1622.MLV


Screenshots when recording 10bit




starbase64

can someone please update the download links in the first post...

Tested lua fix build 30.01.2018 and works fine for me. THANKS.

a1ex

Done.

Reminder: nobody ran the Lua tests on this camera yet. They were linked earlier in this thread ~ 1 month ago.

Updated 10/12-bit builds with memory management from crop_rec_4k and attempted some integration to raw_video_10bit_12bit. Will it work or will it break raw video on all other cameras?

Danne

Quick dirty test:
magiclantern-raw_video_10bit_12bit.2018Jan31.1100D105
Tested two 10bit MLV recordings: Data corruption at slot 51(file is recorded only two files extracted as before)
Tested on 12bit recording: Data corruption at slot 61(file is recorded only two files extracted as before)


Tests following this:
https://www.magiclantern.fm/forum/index.php?topic=14828.msg194706#msg194706
magiclantern-lua_fix.2018Jan30.1100D105
LUATEST.LOG
https://drive.google.com/file/d/1SivdU_wQi4Et1Mc4r5ygAF0lNbDIRDY3/view?usp=sharing
benchmarks memory
(in liveview window)
malloc error :((sad smiley from liveview window)


Should test my 100D with the latest changes in raw_video_10bit_12bit branch....
QuoteUpdated 10/12-bit builds with memory management from crop_rec_4k and attempted some integration to raw_video_10bit_12bit. Will it work or will it break raw video on all other cameras?
magiclantern-raw_video_10bit_12bit.2018Jan31.100D101
Tested a 100D with the latest changes And ran test recordings on 10/12/14 bit for mv1080, movie crop mode 5x zoom, all worked.


a1ex

Data corruption: regression or nondeterministic issue also present in previous build? (this will take some more recordings to figure out)

Can you test the Jan31 build on something else than 100D?

Lua test locked up at testing Canon GUI functions? That's odd - this test is just flipping GUI modes back and forth (MENU, PLAY, LiveView, photo mode). This test also works in QEMU...

Danne

QuoteData corruption: regression or nondeterministic issue also present in previous build? (this will take some more recordings to figure out)
Was not an issue with earlier build. Only "Hack error at 0" Could test a few more files to see if I can reproduce data corruption.

Ran a few files on a 600D with this build and all files were recorded nicely:
magiclantern-raw_video_10bit_12bit.2018Jan31.600D102

Don´t have my 5D mark III available, maybe someone else could test on that one.


Update*
Data corruption issues appears also on januay 30th build when stopping manually or camera automatically stops filming. When checking the MLV files there is a corrupted frame in the later part of the file more or less always present an all files.

Danne

Tested 31th january build on a faster SD card and 10bit was continuous with max resolution1424x480.
Data corruption seems gone. Maybe it was a slow/bad card. Will do more takes with the faster card...Yep, data corruption seems vanished.
Two noob questions. Turning off the beep? Is there a manual mode setting for when recording movies? Cannot find it.

How about lossless... :P


Some more tests:
- regular mlv_lite shooting works fine(mv720, only mode available on this cam)
sample: https://bitbucket.org/Dannephoto/magic-lantern/downloads/M31-1208_samples.zip
- 5xzoom reaches 2160x748 but needs some shaving at the top part it seems:

2160x748(5xzoom)
sample: https://bitbucket.org/Dannephoto/magic-lantern/downloads/M31-1212_samples.zip


1600x748(5xzoom)
sample: https://bitbucket.org/Dannephoto/magic-lantern/downloads/M31-1209_samples.zip




Fullres silent DNG:
Seems problematic.
Sample: https://bitbucket.org/Dannephoto/magic-lantern/downloads/90640003.DNG



starbase64

Quote from: starbase64 on January 29, 2018, 04:24:29 PM
this works fine, but display goes off and not on again.

same problem on 70D, display is off after take a photo

a1ex

@Danne: actually I was thinking at other models with low memory (600D, 650D, 700D, EOSM) and I remembered you had something from this list. These updates were taken from crop_rec_4k, initially written from 5D3, so that test is unlikely to give any surprises.

Can you also take a screenshot of the console (Debug menu - Show console) right after starting the camera in movie mode and enabling raw video, and another one right after turning it off?

Lossless compression on DIGIC 4 is hard - my attempt on 60D (some months ago) was unsuccessful. I'll revisit it at some point, but not right now - my short-term goal is to integrate the current stuff into mainline (QEMU, Lua, 10/12-bit raw video, 100D, 70D, 1200D etc).

The silent DNG looks fine to me (I have something to address the overexposure, work in progress, stay tuned).

@starbase64: if you are talking about the full-res silent picture, it should behave the way you want in the lua_fix build.

starbase64

Quote from: a1ex on January 31, 2018, 12:12:31 PM
@starbase64: if you are talking about the full-res silent picture, it should behave the way you want in the lua_fix build.

cool, THANKS

Danne

On the 1100D:
With RAW video enabled


RAW video disabled


a1ex

Looks fine. I've got another test for you to try.

Start from the allocate-raw-lv-buffer branch and apply this patch:


diff -r 23524a6ed2f9 src/config-defines.h
--- a/src/config-defines.h
+++ b/src/config-defines.h
@@ -47,3 +47,3 @@
     /** warning: it will slow down boot by a few seconds */
-    //~ #define CONFIG_MARK_UNUSED_MEMORY_AT_STARTUP
+    #define CONFIG_MARK_UNUSED_MEMORY_AT_STARTUP

diff -r 23524a6ed2f9 src/raw.c
--- a/src/raw.c
+++ b/src/raw.c
@@ -139,6 +139,6 @@
#ifdef CONFIG_1100D
-#define CONFIG_ALLOCATE_RAW_LV_BUFFER
+#undef CONFIG_ALLOCATE_RAW_LV_BUFFER
#define DEFAULT_RAW_BUFFER MEM(MEM(0x4C64))     /* how much do we have allocated? */
-#define DEFAULT_RAW_BUFFER_SIZE 0               /* is this really overwritten by other code? needs some investigation */
-#define RAW_LV_BUFFER_ALLOC_SIZE (3906*968)
+#define DEFAULT_RAW_BUFFER_SIZE 10*1024*1024    /* is this really overwritten by other code? needs some investigation */
+//#define RAW_LV_BUFFER_ALLOC_SIZE (3906*968)
#endif
diff -r 23524a6ed2f9 src/mem.c
--- a/src/mem.c
+++ b/src/mem.c
@@ -927,4 +927,4 @@

-#define MEMORY_MAP_ADDRESS_TO_INDEX(p) ((int)CACHEABLE(p)/1024 * 720 / 512/1024)
-#define MEMORY_MAP_INDEX_TO_ADDRESS(i) ALIGN32((i) * 512 * 1024 / 720 * 1024)
+#define MEMORY_MAP_ADDRESS_TO_INDEX(p) ((int)CACHEABLE(p)/1024 * 720 / 128/1024)
+#define MEMORY_MAP_INDEX_TO_ADDRESS(i) ALIGN32((i) * 128 * 1024 / 720 * 1024)



Then take two screenshots of the Free Memory dialog:
- in movie mode, without any modules loaded
- in movie mode, with mlv_lite enabled (raw video active).

Danne

nothing loaded


nothing loaded


mlv_lite RAW video loaded


mlv_lite RAW video loaded





a1ex

Wait a minute, that means the raw buffer is not overwritten by anything else. Then, where is all this allocation thingie coming from? Maybe back then it didn't work for some other reason, and we thought it's some sort of memory corruption? How do the same two screenshots look in x5 zoom?

(BTW, the screenshots are easier to compare if you don't add variable-sized white borders; try "convert foo.ppm foo.png")

Danne

A couple of jpegs 5x zoom

Nothing loaded




loaded




a1ex

That's clean, too - we may not even need this allocation stuff. Can you check the photo mode as well? This time, the two states would be:
- global draw off (camera started that way, not turned off afterwards)
- global draw on, with any raw-based overlay enabled (such as raw histogram, raw zebras or raw spotmeter)

Can you define DEFAULT_RAW_BUFFER_SIZE as 8MB (8*1024*1024 - from counting the pixels in the screenshots I get about 8.7), keep CONFIG_MARK_UNUSED_MEMORY_AT_STARTUP (so it would re-check this size, just in case) and try recording? This time, the assert should no longer get triggered.

If successful, can you try this configuration in all the video modes? (photo mode, video mode, x5 zoom, whatever else this camera has).

TODO: I'm not really sure I understand how this raw buffer allocation is done in Canon code, and whether the results are repeatable; will experiment on 60D and see what I get.

Danne

In photo mode, liveview open:

global draw off




global draw on





I added this:
QuoteCan you define DEFAULT_RAW_BUFFER_SIZE as 8MB (8*1024*1024 - from counting the pixels in the screenshots I get about 8.7), keep CONFIG_MARK_UNUSED_MEMORY_AT_STARTUP (so it would re-check this size, just in case) and try recording? This time, the assert should no longer get triggered.
To my patched version from here:
https://www.magiclantern.fm/forum/index.php?topic=1009.msg196592#msg196592

And files comes out clean and nice without errors. But I don´t get errors with today´s raw_video_10bit_12bit either even without the fix. I had a slow card before which I believe corrupted files. 5x zoom still comes out with a fishy top border like described here:
https://www.magiclantern.fm/forum/index.php?topic=1009.msg196582#msg196582

Tets only done i 14bit. I was thinking I could try the same changes in raw_video_10bit_12bit but it will not compile:
../../src/raw.c:146:2: warning: #warning FIXME: using dummy DEFAULT_RAW_BUFFER_SIZE [-Wcpp]
#warning FIXME: using dummy DEFAULT_RAW_BUFFER_SIZE
  ^
../../src/raw.c: In function 'raw_lv_realloc_buffer':
../../src/raw.c:544:39: error: 'DEFAULT_RAW_BUFFER' undeclared (first use in this function)
         if (raw_lv_buffer != (void *) DEFAULT_RAW_BUFFER)
                                       ^
../../src/raw.c:544:39: note: each undeclared identifier is reported only once for each function it appears in
make: *** [raw.o] Error 1
danands-MacBook-Pro:1100D.105 danne$


Anybody else tired of rescale and convert PPM files one by one by the way? ffmpeg does the trick:
for i in *.PPM ; do
    ffmpeg -i "$i" -pix_fmt rgb24 -vf scale=500:-1 $(basename "${i/.PPM}").tif
done