Magic Lantern Forum

Developing Magic Lantern => Modules Development => Topic started by: Francis on April 26, 2013, 03:47:38 PM

Title: 14bit RAW DNG silent pics! (silent.mo)
Post by: Francis on April 26, 2013, 03:47:38 PM
Way to go guys. This is a time lapsers dream come true right here.

Where does the 2040x1428 res come from on the 5d2?

edit: And these DNGs edit just fine in LR4.2, so probably Adobe Camera Raw 7 as well.
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 26, 2013, 04:09:29 PM
There's a debug flag (lv_save_raw) that enables this. Coutts found it a few months ago on 6D, but back then we had no clue what it could be or how to decode it.

So, the missing piece from the puzzle was the 14-bit decoder from the raw histogram experiments, and the location of the LV RAW buffer (visible right away once g3gg0 understood how EDMAC works).

The DNG encoder from CHDK worked pretty much out of the box.

I'm now trying to sync FPS override with shutter speed, so you can take slow exposures too.
Title: Re: 14bit RAW DNG silent pics!
Post by: wolf on April 26, 2013, 04:31:43 PM
Is there a chance for DNG silent pics on other cameras, especially 550D?
Title: Re: 14bit RAW DNG silent pics!
Post by: Stedda on April 26, 2013, 04:32:39 PM
Awesome work!
Title: Re: 14bit RAW DNG silent pics!
Post by: CaptainHook on April 26, 2013, 04:35:22 PM
Great work guys!! :)
Title: Re: 14bit RAW DNG silent pics!
Post by: coutts on April 26, 2013, 04:42:13 PM
Quote from: wolf on April 26, 2013, 04:31:43 PM
Is there a chance for DNG silent pics on other cameras, especially 550D?
In theory yes, the only question is how big will the silent pics be (since buffer size depends on the camera). should be the same resolution as normal silent pics (it is on 6D i know)
Title: Re: 14bit RAW DNG silent pics!
Post by: agsingh on April 26, 2013, 05:43:50 PM
Thats amazing! Does this translate to video? How great would that be to get raw dng's at 24fps!

Great work A1ex and ML team!  :D
Title: Re: 14bit RAW DNG silent pics!
Post by: menoc on April 26, 2013, 06:06:24 PM
Quote from: agsingh on April 26, 2013, 05:43:50 PM
Thats amazing! Does this translate to video? How great would that be to get raw dng's at 24fps!

Great work A1ex and ML team!  :D

Maybe the 5D3. Digic 4 in the older cams may not be fast enough for dng at 24fps. . . . But then again, I could be wrong!
Title: Re: 14bit RAW DNG silent pics!
Post by: trsaunders on April 26, 2013, 06:44:37 PM
Awesome, means I don't have to drag my 50D around for timelapses so that I don't wear out my 5D3  8)

Great job!
Title: Re: 14bit RAW DNG silent pics!
Post by: Greg on April 26, 2013, 07:35:48 PM
It looks great. Can someone upload DNG file?
I would like to see how it looks in the ACR.
Title: Re: 14bit RAW DNG silent pics!
Post by: trsaunders on April 26, 2013, 08:16:21 PM
I just tried it out on my 5D3, but it is corrupted in everything I've tried to open it with - Shotwell, Darktable, gimp, gThumb.

here's an example file: https://www.dropbox.com/s/5c2vah29wis0wjk/52090007.DNG

(http://i.imgur.com/GRO7xx1.jpg)
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 26, 2013, 08:18:25 PM
It opens just fine with ufraw here.
Title: Re: 14bit RAW DNG silent pics!
Post by: agsingh on April 26, 2013, 08:20:06 PM
Quote from: menoc on April 26, 2013, 06:06:24 PM
Maybe the 5D3. Digic 4 in the older cams may not be fast enough for dng at 24fps. . . . But then again, I could be wrong!

Well i have a 5d3 so im hopefull  :)
Title: Re: 14bit RAW DNG silent pics!
Post by: haavard on April 26, 2013, 08:24:35 PM
I did a quick check, and I have the same corrupt data on the bottom of the DNG, it seems that I get the same picture saved as I can see on the LiveView screen, so basically the corrupt bottom part is just additional data, not a corruption of the image data itself
Title: Re: 14bit RAW DNG silent pics!
Post by: haavard on April 26, 2013, 08:25:17 PM
I opened the dng with Photoshop and Adobe Camera Raw, by the way
Title: Re: 14bit RAW DNG silent pics!
Post by: trsaunders on April 26, 2013, 08:29:58 PM
Quote from: a1ex on April 26, 2013, 08:18:25 PM
It opens just fine with ufraw here.
I just tried with ufraw 0.19 and got the same result:
(http://i.imgur.com/Bwd5G4Il.jpg) (http://imgur.com/Bwd5G4I)
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 26, 2013, 08:38:00 PM
Yea, just crop it. Since I don't know how to autodetect the image height (other than hardcoding the image resolutions), I've just assummed a 2:3 ratio.

I've also tried with dcraw - opens fine. Not sure what's wrong.

(http://a1ex.magiclantern.fm/bleeding-edge/raw/ufraw.png)

dcraw.tiff (http://a1ex.magiclantern.fm/bleeding-edge/raw/dcraw.tiff)
Title: Re: 14bit RAW DNG silent pics!
Post by: trsaunders on April 26, 2013, 08:43:04 PM
If I set the temperature, green and exposure to match yours I get the same image, so it must be guessing them wrongly for some reason. Very strange!
Title: Re: 14bit RAW DNG silent pics!
Post by: Kabuto1138 on April 26, 2013, 08:51:36 PM
Great work guys!!  just when I was thinking of leaving Canon for a panasonic camera you guys open the door for huge possibilities!!

I have a 5D2 if there is anything to test I'll be more than happy to.
Title: Re: 14bit RAW DNG silent pics!
Post by: Greg on April 26, 2013, 08:59:30 PM
Quote from: trsaunders on April 26, 2013, 08:16:21 PM
here's an example file: https://www.dropbox.com/s/5c2vah29wis0wjk/52090007.DNG
thank you ;)

Here an example of ACR 7.4
http://img5.imageshack.us/img5/7403/acra.jpg

(http://img19.imageshack.us/img19/6696/setvv.jpg)
Title: Re: 14bit RAW DNG silent pics!
Post by: Yoshiyuki Blade on April 26, 2013, 09:53:31 PM
Great work guys! I tried it on the 5D2 and cropped off 160px on the left-hand side and 18 at the top. The ratio appears to be 4:3, 1880x1410. I haven't tested this yet, but will the resolution be different when taking silent pics while recording (like the old way was)? edit: Just checked, it doesn't so this is the theoretical max I assume. edit2: Oops, I didn't notice the big white border at the bottom. It's closer to 3:2 now, at 1880x1248.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 26, 2013, 10:15:03 PM
so, then, is FPS override still better as a resolution, at a solid, 1k, asides from it being .dng ..
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 26, 2013, 10:38:26 PM
Mine have that shit on the bottom too... at least it helped me figure out Hskip and vskip.

I'm trying to get the recorder working but so far no dice. Nothing is written.

*They are 1920x1344 ... bigger than silent pics for sure but not like 40MB matrix file.

Heh, I think it needs to remap the AF button like regular silent pics.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 26, 2013, 11:22:00 PM
+1 for af button re-map!
Title: Re: 14bit RAW DNG silent pics!
Post by: Yoshiyuki Blade on April 26, 2013, 11:34:20 PM
I must say that the quality is much, much better than the original silent pic. Before, the difference in actual resolved detail between video (at Q-16) and the raw silent pic was virtually nonexistent. Sure there was a quality difference with the absence of H.264 compression artifacts, but it didn't really increase resolution by much. The raw DNG definitely captures more detail. I can read smaller text that's very blurry in video more clearly. For kicks, I compared it to a "reference" full resolution 21MP still image downscaled to 1920x1080 and although it obviously can't compare (still has aliasing issues among other artifacts), it's still pretty darn good and has similar resolution qualities.

The only problem atm is that there's no way to properly apply a picture style to the raw DNG. Video and the old silent pic basically had the picture styles "baked in," but that's not the case for raw DNGs now. So although I can still get a rough idea on the resolution improvement, the colors are way off and won't be easy to adjust.

It makes me wonder what else utterly destroys the video quality when it goes from the raw data to the final output. As I've mentioned before, the old silent pic didn't look much different from normal video, save for the lack of compression artifacts. Being 4:2:2 and uncomrpessed didn't make as a huge perceptual difference as the raw DNG does now.
Title: Re: 14bit RAW DNG silent pics!
Post by: dfsgfeiuhd on April 26, 2013, 11:42:37 PM
Wow, you guys rock! This is amazing! I hope this is also going to work on my 600d and I will keep my fingers crossed for a 25 fps version RAW DNG recording solution. Maybe I do not need to buy the blackmagic pocket cinema camera after all...

I'll keep this development under close observation.
Title: Re: 14bit RAW DNG silent pics!
Post by: Kabuto1138 on April 26, 2013, 11:51:25 PM
Amazing development the same day the 5d mark III HDMI out leaked to underwhelming results.

You guys rock!
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 27, 2013, 12:07:30 AM
I'll try to figure it out for 600D, we'll see how that goes. Lvrec working for me now.

Quote
It makes me wonder what else utterly destroys the video quality when it goes from the raw data to the final output.

Simple, throwing away color to 4:2:0 and repeated resizing. 4MB of data to less than 1
Title: Re: 14bit RAW DNG silent pics!
Post by: hookah on April 27, 2013, 12:28:53 AM
This is so COOL !!!

Just did an compile for my 5D3.. ( holy f.. dammm so many cool feuture in there !!!! )
Works great in lightroom (but like you guys i also get that black bar and data noise in the bottom.)
Title: Re: 14bit RAW DNG silent pics!
Post by: woodybrando on April 27, 2013, 12:43:59 AM
Hi Hookah,
could you share your autoexec.bin for the 5d3?
thx,
Jayson
Title: Re: 14bit RAW DNG silent pics!
Post by: Yoshiyuki Blade on April 27, 2013, 12:48:27 AM
Quote from: 1% on April 27, 2013, 12:07:30 AM
Simple, throwing away color to 4:2:0 and repeated resizing. 4MB of data to less than 1

I don't think chroma sub-sampling is a factor in this case, unless Canon's subsampling method is so awful that it affects the luma channel :D. Looking at the stills in 4:0:0 (greyscale), there's a definite difference in the luma resolution. Is there anything big happeneing between the point where the raw DNGs are being captured to the point where 4:2:2 raw-silent-pics-of-old were recorded? Other than chroma sub-sampling, which has a very minimal perceptual impact, the difference in resolved detail is still quite clear.
Title: Re: 14bit RAW DNG silent pics!
Post by: Francis on April 27, 2013, 12:52:51 AM
Quote from: woodybrando on April 27, 2013, 12:43:59 AM
Hi Hookah,
could you share your autoexec.bin for the 5d3?
thx,
Jayson

Build your own. It isn't very hard.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 27, 2013, 01:06:47 AM
in 720P overriden to 24fps I got over 150 frames.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 27, 2013, 01:10:13 AM
what does that mean?
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 27, 2013, 01:12:30 AM
720P 422 is doable... the raw size never changes no matter what mode you're in.
Title: Re: 14bit RAW DNG silent pics!
Post by: woodybrando on April 27, 2013, 01:29:33 AM
Quote from: Francis on April 27, 2013, 12:52:51 AM
Build your own. It isn't very hard.

hah, it isn't hard if you know what you're doing. I've spent about 20 hours trying to get compilers working for android or magic lantern or linux and haven't ever gotten them to work. There's always some tool in the tool chain that's .1 version too old or too new and 5 hours later troubleshooting I remember why I quit studying computer science.  :)

i.e.
USERNAME@USERNAME ~/magic-lantern/platform/5D3.113
$ make
[ AS       ]   entry.o
/bin/sh: /home/USERNAME/gcc-arm-none-eabi-4_7-2012q4/bin/arm-none-eabi-gcc-4.7.3.exe: No such file or directory
../../Makefile.inc:744: recipe for target `entry.o' failed
make: *** [entry.o] Error 127


Title: Re: 14bit RAW DNG silent pics!
Post by: Francis on April 27, 2013, 02:32:52 AM
Quote from: woodybrando on April 27, 2013, 01:29:33 AM
hah, it isn't hard if you know what you're doing. I've spent about 20 hours trying to get compilers working for android or magic lantern or linux and haven't ever gotten them to work. There's always some tool in the tool chain that's .1 version too old or too new and 5 hours later troubleshooting I remember why I quit studying computer science.  :)

i.e.
USERNAME@USERNAME ~/magic-lantern/platform/5D3.113
$ make
[ AS       ]   entry.o
/bin/sh: /home/USERNAME/gcc-arm-none-eabi-4_7-2012q4/bin/arm-none-eabi-gcc-4.7.3.exe: No such file or directory
../../Makefile.inc:744: recipe for target `entry.o' failed
make: *** [entry.o] Error 127

Download prebuilt toolchain. Point Makefile.user to the location.

I had no idea what I was doing when I tried for myself.
Title: Re: 14bit RAW DNG silent pics!
Post by: Greg on April 27, 2013, 02:39:55 AM
Can we compress dng?

Example:
Original - 5.09MB
After compression in Adobe DNG Converter - 2.63MB

2.63MB x 24FPS = 63,12MB/s
I do not know rules of operation dng in the camera, it's just an example.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 27, 2013, 02:44:30 AM
what is this 24fps thing with this? how can you take 24fps silent pic?
Title: Re: 14bit RAW DNG silent pics!
Post by: N/A on April 27, 2013, 03:43:58 AM
Quote from: 1% on April 27, 2013, 01:12:30 AM
720P 422 is doable... the raw size never changes no matter what mode you're in.

What about through the hdmi out into an Atomos Ninja as opposed to straight to the SD? Would love to test this but I'm guessing even the Sandisk 95 buffers out after a second or two.
Title: Re: 14bit RAW DNG silent pics!
Post by: premini on April 27, 2013, 04:14:13 AM
Oh man, this is exciting!
Any chance that my good ol 7d can handle this?
Title: Re: 14bit RAW DNG silent pics!
Post by: squig on April 27, 2013, 05:27:58 AM
Quote from: 1% on April 27, 2013, 01:12:30 AM
720P 422 is doable... the raw size never changes no matter what mode you're in.

How does the 720p resolution compare to H.264 1080p? This could be a great option for anamorphic shooting. We could talk to Dan at Convergent Designs about a recording option for the Odyssey.

Great work guys.
Title: Re: 14bit RAW DNG silent pics!
Post by: Kabuto1138 on April 27, 2013, 05:31:29 AM
I tried the nightly build.  DNG goodness, but I'm only getting like 1 file per second.  Also, every few images it goes to one of my costume profiles, not "RAW" looking.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 27, 2013, 06:42:57 AM
Right now can't decode the solid files.

*Looks like size does change but DNG is set to write fixed size.
Title: Re: 14bit RAW DNG silent pics!
Post by: fauxtographer on April 27, 2013, 06:56:45 AM
YAY!! This is so frickin' cool!!!
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 27, 2013, 07:59:44 AM
The nightly seems broken, use this link for 5D2 (http://nanomad.magiclantern.fm/nightly/archive/2013-04-27-58ab08005dcc/) until it's fixed. For 5D3 you need to compile yourself for now. The user-friendly installer still needs some work, so you have to use the developer kit and understand what you are doing. I'll look into it these days.

Some things to clear up:

- It doesn't save CinemaDNG, but just plain DNG, copied from CHDK. They have been doing this for years. (http://chdk.wikia.com/wiki/Raw)

- This will not improve rolling shutter at all.

- Don't hold your breath for raw video. 24fps at 1080p is unlikely IMO, even on 1000x cards.

- HDMI out doesn't help. Here's why: raw image buffer (http://a1ex.magiclantern.fm/bleeding-edge/raw/ok.tif) vs recorded (http://a1ex.magiclantern.fm/bleeding-edge/raw/hdmi.tif) with hyperdeck shuttle uncompressed. Ignore the overlays.

- In continuous mode I've got 4fps of raw data (not DNG, just the raw image buffer) on a 266x card. G3gg0 got 12.5fps of YUV422 video (http://www.magiclantern.fm/forum/index.php?topic=5247) on the Lexar 1000x card.

- I was able to record around 30 frames in burst mode, at LiveView FPS (24, 25, 30, 50, whatever). It takes roughly 1 second of "recording" to RAM and then half a minute of saving the DNGs to card.

Nevertheless, I find this very useful as a fast burst mode for stills, or when you want to save shutter clicks or just avoid noise.

Luke Neumann already tried this stuff and made a short video.

http://neumannfilms.net/2013/04/27/14bit-raw-video-on-canon-dslrs-with-magic-lantern/

Title: Re: 14bit RAW DNG silent pics!
Post by: squig on April 27, 2013, 08:15:29 AM
Are you able to do a crop like 1365x803 to write 24fps to a 1000x card? which is 2k 2.55:1 with a 1.5x anamorphic lens.
Toshiba just announced a CF card with 65MB/s minimum write speed http://www.gizmodo.com.au/2013/04/behold-the-fastest-cf-card-ever-made-for-now/
Title: Re: 14bit RAW DNG silent pics!
Post by: DTSET123 on April 27, 2013, 08:20:30 AM
I think there is no crop mode, it only records one size. I wonder if it is that CPU is not fast enough or what? How is it that pocket cinema is able to do it on a SD?
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 27, 2013, 08:22:48 AM
Not really, width is hardcoded to 2080. I may be able to crop the height, but I can't control the buffer size yet.

Continuous recording is a lot harder than short bursts, and right now it's beyond my skills, sorry.
Title: Re: 14bit RAW DNG silent pics!
Post by: N.Mendes on April 27, 2013, 08:27:39 AM
Quote from: a1ex on April 27, 2013, 07:59:44 AM

- Don't hold your breath for raw video. 24fps at 1080p is unlikely IMO, even on 1000x cards.



This part is the most important sentence that has been said since this crazy announcement.. So, people, this huge news is for "time lapsers", not for film makers, i guess.. Unfortunately, don't expect our 5d3 to be a BMC competitor..

Whatever, ML team, you do a great job..
Title: Re: 14bit RAW DNG silent pics!
Post by: squig on April 27, 2013, 08:30:48 AM
If you can control the height and make it 2080x535 we could just shoot with the anamorphic lens the other way around. Yeah the buffer is the key, the card write speed shouldn't be a problem.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 27, 2013, 09:08:53 AM
The dng data is roughly the same file size as the 422 files. Like 4MB... but all the files are the same size because H isn't read. I recorded pretty continuously with 720P slowed down, if it was timed right and the DNG was cut to the right height it might... I don't think it updates at like 24fps tho... more like 7 or 8 it will look like jpeg eos utility stuff out of that one utility with frame skips and all... but it would be raw.

Just dunno how to play the files back :(
Title: Re: 14bit RAW DNG silent pics!
Post by: Audionut on April 27, 2013, 09:14:58 AM
Quote from: 1% on April 27, 2013, 09:08:53 AM
Just dunno how to play the files back :(

You'd have to decode the dng's into something that something else can use to combine the images into video.
Avisynth might have a plugin that can read DNG's natively, but I doubt it.  You'd probably still need to convert to TIFF or JPG first.
Title: Re: 14bit RAW DNG silent pics!
Post by: squig on April 27, 2013, 09:16:07 AM
Frame skips, hehe, could be the next big thing!
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 27, 2013, 09:19:21 AM
I feel like the DNGs should be separate not one file... also they would be like 1MB if it wasn't writing  the extra garbage data.  Also I have to see what happens when you zoom in.

If it was possible to compress this data with something else we'd have 1080p probably.
Title: Re: 14bit RAW DNG silent pics!
Post by: squig on April 27, 2013, 09:22:48 AM
For DNG playback I'm thinking an animation app like Cinema 4D or Flash might do it. Have you tried VLC?
Title: Re: 14bit RAW DNG silent pics!
Post by: Rush on April 27, 2013, 09:40:16 AM
Quote from: 1% on April 27, 2013, 09:08:53 AM
The dng data is roughly the same file size as the 422 files. Like 4MB...
g3gg0 made possible 12.5 fps recording with 422 files. May his method of writing will help? Or RAW burst have other limitatons?
Title: Re: 14bit RAW DNG silent pics!
Post by: trsaunders on April 27, 2013, 02:30:36 PM
How is the RAW_LV_EDMAC address found? I opened the EDMAC tool on my 5D3 but none of the addresses it finds seem similar to the one in consts.h.

silent_pic_take_raw() calls 'lv_save_raw', is there a way to find all callable functions (I assume this is a canon function?) from a firmware dump?

I'd like to help out with ML dev so I'm trying to understand what is going on here  ;) maybe there are some experiments I can run on my 50D to see if that can support raw silent pics?
Title: Re: 14bit RAW DNG silent pics!
Post by: nanomad on April 27, 2013, 02:35:27 PM
search for the string lv_save_raw in the disassembled firware, you'll get one reference just before a call to register_func. One of the paramters of that call is a pointer to the location of the function.
Title: Re: 14bit RAW DNG silent pics!
Post by: g3gg0 on April 27, 2013, 02:47:44 PM
lv_save_raw just sets up a variable.

search for "_StartMem1Edmac".
those will connect write edmac #18 to connection 0 (raw source)
Title: Re: 14bit RAW DNG silent pics!
Post by: Alia5 on April 27, 2013, 03:13:22 PM
Besides Video...
If it works on 600D etc. we have finally working smaller RAW images! Great for doing longer sequences!
Title: Re: 14bit RAW DNG silent pics!
Post by: sl0w0rm on April 27, 2013, 03:39:08 PM
It should work on 600D because it does work on my 60D. I didn't know how to find edmac address so I used "lv_raw_dump2" which saves raw data as "*.mm1" file (it's just a memory dump).
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 27, 2013, 05:13:29 PM
Well 600D already had mraw/sraw in a  similar form... I guess this one is easier to process and more supported.

I think that register the same for many: It is used by LVx5_StopPreproPath

There are 2 in there ... 08 and 208...

600D and eos M are both c0f26208 from the looks of it.

Title: Re: 14bit RAW DNG silent pics!
Post by: sl0w0rm on April 27, 2013, 06:54:38 PM
c0f26208 also works for 60D, thanks 1%
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 27, 2013, 07:09:13 PM
where's burst mode, on the mkii?
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 27, 2013, 07:25:08 PM
Watch Luke's video (with sound!) to find out.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 27, 2013, 07:46:52 PM
Ive been using ML for a while, Alex, I'm very keen on the menu...its not in my options, as to the reason, why I asked.  So, instead of making me watch the video again, whilst being a great video, it's not answering the question, at hand, so maybe you could answer, instead of not..?
Title: Re: 14bit RAW DNG silent pics!
Post by: sl0w0rm on April 27, 2013, 07:49:11 PM
Quote from: squig on April 27, 2013, 08:30:48 AM
If you can control the height and make it 2080x535 we could just shoot with the anamorphic lens the other way around. Yeah the buffer is the key, the card write speed shouldn't be a problem.

Height can be changed by changing resolution in movie mode.

On 60D in movie mode I get:
1080p/720p: buffer 1888x1321, usable 1736x1154
480p: buffer 1888x1321, usable 1736x694
480p crop: buffer 920x644, usable 768x598
Title: Re: 14bit RAW DNG silent pics!
Post by: trsaunders on April 27, 2013, 08:07:18 PM
The 5D2 address / settings works for 50D. Pictures are 1664x1164. Sample Image (https://www.dropbox.com/s/7a5nv4b049w0kiy/14830004.DNG)
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 27, 2013, 08:22:51 PM
You will also need to copy the color matrix and white level from dcraw. It will also work with 5D2/3 values, but colors may be a little off.

For white level I'm thinking at some autodetection, because it seems to vary from camera to camera and also with ISO and shutter speed. Maybe hardcode it as around 13000, and auto increase it if there are pixels brighter than that. Better ideas?

I'm also trying to reorganize the code a bit, so the DNG converter would be completely platform-independent (without camera-specific constants, so we can load it as a module), and also I want to add macros for converting pixel coordinates from screen to raw image buffer (e.g. BM2RAW, LV2RAW etc). So I won't enable it on the other cameras right now, but I'll do it once the implementation will be a bit cleaner.

Feel free to experiment with it.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 27, 2013, 08:32:25 PM
So that's why everything starts out pink? I'll look for matrix on EOSM + 6D.

Seems the code to remap the button to the back is now only left for trap focus. If you load as module some cameras will have issues like 600D... I dunno maybe possible to give it 640K bins but then I'll only have less than an MB of ram even putting the mirror in shoot malloc.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 27, 2013, 08:40:09 PM
so... burst on mkii?
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 27, 2013, 09:15:55 PM
Module support is not yet mature, but it's the way to go for low-mem cameras. Not right now, but in the future.

For burst: Luke's video at 1:25. Don't expect everything overnight.
Title: Re: 14bit RAW DNG silent pics!
Post by: Kabuto1138 on April 27, 2013, 09:40:05 PM
To import and play the DNGs as a sequence just import them in After Effects and render them out.   If you have CS6 just import the After Effects composition to premiere so you can edit with it.

Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 27, 2013, 09:57:42 PM
The dng sequence is one file. i can do that with 422s tho.

I think would be doing continuous 422 if the module (or any module) loaded on EOSM
Title: Re: 14bit RAW DNG silent pics!
Post by: trsaunders on April 27, 2013, 10:26:13 PM
Is it possible to directly access the storage (CF/SD) at the block level? I'm thinking that for cameras with dual card slots, an unformatted (hopefully this would stop other canon code accessing it) card could be used to directly store the raw LV data without a filesystem. This might be a wild idea, and I have no knowledge of how the DMA works on the canon chips, but other ARM devices allow you to do DMA from one peripheral to another without involving the cpu - could we pipe the raw LV data directly to the CF/SD periphal, writing a block at a time and some sort of basic linear file system?
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 27, 2013, 10:35:02 PM
Yes, look in bootflags.c. No idea about DMA on the cards; for that, one has to investigate the low-level file i/o routines.

I only scratched the surface and "fixed" a compatibility bug with certain cards on 5D3 (well, only a dumb workaround, but works for my kingston 266x): http://www.magiclantern.fm/forum/index.php?topic=2528.0
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 27, 2013, 11:34:42 PM
Lol... or mount a file as CF card on SD only cameras? Then canon wouldn't fuck with it? This is some serious stuff tho.
Title: Re: 14bit RAW DNG silent pics!
Post by: hjfilmspeed on April 28, 2013, 01:51:42 AM
OHHHHHH MYYYYYYYY WOOOOOOORRRRRRRDDDDDDD!!!!!!!!! THIIISSSS IS BEYOND WORDS I CANT EVEN TYPE RIGHT NOW!!!!!!!!!!!
Title: Re: 14bit RAW DNG silent pics!
Post by: hjfilmspeed on April 28, 2013, 04:17:18 AM


I made this video with sRAW stills and processed them with adobe camera raw. After making this i never want to shoot with h.264 again. If ML can tame this dng sequence we are in for a a real treat to say the least!
Title: Re: 14bit RAW DNG silent pics!
Post by: Kabuto1138 on April 28, 2013, 05:11:58 AM
@hjfilmspeed

Great job!!
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on April 28, 2013, 05:37:49 AM
To the magic lantern team, I just want to drop by and say good job guys. I look forward to making this a usable feature.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 28, 2013, 06:11:40 AM
The raw is pretty but the 422 will give similar results, esp after compression. Probably would be H264 -> jpeg -> yuv422 -> 14 bit raw.

frame size much over 1-2MB is a tough prospect tho.
Title: Re: 14bit RAW DNG silent pics!
Post by: Brian@202020 on April 28, 2013, 06:59:28 AM
Hi everyone I'm new to this forum.

Alex,

I saw how you said no cards are fast enough for 24p and the buffer is an issue with 24p when going out HDMI. Forgive my ignorance, but would it be possible to get 12fps recorded to a card and the other 12fps going out the HDMI, and then combine in post to get 24p?
Title: Re: 14bit RAW DNG silent pics!
Post by: engardeknave on April 28, 2013, 09:12:19 AM
I am basically foaming at the mouth at the possibilities for bracketing with DNGs. I could have sworn I saw take_silent_pic() in the pico API, but it's not there.

Would it be possible to add this (ideally with choice of file name)?
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 28, 2013, 09:17:49 AM
Just take normal pics and enable silent mode in menu.
Title: Re: 14bit RAW DNG silent pics!
Post by: sl0w0rm on April 28, 2013, 10:03:45 AM
There should be a way to get better resolution even on older bodies because in 5x/10x zoom on 60D I get dng 2520x1764, usable 2520x1080
Title: Re: 14bit RAW DNG silent pics!
Post by: Northernlight on April 28, 2013, 10:54:16 AM
Quote from: N.Mendes on April 27, 2013, 08:27:39 AM

This part is the most important sentence that has been said since this crazy announcement.. So, people, this huge news is for "time lapsers", not for film makers, i guess.. Unfortunately, don't expect our 5d3 to be a BMC competitor..

Whatever, ML team, you do a great job..

I agree that it is important to be aware that we are unlikely to see 24fps RAW video. But I do not necessary agree that this "new DNG/RAW feature" is a hallelujah for time-lapsers. I would never accept being limited to only shooting at 2K when doing time-lapses! I and many more would require to be able to do the footage in full resolution (minimum 4K)!!  I would rather take the relatively low cost of switching the mirror shutter mechanism when it is worn out, than being limited to only 2K RAW files !!

So if this new feature will not implement RAW video, I will probably never get any use of it!



Title: Re: 14bit RAW DNG silent pics!
Post by: Primary on April 28, 2013, 11:59:42 AM
What about the lower resolution? Could we get higher frame rate?
I think the 24 fps raw video in 720p better than 1080p normal video.
Title: Re: 14bit RAW DNG silent pics!
Post by: Steven on April 28, 2013, 03:38:59 PM
I guess it has been mentiones before, but in case it has not: What about spanning the recording over the dual slots? Is it possible to store every second frame onto the SD card for example? Didn't read the whole thread, sorry if it has been discussed before...
Title: Re: 14bit RAW DNG silent pics!
Post by: wolf on April 28, 2013, 05:59:38 PM
Does anybody know the "magic" register address for the  550D ?
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 28, 2013, 06:14:44 PM
QuoteI would rather take the relatively low cost of switching the mirror shutter mechanism when it is worn out, than being limited to only 2K RAW files !!

I think all the modes have a place. The matrix 422 files were 40MB... bigger than a raw. FPS override is great for convenience. And raw is a great replacement for 1:1 422 pics but the burst rate on those was nicer. Seems like having all of them would be best option vs throwing them away.

I think the only way to get 24fps "raw" would be compressing the raw to jpeg but keeping the color data, etc. I have to try this on 600D and see what (if anything) happens in crop mode.


I just discovered in 5x zoom mode my resolution is over 2k (at least reported... I'm collecting files to update morning changes).


Opened the files and its weird... the width is real... but it looks cut in half height wise... can both parts of the image be gotten by moving the zoom box to the bottom?
Title: Re: 14bit RAW DNG silent pics!
Post by: Greg on April 28, 2013, 07:06:01 PM
500D :
(http://img716.imageshack.us/img716/5006/vram0.png)
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 28, 2013, 07:32:34 PM
You have to look in the disassembly.. your lV reg is probably the same... for raw histogram its a bit harder.

And yes... appears you can move the zoom box... my zoom height is 981 so like 3 pics and I guess you'd have the whole 3.5k image. How is that for not being limited to 2k?
Title: Re: 14bit RAW DNG silent pics!
Post by: wolf on April 28, 2013, 08:13:40 PM
@Greg Thanks.
@1% My understanding is too little, I even wonder if you reply to my post :-)
I followed the thread and remember looking in the disassembly, but my machine code knowledge is limited to 6502 MOS Technology. Since when does a disassembler name register?
I hope it will work on a 550D and improve the image quality for silent pics  in the future and better be patient...
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 28, 2013, 09:02:00 PM
Small update, just added some cool stuff to the repo:

- silent burst mode working on 5D2 and 3 (roughly 30 pics at LiveView FPS and resolution)
- raw histograms and zebras are now showing the dynamic range from DxO

If you look at the uncompressed 422 recorder (http://www.magiclantern.fm/forum/index.php?topic=5247), you'll notice that 1920x720 is doable on the 1000x card. It's not working yet for raw, but the frame sizes are comparable (2741760 bytes for a 1904x720 YUV422 frame, vs 2399040 bytes for 2080*720 RAW).

I've also reworked the raw code to make it easier for porting - all camera-specific stuff is now in raw.c. The porting process goes roughly like this:
- add raw.o and chdk-dng.o to camera Makefile (in platform dir)
- read the comments from raw.c (https://bitbucket.org/hudson/magic-lantern/src/tip/src/raw.c)
- fill in the "blanks" in raw.c (look for #ifdef CONFIG_5D2/5D3 and add similar blocks for your camera)
- most values are found by trial and error; it's not hard, but it takes time
- enable the raw features in features.h (look at 5D2/3 for valid names)
- enable the debug stuff for testing / troubleshooting (RAW_DEBUG, RAW_DEBUG_DUMP, RAW_ZEBRA_TEST etc)
- probably the easiest one to port is FEATURE_SILENT_PIC_RAW (simple, not burst). After this is working, try burst. Finally, try the photo mode overlays, these are the hardest to get right.
- submit a pull request (or just paste your modifications here)

Enjoy!
Title: Re: 14bit RAW DNG silent pics!
Post by: Kabuto1138 on April 28, 2013, 09:26:11 PM
Awesome Alex!!  Can't wait to try it.  Will this be in the nightly build?
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 28, 2013, 09:30:20 PM
stoked!...loaded it up, stil cant find that burst option, in live view, in the 5dmkii..can someone help me locate it please!>?
Title: Re: 14bit RAW DNG silent pics!
Post by: ozcancelik on April 28, 2013, 10:04:08 PM
For me just shutter speed limit is problem. But i guess it can solve after RAW Fps Override feature.
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 28, 2013, 10:19:37 PM
You can combine it with FPS override. The current nightly may not sync correctly, but the next one will.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 28, 2013, 10:22:22 PM
combining it with fps override will be a pretty great implementation!
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 28, 2013, 10:37:58 PM
The 422s worked with fps override and seemed in sync.

Haven't tried burst yet with FPS tho.

52 frames in 720P 60 overridden to 24.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 28, 2013, 10:41:53 PM
can we get a midday release?!
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 28, 2013, 10:59:04 PM
I just uploaded 6D... I dunno about anyone else... With FPS override on for some reason the 720P image has more missing pixels on the right... fps override off its perfect  ???
Title: Re: 14bit RAW DNG silent pics!
Post by: hirethestache on April 28, 2013, 11:24:58 PM
Quote from: 1% on April 28, 2013, 10:59:04 PM
I just uploaded 6D... I dunno about anyone else... With FPS override on for some reason the 720P image has more missing pixels on the right... fps override off its perfect  ???

Just posted a new topic, would love yours or Alex's input.

http://www.magiclantern.fm/forum/index.php?topic=5261.0

Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 29, 2013, 01:16:41 AM
It still needs to remap the button...  AF moves and can ruin your burst.
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 01:19:04 AM
I always keep AF on the back button
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 29, 2013, 01:35:42 AM
Thats good for 5d3 and 6D in theory but not for really for EOSM or 600D and the like.
Title: Re: 14bit RAW DNG silent pics!
Post by: hjfilmspeed on April 29, 2013, 01:39:49 AM
this is sooooo great. I was testing the dng sequence from neumannfilms. I know that you need to crop down to get rid of black areas and funny stuff on bottom of frame but then it made the stair stepping/little artifacts a little more noticeable (better than all-1 tho). i kept the same 16x9 crop but zoomed out a bit (in adobe camera raw) and found the detail to be much more pleasing (but there was a black bar on left). Even if we had to give it more of 2:35:1 crop (to save buffer) using the full with with of that image would be worth it. Is this possible? I know its in early stages and again i know nothing about firmware code so sorry if this is a dumb question. Again amazing work as always ML and all who helped!
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 29, 2013, 01:53:29 AM
I don't think any-more black. I didn't check lvrec. I don't think it had any issues before.

You can crop the DNGs smaller in theory and do fps override.  I guess longest mode was 760P with 24P override. 640-30P is full HD sized on 6D, they probably just feed the LV image to the encoder and leave the regular buffer running.

Its like 2 seconds of DNG burst or less... I have to make files I took into videos and see how fluid they play back. No clue what dude did.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 29, 2013, 04:38:36 AM
burst on mkii yayyyyyyyyy!!!!!!!!!
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 29, 2013, 04:51:51 AM
what is, burst with end trigger. records loads it says, really only 36..?
Title: Re: 14bit RAW DNG silent pics!
Post by: Yoshiyuki Blade on April 29, 2013, 06:08:04 AM
Wow, even if it's only 1.5s @ 24 fps, getting a glimpse of the camera's absolute best quality in real-time is incredible! Can anything out there capture these frames at 105 MB/s continuously? :P

Gonna play around with this a little more!

edit: Tested a clip at lossless 4:4:4 8-bit, but my computer couldn't decode it in real-time lol. As expected, there's not a much of a noticeable difference in real life footage.
Title: Re: 14bit RAW DNG silent pics!
Post by: N.Mendes on April 29, 2013, 07:08:10 AM
And what about a recorder like the "Odyssey 7" (which is not available yet)

http://www.cinema5d.com/news/?p=17127

Raw 24p on the 5d3 sounds possible with this, am i right?
Title: Re: 14bit RAW DNG silent pics!
Post by: Rush on April 29, 2013, 07:28:15 AM
Quote from: N.Mendes on April 29, 2013, 07:08:10 AM
And what about a recorder like the "Odyssey 7" (which is not available yet)
HDMI is limited to "up to 1080p60 422 8-bit" - it says there.
5D3 don't have SDI output which needed to feed RAW to recorder.
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 07:31:25 AM
Can it record uncompressed?

With hyperdeck shuttle I couldn't record the RAW (sharpening, luma averaging, 16-235 scaling), so I gave up after trying for a few days. I'm not 100% sure if these artifacts are done in the camera or in the recorder, but the sharpening halos can be visible on my monitor too.
Title: Re: 14bit RAW DNG silent pics!
Post by: squig on April 29, 2013, 07:50:12 AM
Quote from: a1ex on April 29, 2013, 07:31:25 AM
Can it record uncompressed?

Yep.
http://www.convergent-design.com/Products/Odyssey7.aspx#Recording-Options-25
Title: Re: 14bit RAW DNG silent pics!
Post by: N.Mendes on April 29, 2013, 08:00:28 AM
Yes, it can record uncompressed, maybe this recorder will be a part of the solution? It will be available this summer (july for the "q" version, and august for the basic one).

The hyperdeck shuttle isn't supposed to record raw, only prores 4:2:2 10bits.. that's probably why you couldn't record raw i guess?

Or give a try again with the official firmware that canon will release tomorrow, the result can be different, i don't know..
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on April 29, 2013, 09:08:54 AM
@Alex and other developers,

From someone who does commercials and corporate videos, all I need is a minute! If you can sustain the dng for 1440 frames we are good to go! If that is difficult then at least 30 seconds, 720 frames. Commercials are just between a minute or 30 seconds in its entirety anyway.

I tested the nightly builds dated 29th and its at 10-11 frames in burst. I'm using a 5DII

I noticed in my test that the 1st recording is ok. No more garbled bottom part of the frame and no more black frame on the left. However on the 2nd recording immediately after the first, I noticed a strong pink cast to the image.

Title: Re: 14bit RAW DNG silent pics!
Post by: ozcancelik on April 29, 2013, 10:34:16 AM
In the future is it possible to preview DNG images from the camera?
Title: Re: 14bit RAW DNG silent pics!
Post by: haeki on April 29, 2013, 10:50:32 AM
Is the datarate limited by internal hardware or by the cf/sd-card?

Maybe there's a hardware workaround, if it's only the card which is limiting? Using some CF-Adapter you can write to ssd with or something?

Anyways, great work so far!
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 10:56:08 AM
Run the benchmarks from the Debug menu to find out.
Title: Re: 14bit RAW DNG silent pics!
Post by: nanomad on April 29, 2013, 11:00:54 AM
Quote from: haeki on April 29, 2013, 10:50:32 AM
Is the datarate limited by internal hardware or by the cf/sd-card?

Maybe there's a hardware workaround, if it's only the card which is limiting? Using some CF-Adapter you can write to ssd with or something?

Anyways, great work so far!

I'd say that the main limit is the memory fragmentation followed by the data rate of the cad. If memory chunks were bigger (16M?) we would reach higher writing speeds.

That being said, if you have a crappy card there's not much you can do (run the bench as a1ex suggested and you'll see what's the theoretical limit of the card/controller combination)
Title: Re: 14bit RAW DNG silent pics!
Post by: haeki on April 29, 2013, 11:55:39 AM
Quote from: nanomad on April 29, 2013, 11:00:54 AM
I'd say that the main limit is the memory fragmentation followed by the data rate of the cad. If memory chunks were bigger (16M?) we would reach higher writing speeds.

That being said, if you have a crappy card there's not much you can do (run the bench as a1ex suggested and you'll see what's the theoretical limit of the card/controller combination)

On my 7D with a 400x Transcend it's 35MB/s... I don't know what's the limiting factor here. I know that the fragmentation is a big factor as well, but if it's not only one factor my thought was that an external ssd which can be plug in the cf-slot via an adapter could be a solution and maybe the chunksize can be improved as well (maybe)... but that was just theoretical thinking without knowing what's possible, that's why I was asking. :-)
Title: Re: 14bit RAW DNG silent pics!
Post by: P337 on April 29, 2013, 01:17:21 PM
OMG A1ex, what did you say to EOSHD?
http://www.eoshd.com/content/10250/canon-5d-2k-raw-feed-update-1920x720-possible-on-1000x-card

btw haeki, a "400x" card should be about 60MB/s but that 400x rating is probably for read speeds, the 7D supports UDMA 7 speeds which would imply it's faster then UDMA 6, which puts its potential max write speed somewhere between 133MB/s and 167MB/s (that's in 1000x territory).  The 7D buffer also supports up to 25 17.9MP 14bit RAW images at about 20MBs per file...  I think your bottleneck is your card's write speed.

(The 5D3 also supports UDMA 7 speeds but with apparently a smaller RAW buffer and bigger RAW files)
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 01:17:41 PM
Exactly what I said in this thread earlier...
Title: Re: 14bit RAW DNG silent pics!
Post by: squig on April 29, 2013, 02:59:10 PM
"[Canon's] image pipeline is probably optimized for fast math, not for high quality."

Now there's the understatement of the year.

When I started working for Apple one of the first things I discovered was all development decisions are made by marketing, not engineering. They don't give a shit about quality.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on April 29, 2013, 04:15:40 PM
on my 5dmkii, when I change from raw+large stepped > just large stepped I'm getting 51 frames, as opposed to 36!
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 29, 2013, 04:55:33 PM
Oh man... the bursts are smooth as butter. The pink cast is easy to get rid of,  its very light... I thought it was from digic black level adjusting but still happens... maybe can adjust that down via digic.
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 05:04:46 PM
Can you post some DNGs with pink cast?

Most probably it's the black level calibration (maybe it varies with time, not sure).
Title: Re: 14bit RAW DNG silent pics!
Post by: critycal on April 29, 2013, 05:11:50 PM
Here you have Alex ;) (http://i41.tinypic.com/lxpy1.jpg)
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 05:25:03 PM
I'd like the raw DNG, not the JPEG.
Title: Re: 14bit RAW DNG silent pics!
Post by: Audionut on April 29, 2013, 05:30:33 PM
https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/more/78430001.DNG
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 05:32:07 PM
Can't see any pink cast here (5500K/green=1).
Title: Re: 14bit RAW DNG silent pics!
Post by: minimimi on April 29, 2013, 05:36:35 PM
I'm tried this function, then I found interesting thing.
You may push Zoom button and take a silent pic.
We can get 3592x1320 DNG.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 29, 2013, 05:43:55 PM
On 600D? My zoom is 1/2 height on 6D. I only made it work there so far.. havn't put my UHS card in to see how well the burst bursts.

BTW.. it uprezes real nicely to 2k.

Do you have offsets already, I can copy and paste vs taking more pics, I think I adjusted 1 only.

https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/src/bda344856cf9096c5f593644e9e0cccae2b75510/src/raw.c


Pink Cast:
http://www.qfpost.com/file/d?g=KpnOVQP3T


The raw viewers sometimes adjust it out like ufraw and stuff.
Title: Re: 14bit RAW DNG silent pics!
Post by: Audionut on April 29, 2013, 05:44:03 PM
Quote from: a1ex on April 29, 2013, 05:32:07 PM
Can't see any pink cast here (5500K/green=1).

In that sample of mine, it makes it to green.  Looks like the correct WB values are not being written to the DNG (along with ISO, shutter, etc etc).

They all load with 6050K/M78 regardless of camera settings.

Here's a shot of a color checker if that makes life easier.
https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/more/78460002.DNG
Title: Re: 14bit RAW DNG silent pics!
Post by: Audionut on April 29, 2013, 05:49:27 PM
Quote from: minimimi on April 29, 2013, 05:36:35 PM
I'm tried this function, then I found interesting thing.
You may push Zoom button and take a silent pic.
We can get 3592x1320 DNG.

Yeah but when it does work, it's just noise.
https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/more/78460003.DNG
Title: Re: 14bit RAW DNG silent pics!
Post by: minimimi on April 29, 2013, 05:54:12 PM
Quote from: 1% on April 29, 2013, 05:43:55 PM
On 600D? My zoom is 1/2 height on 6D. I only made it work there so far.. havn't put my UHS card in to see how well the burst bursts.
On my 5D3 yey!
yes 3592x1320 is 1/2 height. But the picture quality is not so bad , So I think it's a simuler function with 600D's x3 zoom. I mean I guess this is recording pixcel ratio is 1:1 maybe.
I have no idea yet , but I think this is one of hint for clalyfy this function IMHO.

Quote from: Audionut on April 29, 2013, 05:49:27 PM
Yeah but when it does work, it's just noise.
https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/more/78460003.DNG
hmm, My DNG has no noise....
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 06:00:06 PM
I may be color blind, but this doesn't look like pink to me:

(http://a1ex.magiclantern.fm/bleeding-edge/raw/nopink.png)

Ufraw uses the black level computed by ML. If it's wrong, you can't correct it with white balance.

Exif info is not implemented, just some dummy values.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 29, 2013, 06:01:30 PM
Yea, the cast is real easy to remove... but shows up in AE and photoshop. Once you do color balance its perfect..

P.S. its real faint. Change WB and see what I mean.
Title: Re: 14bit RAW DNG silent pics!
Post by: chauffeurdevan on April 29, 2013, 06:17:14 PM
Looked at chdk-dng.c and it is using the old DNG specs 1.3.0.0

It is possible to get LinearRAW jpeg lossy compression in DNG since specs 1.4.0.0 and get a 2:1 to 3:1 compression ratio.

see page.19 http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf (http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf)

You could see some example here : http://blogs.adobe.com/lightroomjournal/2012/10/dng-1-4-specification-notes.html (http://blogs.adobe.com/lightroomjournal/2012/10/dng-1-4-specification-notes.html)

Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 29, 2013, 06:32:01 PM
Oh man.. this would fix it right up... mjpeg anyone?
Title: Re: 14bit RAW DNG silent pics!
Post by: hookah on April 29, 2013, 06:40:15 PM
Little DNG video from 5D3... but there was no sun today, so can not really showoff the latitude.
- just some random shots

Title: Re: 14bit RAW DNG silent pics!
Post by: Vegandelight on April 29, 2013, 07:04:55 PM
Quote from: hookah on April 29, 2013, 06:40:15 PM
Little DNG video from 5D3... but there was no sun today, so can not really showoff the latitude.
- just some random shots

This looks great.

I ordered a 1000x card yesterday to try this stuff out along the way.
Title: Re: 14bit RAW DNG silent pics!
Post by: much on April 29, 2013, 07:14:40 PM
Can someone do overexposured video so we can get the right exposure in post? RAW should be overexposued to get more detail in post ;) I will be glad with some samples to :)
Title: Re: 14bit RAW DNG silent pics!
Post by: Rush on April 29, 2013, 07:15:20 PM
Quote from: hookah on April 29, 2013, 06:40:15 PM
Little DNG video from 5D3... but there was no sun today, so can not really showoff the latitude.
- just some random shots
Trees details is bad :( It is not improvement over stock H.264 video.
Title: Re: 14bit RAW DNG silent pics!
Post by: much on April 29, 2013, 07:16:48 PM
Quote from: Rush on April 29, 2013, 07:15:20 PM
Trees details is bad :( It is not improvement over stock H.264 video.

This is i guess 720p ;)
Title: Re: 14bit RAW DNG silent pics!
Post by: deleted.account on April 29, 2013, 07:19:05 PM
Magenta cast due to choosing too high a sensor saturation level in dcraw.

This link suggests leave dcraw to set black level but specify -S saturation point to suit camera performance.

http://www.guillermoluijk.com/tutorial/dcraw/index_en.htm

Also I'm finding just interpreting as rec709 sRGB primaries in linear domain gives more 'accurate' color than some of the default settings in ufraw, darktable etc which appear much warmer.
Title: Re: 14bit RAW DNG silent pics!
Post by: much on April 29, 2013, 07:22:32 PM
Quote from: hookah on April 29, 2013, 06:40:15 PM
hookah

Could you upload some DNG sample where the girl is?
Title: Re: 14bit RAW DNG silent pics!
Post by: hookah on April 29, 2013, 07:23:47 PM
Quote from: Rush on April 29, 2013, 07:15:20 PM
Trees details is bad :( It is not improvement over stock H.264 video.

Remenber this is encoded 2 times.. First to h264 from premiere, then Vimeo will compress it again.

720 DNG´s looks better than stock h264 at 1080p IMHO
Title: Re: 14bit RAW DNG silent pics!
Post by: hookah on April 29, 2013, 07:30:09 PM
Quote from: much on April 29, 2013, 07:22:32 PM
Could you upload some DNG sample where the girl is?

Yeah sure: (including the famous pink cast)
https://docs.google.com/file/d/0BxHiL_qPbYP0QzlPODJtSkJSWG8/edit?usp=sharing
Title: Re: 14bit RAW DNG silent pics!
Post by: critycal on April 29, 2013, 08:16:45 PM
How do you do the 720p in the MK II?
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 08:18:48 PM
Double-checked the black and white levels, still can't find any pink cast. White balance on the shoulder.

(http://a1ex.magiclantern.fm/bleeding-edge/raw/55660037.jpg)
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on April 29, 2013, 08:25:41 PM
Is it possible to try this out on 6d or only 5d3/2 in current nightly build?
Title: Re: 14bit RAW DNG silent pics!
Post by: Kabuto1138 on April 29, 2013, 08:26:19 PM
Just got my 5d2 to get 51 frames when taking out the raw+jpeg still function.

I'm gonna name my first to Kids after Magic Lantern programers starting with Alex.

This is taking the internet by storm.

Alex, put up again the donating link!!!
Title: Re: 14bit RAW DNG silent pics!
Post by: SlimSpydey on April 29, 2013, 08:30:34 PM
This is awesome guys, well done!

A few people have talked about using 1000x CF cards for this, will anything like this be possible on SD cards?
Title: Re: 14bit RAW DNG silent pics!
Post by: Rush on April 29, 2013, 08:38:01 PM
Quote from: SlimSpydey on April 29, 2013, 08:30:34 PM
A few people have talked about using 1000x CF cards for this, will anything like this be possible on SD cards?
CF (5D2, 5D3, 7D) - up to 150 mb/s
SD UHS-I (6D, 650D, 700D) - up to 90 mb/s
SD Class 10 (550D, 600D, 60D) - up to 22 mb/s

Required badwitch 2.4 mb * 25 = 60 mb/s, so it seems that new cameras will handle this speeds... but 1% mention fragmenting which affects speed, so I'm unsure about UHS-I
Title: Re: 14bit RAW DNG silent pics!
Post by: Vegandelight on April 29, 2013, 08:44:05 PM
Quote from: Rush on April 29, 2013, 08:38:01 PM
CF (5D2, 5D3, 7D) - up to 150 mb/s
SD UHS-I (6D, 650D, 700D) - up to 90 mb/s
SD Class 10 (550D, 600D, 60D) - up to 22 mb/s

Required badwitch 2.4 mb * 25 = 60 mb/s, so it seems that new cameras will handle this speeds... but 1% mention fragmenting which affects speed, so I'm unsure about UHS-I

The 5d3 SD slot maxes out on 133x afaik. Don't forget that people. That's why i bought the CF instead.
Title: Re: 14bit RAW DNG silent pics!
Post by: VlastaS on April 29, 2013, 08:45:49 PM
Hi, 1% mentioned mjpeg, do you think that it could theoretically be possible to compress the DNG stream to get mjpeg video in camera? Even on SD cards like Sandisk Extreme? If you could make it output 2k mjpeg that would be more exciting to me personally than raw. I don't even have to tell you how great you guys are, that's a well known fact!
Title: Re: 14bit RAW DNG silent pics!
Post by: hookah on April 29, 2013, 08:47:24 PM
Quote from: a1ex on April 29, 2013, 08:18:48 PM
Double-checked the black and white levels, still can't find any pink cast. White balance on the shoulder.

(http://a1ex.magiclantern.fm/bleeding-edge/raw/55660037.jpg)

thats right, when you manually set the correct white ballance there is no pink cast.
- Its a metadata ting in the DNG file. The picture data in the DNG file is fine.

Adobe Lightroom see this:
1, 0 Sec exposure (wrong, think it was 1/60)
f/1,0 (Wrong, was 1,4)
Temp 6050  (wrong)
Tint = +78  (wrong, this is where the pink color is comming from)
Iso and focal lenght are missing.
Title: Re: 14bit RAW DNG silent pics!
Post by: nanomad on April 29, 2013, 08:53:33 PM
Again : all exif data is a placeholder. Ignore it
Title: Re: 14bit RAW DNG silent pics!
Post by: hookah on April 29, 2013, 08:58:43 PM
Quote from: nanomad on April 29, 2013, 08:53:33 PM
Again : all exif data is a placeholder. Ignore it

Yeahhh i know.. its no problem  :D
Title: Re: 14bit RAW DNG silent pics!
Post by: rawfa on April 29, 2013, 09:19:26 PM
What a great age we live in! It´s with the help of amazing people like the people we have here that "older" cameras are getting a new life span. Hopefully we´ll see RAW 24p video from the MarkII....or even uncompressed HDMI.
Title: Re: 14bit RAW DNG silent pics!
Post by: aaphotog on April 29, 2013, 10:16:47 PM
Can anyone provide a link with instructions on how to compile one of the nightly builds(with a mac) so that I can install on my 5d3?
Title: Re: 14bit RAW DNG silent pics!
Post by: ch_d on April 29, 2013, 10:45:14 PM
awesome!
here is a dng with zoom *5 with the VAF-5D2b Optical Anti-Aliasing Filter on the 5D MII.
and one without the filter.
the lens: samyang 85mm 1:1,4 on aperture 4.

there is a little bit less aliasing with the anti-aliasing filter (you can see it on the deadcat)
the text is a little bit blurry but less aliasing.

besides: i often get corrupt files. and i always have to boot twice (take out the battery...)

https://dl.dropboxusercontent.com/u/3447700/dngtest_aafilter.DNG (https://dl.dropboxusercontent.com/u/3447700/dngtest_aafilter.DNG)
https://dl.dropboxusercontent.com/u/3447700/dngtest_withoutaafilter.DNG (https://dl.dropboxusercontent.com/u/3447700/dngtest_withoutaafilter.DNG)
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 10:56:03 PM
The VAF filter is too strong for zoom in (it's for plain 1080p).

corrupted files with today's nightly? post an example.

always have to boot twice == did not read the manual?

http://magiclantern.fm/bestpractices
Title: Re: 14bit RAW DNG silent pics!
Post by: ch_d on April 29, 2013, 11:07:46 PM
Quote from: a1ex on April 29, 2013, 10:56:03 PM
The VAF filter is too strong for zoom in (it's for plain 1080p).
yey - i thought of this.

Quote from: a1ex on April 29, 2013, 10:56:03 PM
corrupted files with today's nightly? post an example.
here we go: https://dl.dropboxusercontent.com/u/3447700/05090128.DNG (https://dl.dropboxusercontent.com/u/3447700/05090128.DNG)
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 11:16:02 PM
Uh... that looks like one frame was saved twice.

Was this in end trigger mode?
Title: Re: 14bit RAW DNG silent pics!
Post by: ch_d on April 29, 2013, 11:21:07 PM
Quote from: a1ex on April 29, 2013, 11:16:02 PM
Uh... that looks like one frame was saved twice.

Was this in end trigger mode?

end trigger mode?
Title: Re: 14bit RAW DNG silent pics!
Post by: ch_d on April 29, 2013, 11:25:07 PM
here are the NONE ZOOMED DNGs with and without the AA filter

https://dl.dropboxusercontent.com/u/3447700/05090013withoutAA.DNG (https://dl.dropboxusercontent.com/u/3447700/05090013withoutAA.DNG)

https://dl.dropboxusercontent.com/u/3447700/05090022withAA.DNG (https://dl.dropboxusercontent.com/u/3447700/05090022withAA.DNG)

i´ll make a few motion tests - but looks like the aa filter isn´t needed anymore?
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 29, 2013, 11:29:15 PM
Can you make them at the same focal length and framing?

Aliasing is clearly visible on the non-AA shot.
Title: Re: 14bit RAW DNG silent pics!
Post by: ch_d on April 29, 2013, 11:36:34 PM
Quote from: a1ex on April 29, 2013, 11:29:15 PM
Can you make them at the same focal length and framing?

no - sorry, focal length is changing with the filter attached.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 29, 2013, 11:58:37 PM
QuoteIs it possible to try this out on 6d or only 5d3/2 in current nightly build?

Yup, look in the 6D thread.

I just shot a bunch, will have to look at them. 720P-24fps override = 53 frames.
1080P or zoom modes = like 27-29 frames.

The "1080P" stretches to 2k easily... bet the 720P will stretch to 1080P. Color looks way better than H264.

600D only shot like 16 frames... the first chunk is only 60MB... the next chunk is the big chunk. I think it needs to allocate the largest chunk available on older cameras.
Title: Re: 14bit RAW DNG silent pics!
Post by: woodybrando on April 30, 2013, 12:09:50 AM
This is pretty cool it's like when I first got my super 8 camera and only shot for seconds @ a time. How are people batch processing the raw files. I tried to open up about 400 frames in after effects and got tired of clicking ok on the raw data window.
Thx A1ex and company this is pretty great!
cheers,
Jayson
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 30, 2013, 12:11:57 AM
In CS6 I just import all the files and then create comp... maybe I should only click 1 as it dupes the frames so the sequence repeats, kinda weird but I haven't had to click a single raw window yet.
Title: Re: 14bit RAW DNG silent pics!
Post by: PhotogMike on April 30, 2013, 12:22:51 AM
I noticed a weird thing where as I'm in the ML menu the Date/Time from the Canon menu flashes on my screen. Running 4.29 Nightly Build.

http://hiddenmoonproductions.files.wordpress.com/2013/04/photo.png

Wondering if anyone else has noticed this.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 30, 2013, 12:24:48 AM
This was supposedly the bug that 6D firmware fixed... I'm guessing thats on 5d3 and still not fixed?
Title: Re: 14bit RAW DNG silent pics!
Post by: PhotogMike on April 30, 2013, 12:26:35 AM
Thanks. Sorry. Using a 5D II and I hadn't noticed it until testing the silent pic functions so I didn't know if it was new or not.
Title: Re: 14bit RAW DNG silent pics!
Post by: woodybrando on April 30, 2013, 12:30:26 AM
ok figured it out. Just click import in after effects, then highlight the sequence of dng's and pick import as "Footage" and click "Camera Raw Sequence" box. And it'll let you choose raw settings for the sequence. Also, I'm not seeing the corrupted data anymore that was on the bottom and left side. The whole image is clean now.
Also, getting 49 frames when I choose to shoot raw only and shoot 720p. But it only fills half the frame with a squished image.
This would be great for 5 second films.
:)
Jayson
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 30, 2013, 12:55:31 AM
Other possibility is that time dialog is being used to make the scroll wheel work on 5d2. Seriously need a better fix for this... flashing menu is hell annoying on 6D/EOSM
Title: Re: 14bit RAW DNG silent pics!
Post by: Audionut on April 30, 2013, 02:15:37 AM
Quote from: a1ex on April 29, 2013, 06:00:06 PM
Ufraw uses the black level computed by ML. If it's wrong, you can't correct it with white balance.

Exif info is not implemented, just some dummy values.

I assume Adobe products expect the information to be in the exif.  Hit auto WB and it works it out.  It's no big deal, anyone using DNG's should be PPing anyway.
Title: Re: 14bit RAW DNG silent pics!
Post by: Yoshiyuki Blade on April 30, 2013, 02:40:57 AM
Recorded about 10s of random footage. Applied WB, converted to 8-bit tiff in photoshop's DNG converter, (slightly) stretched and cropped for 1080p and encoded it losslessly. No denoising or sharpening applied.

The 8-bit 4:2:0 is 279 MB while 4:4:4 is 508 MB lol. As expected, the visual difference isn't huge unless I find some bright vivid reds. I'll edit this post later with a link of both clips for you guys to compare.
Here are the lossless encodes for comparison:

Edit: (Links removed due to excessive dropbox traffic causing suspension. :o)

Raw is definitely way overkill for any kind of practical recording, but will there be a possibility for a compromise? The improvement in resolution alone makes it worthwhile, but what happens when the image goes from the raw buffer to YUV 4:2:2 that kills the detail so much? You'll see that even bringing a raw image down to 4:2:0 won't affect the image quality that badly. Is there some kind of filtering applied to increase compressibility at the cost of detail?

Quote from: Kabuto1138 on April 29, 2013, 08:26:19 PM
Just got my 5d2 to get 51 frames when taking out the raw+jpeg still function.

Wow, same here. I had it on raw and was getting 36 frames, but now I get 51 too when I set it to JPEG. Doesn't seem to affect quality in any way. I wonder what happened? :D
Title: Re: 14bit RAW DNG silent pics!
Post by: menoc on April 30, 2013, 04:59:36 AM
Wow . . . wait a minute, Would I be too presumptuous to assume that if this works on the 5D2, this may very well work on the 50D!? Remember, the advantage of the 50D is that it takes CF cards - including the 1000x and has less overhead without audio . . .

Please say it will, Please say it will, Please say it will . . .
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on April 30, 2013, 05:20:32 AM
Reporting my progress with the builds.
Using 5D mkii with a transcend 133x 8gig card, I use this small card for tests since my 32gig cards are used in projects with the stable 2.3.
Builds:
April 28, I could only record around 8-9frames with corrupted bottom of frame and black bar on side, also exposure seem to vary per frame.
April 29, I could already record 11-12 frames and the frames are now clean, exposures stable throughout.
Now, just uploaded,
April 30 build. I was now able to record 102 frames, DNG!

I bet those using the newer mkIII with fast cf cards are getter longer results especially using 720P which the mkII cannot do.

Go for it guys! If we can get at least 15-30 seconds of raw this would be awesome already!
Title: Re: 14bit RAW DNG silent pics!
Post by: hjfilmspeed on April 30, 2013, 06:26:29 AM
Canon just released the firmware (i know it was leaked) But all of a sudden I dont feel the need to update ha ha
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on April 30, 2013, 08:00:10 AM
I have edited my previous post that my test using the april 30 build is not 36 frames but 102 frames dng!

-edit sorry its only 51. Half of 102. I missed the break in the take since the first tests I did was locked down on tripod.
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 30, 2013, 08:06:33 AM
I don't believe it, post them :D
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on April 30, 2013, 08:24:23 AM
I'm editing together a resolution chart test to show if it is an improvement versus h264 and also comparing it with a mosaic filter installed.
Title: Re: 14bit RAW DNG silent pics!
Post by: Pelican on April 30, 2013, 08:57:33 AM
Quote from: P337 on April 29, 2013, 01:17:21 PM
btw haeki, a "400x" card should be about 60MB/s but that 400x rating is probably for read speeds, the 7D supports UDMA 7 speeds which would imply it's faster then UDMA 6, which puts its potential max write speed somewhere between 133MB/s and 167MB/s (that's in 1000x territory).  The 7D buffer also supports up to 25 17.9MP 14bit RAW images at about 20MBs per file...  I think your bottleneck is your card's write speed.
The Transcend 400x 32GB card (which I have) can read more than 100 MB/s (specification says 90 MB/s) and write around 50 MB/s (spec: 60 MB/s) in my card reader. In the 7D the write speed of this Transcend card is only 32 MB/s. The fastest card I've ever tried in my 7D is the Lexar Professional 1000x and I measured 68.2 MB/s write speed, so the 7D is very very far from the speed that you mentioned.
http://pel.hu/down/Lexar1000x7D.bmp
http://pel.hu/down/Transcend400x7D.bmp
Title: Re: 14bit RAW DNG silent pics!
Post by: Pelican on April 30, 2013, 08:58:59 AM
Quote from: PhotogMike on April 30, 2013, 12:22:51 AM
I noticed a weird thing where as I'm in the ML menu the Date/Time from the Canon menu flashes on my screen. Running 4.29 Nightly Build.

http://hiddenmoonproductions.files.wordpress.com/2013/04/photo.png

Wondering if anyone else has noticed this.
Yes, it is a known bug. It happens every minute if you put the Date/time menu item to your custom menu.
Title: Re: 14bit RAW DNG silent pics!
Post by: P337 on April 30, 2013, 10:06:27 AM
Quote from: Pelican on April 30, 2013, 08:57:33 AM
The Transcend 400x 32GB card (which I have) can read more than 100 MB/s (specification says 90 MB/s) and write around 50 MB/s (spec: 60 MB/s) in my card reader. In the 7D the write speed of this Transcend card is only 32 MB/s. The fastest card I've ever tried in my 7D is the Lexar Professional 1000x and I measured 68.2 MB/s write speed, so the 7D is very very far from the speed that you mentioned.
http://pel.hu/down/Lexar1000x7D.bmp
http://pel.hu/down/Transcend400x7D.bmp

Thanks Pelican, that's a little eye opening. 

UDMA 6 spec is speeds up to 133MB/s while UDMA 7 is 167MB/s so I would expect that a Card or Camera that advertises UDMA 7 is surpassing the limits of UDMA 6, but marketing might not see it that way. 

Were you using the newer 7D firmware (1.2.5 I think) that supports faster buffer and UDMA 7 speeds with that 1000x card?  The bottle-neck could also be on the card's max write speed; the fastest benchmarks I could find the Lexar 1000x card is 65.2MB/s for sequential write and a 135MB/s read speed, which I guess technically makes it a UDMA 7.




Title: Re: 14bit RAW DNG silent pics!
Post by: P337 on April 30, 2013, 10:45:18 AM
I'm considering a 1000x card for 14bit DNG, but how many frames are you 1000x card owners getting?

Because A1ex seems to be using a 266x card and mentioned that it took about 30 seconds to clear the buffer of about 30 frames on the 5D3.  Each DNG is about 5.09MBs right?  So 30 would be about 153MBs and if it took 31 seconds to write all that to card that's only like 4.9MB/s and I think Alex's 266x card benchmarked at 20MB/s write so doesn't this point to the frame buffer as the bottle-neck?  If that's right then it wouldn't matter how fast the card is, since the camera is limiting the write speed to 4.9MB/s.

That would mean the only answer to get a sustained 24fps is compression before the "frame buffer" (or increasing the frame buffer speeds) and if the buffer really is limiting speeds to 4.9MB/s it would require a 25:1 compression ratio to fit 24 frames through it.  (5.09*24)/4.9=24.9 

If using Jpeg compression that is very close to a medium level JPEG (23:1 ratio), I don't know much about DNGs Lossy or Lossless compressions but can it do 25:1?  I've only ever seen 3:1 DNG lossy compressions.

http://blogs.adobe.com/lightroomjournal/2012/10/dng-1-4-specification-notes.html

Update: With the camera's photo mode set to jpeg only people are getting 53 frames and about 36 seconds to clear which is about 7.5MB/s and would require a 17:1 compression, similar to a "High Quality" 15:1 JPEG compression.  Ted Ramasola just got 53 frames with a 133x card,  I'm pretty sure the cards are not the bottle-neck here, anything that writes faster than an x50 speed (7.5MB/s) seems to have no benefit for these 14bit DNGs. (unless someone can find a way to drastically increase the in camera's buffer speeds)



Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on April 30, 2013, 10:50:30 AM
Quote from: a1ex on April 30, 2013, 08:06:33 AM
I don't believe it, post them :D

oops after checking my takes its consistent only at 51 frames NOT 102. I missed the break in the takes since the shot was locked down.

51 frames using a 133x cf card on 5d mk2.
Title: Re: 14bit RAW DNG silent pics!
Post by: P337 on April 30, 2013, 10:56:02 AM
Quote from: ted ramasola on April 30, 2013, 10:50:30 AM
oops after checking my takes its consistent only at 51 frames NOT 102. I missed the break in the takes since the shot was locked down.

51 frames using a 133x cf card on 5d mk2.

Mind if I ask how long does it take to write those 51 frames to your card?  And what is your card's write speed benchmarking at in magic lantern?
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on April 30, 2013, 12:37:33 PM
Quote from: P337 on April 30, 2013, 10:56:02 AM
Mind if I ask how long does it take to write those 51 frames to your card?  And what is your card's write speed benchmarking at in magic lantern?

Between 31-36 seconds. Have not tried the benchmarking routine yet.
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on April 30, 2013, 12:39:08 PM
For those who are interested at how it fares on a reschart compared to native h264 .mov and compared to using a mosaic filter and without.

http://www.youtube.com/watch?v=nOuYDAMa9DU&feature=youtu.be
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 30, 2013, 12:42:41 PM
Can you also upload some DNGs and some frames extracted from the video?
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on April 30, 2013, 12:49:26 PM
Quote from: ted ramasola on April 30, 2013, 12:39:08 PM
For those who are interested at how it fares on a reschart compared to native h264 .mov and compared to using a mosaic filter and without.

http://www.youtube.com/watch?v=nOuYDAMa9DU&feature=youtu.be

Well that answers whether or not this hack helps with moire! Cheers :)
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on April 30, 2013, 01:45:09 PM
Quote from: a1ex on April 30, 2013, 12:42:41 PM
Can you also upload some DNGs and some frames extracted from the video?
How can I post dng's here?
I can zip them and email it to you.
Title: Re: 14bit RAW DNG silent pics!
Post by: ch_d on April 30, 2013, 01:50:07 PM
Quote from: ted ramasola on April 30, 2013, 01:45:09 PM
How can I post dng's here?
I can zip them and email it to you.
just put them in the public dropbox folder and post the url here with the "url" tags
Title: Re: 14bit RAW DNG silent pics!
Post by: trsaunders on April 30, 2013, 02:10:27 PM
Quote from: menoc on April 30, 2013, 04:59:36 AM
Wow . . . wait a minute, Would I be too presumptuous to assume that if this works on the 5D2, this may very well work on the 50D!? Remember, the advantage of the 50D is that it takes CF cards - including the 1000x and has less overhead without audio . . .

Please say it will, Please say it will, Please say it will . . .
raw pictures work for 50D, I posted an example earlier in the thread, but burst pictures will need CONFIG_FULL_EXMEM_SUPPORT for 50D, which isn't yet available. I started adding support for 50D in raw.c but I've not found time to work out the correct offsets etc yet, hopefully I'll manage that in the next few days (or someone will beat me to it!).
Title: Re: 14bit RAW DNG silent pics!
Post by: Turtlevanbry on April 30, 2013, 02:45:15 PM
For some reason my photos are 1880x1250 anyone know why?

I was able to get 54 DNG files in Jpeg mode on the 5DMK2 - Max I am able to get is 35 with Raw selected. 

Canon 5dmk2 - duracell 600x CF card
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 30, 2013, 02:51:14 PM
I don't think any Canon engineers are following this thread. They know why ;)
Title: Re: 14bit RAW DNG silent pics!
Post by: squig on April 30, 2013, 03:21:50 PM
So do you think Canon could enable 1920x752 RAW 24p recording even if you guys don't quite get there? It's just a matter of compressing the DNGs now isn't it?

I don't mean to sound pessimistic, I reckon you guys are gonna nail it.
Title: Re: 14bit RAW DNG silent pics!
Post by: Primary on April 30, 2013, 03:46:27 PM
Could we record video with 24fps, but only 2-3 seconds?
Title: Re: 14bit RAW DNG silent pics!
Post by: coutts on April 30, 2013, 04:00:29 PM
Quote from: squig on April 30, 2013, 03:21:50 PM
So do you think Canon could enable 1920x752 RAW 24p recording even if you guys don't quite get there? It's just a matter of compressing the DNGs now isn't it?

I don't mean to sound pessimistic, I reckon you guys are gonna nail it.
the current bottleneck for everything is sd/cf card technology, cards aren't fast enough yet
Title: Re: 14bit RAW DNG silent pics!
Post by: DjJuvan on April 30, 2013, 04:45:45 PM
What about using RAID technology on 5dmk3 with CF and SD at the same time? It could give some boost.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 30, 2013, 05:43:30 PM
The only way out of the writing problem is to compress the frames to around 1MB. Canon doesn't need to follow the thread, they already made MJPEG on other cameras.
Title: Re: 14bit RAW DNG silent pics!
Post by: chauffeurdevan on April 30, 2013, 06:21:47 PM
Quote from: 1% on April 30, 2013, 05:43:30 PM
The only way out of the writing problem is to compress the frames to around 1MB. Canon doesn't need to follow the thread, they already made MJPEG on other cameras.

Just did a conversion to DNG Lossy using Adobe DNG Converter 7.4 and I got file size between 907k and 1136kb instead of 4965kb.
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on April 30, 2013, 06:24:45 PM
Good luck doing that in camera ;)
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 30, 2013, 06:34:59 PM
Can't make a module with TCC to do it or not enough processing power? Or maybe using the JPEG encoder? It should take both raw input and YUV since it does that for photos/eos utility preview.

*imo... that file size is enought to write continuously.
Title: Re: 14bit RAW DNG silent pics!
Post by: VlastaS on April 30, 2013, 07:37:16 PM
Quote from: a1ex on April 30, 2013, 06:24:45 PM
Good luck doing that in camera ;)

But could it be done in camera? Either to Lossy DNG or mjpeg. Do you think there is enough processing power?
Title: Re: 14bit RAW DNG silent pics!
Post by: menoc on April 30, 2013, 07:53:01 PM
Quote from: trsaunders on April 30, 2013, 02:10:27 PM
raw pictures work for 50D, I posted an example earlier in the thread, but burst pictures will need CONFIG_FULL_EXMEM_SUPPORT for 50D, which isn't yet available. I started adding support for 50D in raw.c but I've not found time to work out the correct offsets etc yet, hopefully I'll manage that in the next few days (or someone will beat me to it!).

Cheers! You guys rock!
Title: Re: 14bit RAW DNG silent pics!
Post by: g3gg0 on April 30, 2013, 09:06:45 PM
Quote from: VlastaS on April 30, 2013, 07:37:16 PM
But could it be done in camera? Either to Lossy DNG or mjpeg. Do you think there is enough processing power?
not in real time at all
Title: Re: 14bit RAW DNG silent pics!
Post by: woodybrando on April 30, 2013, 09:32:53 PM
Seems like the only way it could work for more than a second is maybe if it can be subsampled from 14bit to 8bit and sent out the HDMI. are either of those things possible?
Title: Re: 14bit RAW DNG silent pics!
Post by: g3gg0 on April 30, 2013, 09:45:08 PM
short: nope, thats not possible.

but we implemented raw recording already.
see http://www.magiclantern.fm/forum/index.php?topic=5247.msg32533;topicseen#msg32533

its still experimental and not ready for release. but it will show you what is coming next.
Title: Re: 14bit RAW DNG silent pics!
Post by: Rush on April 30, 2013, 09:47:52 PM
One of the possibilities of RAW - you can get rid of most color moire with Lightroom! :D
(http://s2.ipicture.ru/uploads/20130501/nMA6pk6V.png)
Title: Re: 14bit RAW DNG silent pics!
Post by: N.Mendes on April 30, 2013, 09:52:59 PM
i'm not sure if that has been said before but, with the new 5d3 firmware (which allow an official hdmi output), did you try to record raw? With an atamos ninja let's say to an "SSD" ...
Title: Re: 14bit RAW DNG silent pics!
Post by: DjJuvan on April 30, 2013, 09:57:31 PM
You're saying as if there is Atomos Ninja2 laying in everyone homes.... get a grip Ok?  ???
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on April 30, 2013, 11:09:05 PM
Some raw DNG tests:

Title: Re: 14bit RAW DNG silent pics!
Post by: g3gg0 on April 30, 2013, 11:37:15 PM
deshaker for virtualdub is very good when the video is shaky :)
Title: Re: 14bit RAW DNG silent pics!
Post by: wolf on April 30, 2013, 11:43:45 PM
Thanks for sharing.
At 29 sec (second subway shot) there is a  change in exposure. Can you tell how this occurred?
And no more 99'er on wall street ;-)
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 01, 2013, 12:01:59 AM
The expo change is from fps override. Yea.. I need to deshake (in ae or Vdub) and probably fix the black bars on some clips. Maybe I'll get longer ones from lverc now.
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 01, 2013, 12:13:43 AM
Could you please tell us what camera is it and what memory card did you use? Film is made of about 2 seconds clips so ~50 frames per each fragment am i right?
Title: Re: 14bit RAW DNG silent pics!
Post by: ch_d on May 01, 2013, 12:17:20 AM
Quote from: 1% on April 30, 2013, 11:09:05 PM
Some raw DNG tests:



are this tests with 5*zoom? rolling shutter as hell in this mode... you can´t get rid of this bad rolling shutter issues.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 01, 2013, 12:35:06 AM
Some zoom, some 720P24, some 1080P

The 720ps might have gotten rolling shutter depending on which override mode I used.

6D with 64gb patriot EP card. Canon EF 35-135
Title: Re: 14bit RAW DNG silent pics!
Post by: hirethestache on May 01, 2013, 01:59:26 AM
Quote from: DjJuvan on April 30, 2013, 09:57:31 PM
You're saying as if there is Atomos Ninja2 laying in everyone homes.... get a grip Ok?  ???

There's no need for that. He's only asking because he, like quite a few of us, would find great use with this and it would truly turn our cameras into workhorses, and give BMCC a run for its money...

Because I too was curious about this, but am not a developer by any means, would anyone explain in layman's terms why recording through HMDI out would not be feasible...you know, for us dumb monkeys ;)
Title: Re: 14bit RAW DNG silent pics!
Post by: Kenjee on May 01, 2013, 04:20:00 AM
Idea.

Raw > gamma curve (flat out) > Jpeg (or 422)

?
Title: Re: 14bit RAW DNG silent pics!
Post by: P337 on May 01, 2013, 04:23:58 AM
Quote from: hirethestache on May 01, 2013, 01:59:26 AM
Because I too was curious about this, but am not a developer by any means, would anyone explain in layman's terms why recording through HMDI out would not be feasible...you know, for us dumb monkeys ;)

You mean for 14bit raw?

Basically Canon's HDMI out can't display an image with that much detail.

It's limited to 8bit (which is a huge leap down from 14bit) and HDMI can't display RAW images, it needs to be 'debayered' (reordering the raw pixels into an actual image) and 'sub-sampled' (that means deleting pixels then filling those gaps with clones of the left over pixels) also I think the HDMI spec used in Canon DSLRs don't support anything over 1920x1080 so it would need to be resized or cropped too.

RAW images, are really just a mess of colored pixels until something "debayers" it into the actual image; which is either done in Canon's built in JPEG converter, H.264 converter or LiveView converter (which goes to the HDMI and LCD).

However Canon's built in CR2 converter (for RAW photos) or this added DNG converter (which I think still uses Canon's built in CR2 converter but just sticks it into a DNG container) do not "debayer" so the RAW 'pixel mess' is just saved, the debayer is done (usually automatically) by your computer's picture viewer or image editor.
Title: Re: 14bit RAW DNG silent pics!
Post by: aaphotog on May 01, 2013, 05:24:54 AM
Where can I find the latest nightly release so that I may compile and load up to my 5d3?
Title: Re: 14bit RAW DNG silent pics!
Post by: P337 on May 01, 2013, 05:36:18 AM
Quote from: Kenjee on May 01, 2013, 04:20:00 AM
Idea.

Raw > gamma curve (flat out) > Jpeg (or 422)

?

I think that's what this is doing:
http://www.magiclantern.fm/forum/index.php?topic=5247.0

RAW/DNG = 14bit 4:4:4 uncompressed
LiveView(HDMI)/YUV422 Recorder(?) = 8bit 4:2:2 uncompressed
JPEG/"M"-JPEG = 8bit 4:2:2 compressed
H.264 = 8bit 4:2:0 compressed

looks like the uncompressed YUV422 recorder is going to be the answer for video here but is it better then the clean uncompressed HDMI out? (other then the fact you don't need an external recorder)
Title: Re: 14bit RAW DNG silent pics!
Post by: hirethestache on May 01, 2013, 08:46:02 AM
Quote from: P337 on May 01, 2013, 04:23:58 AM
You mean for 14bit raw?

Basically Canon's HDMI out can't display an image with that much detail.

It's limited to 8bit (which is a huge leap down from 14bit) and HDMI can't display RAW images, it needs to be 'debayered' (reordering the raw pixels into an actual image) and 'sub-sampled' (that means deleting pixels then filling those gaps with clones of the left over pixels) also I think the HDMI spec used in Canon DSLRs don't support anything over 1920x1080 so it would need to be resized or cropped too.

RAW images, are really just a mess of colored pixels until something "debayers" it into the actual image; which is either done in Canon's built in JPEG converter, H.264 converter or LiveView converter (which goes to the HDMI and LCD).

However Canon's built in CR2 converter (for RAW photos) or this added DNG converter (which I think still uses Canon's built in CR2 converter but just sticks it into a DNG container) do not "debayer" so the RAW 'pixel mess' is just saved, the debayer is done (usually automatically) by your computer's picture viewer or image editor.

Awesome, that makes it a world clear! I may be wrong, but the uncompressed clean HDMI output is 8bit 4:2:2@1080...

Would I be wrong in saying that 8bit DNG would still be higher-quality? Yes, it is undoubtedly lesser quality than 14 bit, but this is one of those "take what you can get" situations.
Title: Re: 14bit RAW DNG silent pics!
Post by: P337 on May 01, 2013, 10:05:27 AM
Quote from: hirethestache on May 01, 2013, 08:46:02 AM
Awesome, that makes it a world clear! I may be wrong, but the uncompressed clean HDMI output is 8bit 4:2:2@1080...

Would I be wrong in saying that 8bit DNG would still be higher-quality? Yes, it is undoubtedly lesser quality than 14 bit, but this is one of those "take what you can get" situations.

No the best part of the DNG is the 14bit color but I don't think the DNG or CR2 converter in Canon's hardware can do lossy compressions anyway, just a locked in Lossless 14bit 4:4:4 1.25:1 compression.  What the Magic Lantern guys are doing in the YUV422 uncompressed recorder is our best bet,  I would expect similar quality to recording the clean HDMI out that Canon just gave us for 5D3, except without the need of requiring a $1000 external HDMI recorder and hopefully they can include sound :)  (yeah Canon added uncompressed video but REMOVED audio from the HDMI out now!)
Title: Re: 14bit RAW DNG silent pics!
Post by: mucher on May 01, 2013, 11:16:02 AM
Honestly, I was wonding if it was possible to feed the buffer into the "crappy" hardware h.264 encoder, it might get a better result than the current one.

One ideal resolution is asking Canon to develop a new video format by adding jpg files together and, with a timeline built in, making it a video, then the 7D's two DIGIC4 processor can be used, which might be able to make it fast enough for 2k@24fps.

Just thinking out loud.
Title: Re: 14bit RAW DNG silent pics!
Post by: Kenjee on May 01, 2013, 01:33:58 PM
Quote from: P337 on May 01, 2013, 05:36:18 AM
I think that's what this is doing:
http://www.magiclantern.fm/forum/index.php?topic=5247.0

RAW/DNG = 14bit 4:4:4 uncompressed
LiveView(HDMI)/YUV422 Recorder(?) = 8bit 4:2:2 uncompressed
JPEG/"M"-JPEG = 8bit 4:2:2 compressed
H.264 = 8bit 4:2:0 compressed

looks like the uncompressed YUV422 recorder is going to be the answer for video here but is it better then the clean uncompressed HDMI out? (other then the fact you don't need an external recorder)

My only idea is to use raw dinamic range and flat it out (gamma curve) to some of the possible formats (jpeg,422)
I think that cinestyle and others curves do that only after conversion in 8 bit space, or I`m wrong?

Title: Re: 14bit RAW DNG silent pics!
Post by: CaptainHook on May 01, 2013, 02:04:56 PM
Quote from: P337 on May 01, 2013, 10:05:27 AMWhat the Magic Lantern guys are doing in the YUV422 uncompressed recorder is our best bet,  I would expect similar quality to recording the clean HDMI out that Canon just gave us for 5D3

I would hope they can improve it!
I did some testing today of the 5D3 hdmi out and compared to the internal codecs, and the BMCC:

Title: Re: 14bit RAW DNG silent pics!
Post by: hirethestache on May 01, 2013, 08:15:43 PM
Quote from: CaptainHook on May 01, 2013, 02:04:56 PM
I would hope they can improve it!
I did some testing today of the 5D3 hdmi out and compared to the internal codecs, and the BMCC:



Hard to see what's going on in my tiny iphone5 screen, but what are your impressions on the new firmware?
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 01, 2013, 09:36:56 PM
Wow, 5d3 really soft compared to BMCC.

The raw DNG looks like BMCC dng tho.
Title: Re: 14bit RAW DNG silent pics!
Post by: CaptainHook on May 01, 2013, 10:23:29 PM
Quote from: hirethestache on May 01, 2013, 08:15:43 PM
Hard to see what's going on in my tiny iphone5 screen, but what are your impressions on the new firmware?

I wrote my thoughts in the description on vimeo, i don't want to go too off-topic here. :)

Quote from: 1% on May 01, 2013, 09:36:56 PM
The raw DNG looks like BMCC dng tho.

And that's why lots of us are hoping you guys can get continuous recording at some kind of decent resolution, or somehow get it uncompressed to the HDMI out (or the YUV422 to the hdmi out if it looks better than canons version!) to breathe new life into the camera. Great work you're all doing. :)
Title: Re: 14bit RAW DNG silent pics!
Post by: Mati on May 02, 2013, 12:04:14 AM
I am on my Dell 2412m I see a big difference between 5dMKIII, and BMCC, the + BMCC. I wonder what they see on the Eizo .... ? :)
Title: Re: 14bit RAW DNG silent pics!
Post by: hirethestache on May 02, 2013, 12:22:36 AM
Dont want to detract too much from the development talk, but I'd like to donate some moolah to the core team--do you guys still have a method for accepting donations? Last I was able, was when we donated for early release of 2.3...
Title: Re: 14bit RAW DNG silent pics!
Post by: VectorZ on May 02, 2013, 02:24:28 AM
Does SRaw mode do anything to help this situation?

Theoretically it could reduce your resolution sampling, or perhaps offer some alternate compression scheme?
Title: Re: 14bit RAW DNG silent pics!
Post by: P337 on May 02, 2013, 02:39:00 AM
Quote from: CaptainHook on May 01, 2013, 10:23:29 PM
I wrote my thoughts in the description on vimeo, i don't want to go too off-topic here. :)

And that's why lots of us are hoping you guys can get continuous recording at some kind of decent resolution, or somehow get it uncompressed to the HDMI out (or the YUV422 to the hdmi out if it looks better than canons version!) to breathe new life into the camera. Great work you're all doing. :)

I just saw a review on the CanonRumors forum(i think) that compared the HDMI out quality before the firmware update and after, the after looked slightly sharper.  Maybe Canon is doing less resizing for the HDMI out now, maybe they're taking a crop from (what I assume) the original 2K 14bit RAW sensor scan for video found by magic Lantern then compressing it down to 4:2:2 8bit and sending it out HDMI.  Maybe that's the best continuous video we can get from these cameras.

1%, in the YUV422 recorder thread did you guys find a way to write to the card faster?  I read 10MB/s faster for round integers and bigger blocks which resulted in double the frames before the recording stopped (400 frames at 24fps/16 seconds I think it was).  COuld that be applied to the 14bit DNGs?  Would we get a few more seconds of recording that way?
Title: Re: 14bit RAW DNG silent pics!
Post by: P337 on May 02, 2013, 02:48:53 AM
Quote from: VectorZ on May 02, 2013, 02:24:28 AM
Does SRaw mode do anything to help this situation?

Theoretically it could reduce your resolution sampling, or perhaps offer some alternate compression scheme?

I don't think so, the "usable image" is already just about 1080p so sRAW might shrink that too far, like 240p (I haven't done the math though). 

They need is a way to compress in real time while saving the resolution where it is at
or a way to crop the image in real time
or a way to increase buffer speeds
or a way to increase buffer capacity
actually all of the above

they're working hard looking for an answer but I wouldn't expect continuous 14bit DNG video, honestly I'd just be happy with 10 seconds (which is already asking a lot from this hardware).

Though I wonder, are the 14bit DNGs are using the same "pipe" that the Camera uses for writing CR2 files to card?  What if they were converted to CR2 rather then DNG?  Would that allow them to use a bigger or faster buffer?  Probably not though cause I assume CHDK's DNG converter is already using Canon's built in CR2 converter.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 02, 2013, 02:53:10 AM
Maybe tweaking exmem a little would help... I too would be happy with like 5 or 10 seconds. Its getting pretty close. Also somehow writing at optimum size for the memory card would help too.



The LV rec is now recording dng and it is twice as fast already... have to figure out the files to see what they look like.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on May 02, 2013, 04:25:46 AM
is LV rec, the 5d3 update, or is it in my mkii ml?
Title: Re: 14bit RAW DNG silent pics!
Post by: Primary on May 02, 2013, 05:47:35 AM
What is the biggest resolution on 5Dmkii with this method? I wouldn't make 24p video (I haven't got fast CF cards), i would capture 1 frame in every 1 second for timelapse.
Title: Re: 14bit RAW DNG silent pics!
Post by: mucher on May 02, 2013, 06:05:50 AM
Another thought. If the developers can add silent pic to 7D, and take the screen dump like 24fps, then I can later convert the dump to jpg, and use jpg to create an avi file, then import to software to process(I can crop the clip into 16:9 in video editing software I hope) -- so all the developers need to do is to add a 24 fps interval timer. Or...port the dump the silent video to h.264, that will save me some work.

The current 7D video is so bad, which doesn't match with its jpg capture very much in many ways(black point, white point,...)

I would mind to donate money for that function because it can enlever my 7D's video capture capability.
Title: Re: 14bit RAW DNG silent pics!
Post by: Francis on May 02, 2013, 07:43:28 AM
Quote from: Primary on May 02, 2013, 05:47:35 AM
What is the biggest resolution on 5Dmkii with this method? I wouldn't make 24p video (I haven't got fast CF cards), i would capture 1 frame in every 1 second for timelapse.

2K-ish is the resolution. It is perfect for time lapse. Not so perfect for video dreamers who immediately expect 2K raw @ 60FPS.
Title: Re: 14bit RAW DNG silent pics!
Post by: P337 on May 02, 2013, 08:53:38 AM
Quote from: Francis on May 02, 2013, 07:43:28 AM
2K-ish is the resolution. It is perfect for time lapse. Not so perfect for video dreamers who immediately expect 2K raw @ 60FPS.

The biggest I got was 1880 x 1248, that was in 1080p mode without a 5x or 10x zoom, not sure how big the zoom mode would be, I know it's wider but shorter though.
Title: Re: 14bit RAW DNG silent pics!
Post by: squig on May 02, 2013, 09:12:14 AM
Looks like the Transcend 1000x is faster than the Lexar. http://www.fredmiranda.com/forum/topic/1202766 They're cheaper too, B&H are selling the 64Gb for $200.
Title: Re: 14bit RAW DNG silent pics!
Post by: domisol on May 02, 2013, 11:47:22 AM
Such great work. Thanks.
Using last nightly build (May02) on 5D2 with 600x Transcend CF.
Works great but sometimes a bad frame.
in case this helps I uploaded the bad frame here :
http://cembalo.free.fr/ml/87840019.DNG (http://cembalo.free.fr/ml/87840019.DNG)
as a reference eventualy good frame is here :
http://cembalo.free.fr/ml/87840020.DNG (http://cembalo.free.fr/ml/87840020.DNG)

Jean-David
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on May 02, 2013, 03:34:04 PM
will someone please direct me to LV rec on the mkii
Title: Re: 14bit RAW DNG silent pics!
Post by: Alia5 on May 02, 2013, 04:05:25 PM
on my 600d raw silent pics are a bit glitchy...
using them with fps ovverride at 7fps
sometimes the frames are 2046x1154 and completely broken

then again, sometimes i get working dngs in size 1740x1154

using 1% autoexe.bin from tragiclantern and did not compile myself tough
Title: Re: 14bit RAW DNG silent pics!
Post by: ch_d on May 02, 2013, 04:06:21 PM
Quote from: shuggyrasklat on May 02, 2013, 03:34:04 PM
will someone please direct me to LV rec on the mkii
Install ML (the nightly builds).
Go into the menu and set the silent pictures to "burst mode".
Turn on LV and hit the shutter halfways and fill your buffer with the dngs.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on May 02, 2013, 04:54:23 PM
sorry, didnt realize, that was the name you coined it..hasnt it been called, what it is, "silent picture, burst mode?"
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 02, 2013, 05:31:25 PM
Quotesometimes the frames are 2046x1154 and completely broken

Its from the less than stellar sync. 5DII has the same problem.

I tried for hours to get a find a good sync and have come up with nothing... everything updates 2 or 3 times per frame or not enough... in fact I can sync it to where MZ is not visible very easily but never to where it doesn't flicker and you can see it. PS. This is why shrinking the message queue worked a little bit.

When no sync is enabled it will use msleep which isn't accurate. Try a different fps if you're getting lots of duds.
Title: Re: 14bit RAW DNG silent pics!
Post by: Alia5 on May 02, 2013, 06:20:15 PM
Why don't use a frameskip formula instead of using FPS-ovverride...

This way we could set like a frequency for silent pics in burst mode while the LV still refreshes with 24 oder 30 fps

For me this would be a whole lot more comfortable, also i havent had any problems when not using fps override
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 02, 2013, 06:27:04 PM
You can use the frame skip, its in LV rec. For dng I don't know, until I figure out a way to containerize the files its pointless and stuck using burst mode.
Title: Re: 14bit RAW DNG silent pics!
Post by: tob on May 02, 2013, 08:08:14 PM
Quote from: squig on May 02, 2013, 09:12:14 AM
Looks like the Transcend 1000x is faster than the Lexar. http://www.fredmiranda.com/forum/topic/1202766 They're cheaper too, B&H are selling the 64Gb for $200.

I don't see any test between lexar 1000x and transcend 1000x?
Title: Re: 14bit RAW DNG silent pics!
Post by: dude on May 02, 2013, 09:44:02 PM
On the Lexar i had like 93 mb/s
Title: Re: 14bit RAW DNG silent pics!
Post by: minimimi on May 03, 2013, 03:54:10 AM
How to calculate saved .yuv fps?

Modified
I'm sorry , I had a mistaken to write another board....
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 03, 2013, 04:16:42 AM
Lol, yea, thats a problem... I try to use 24fps if i'm taking sequential shots.

With frame skip I guess it gets messier. I guess 2 would be 12fps? Estimating usually works but if you looked at the lvrec test video, sometimes it speeds up.

Title: Re: 14bit RAW DNG silent pics!
Post by: Yoshiyuki Blade on May 03, 2013, 07:14:17 AM
There was some breezy weather a couple days ago so I took some shots of tree leaves swaying around as well as other things with high frequency detail (which would fall apart quickly if the bitrate isn't high enough). I'm still constantly amazed by how much better the quality is! This is the way it was meant to be recorded! The fact that my computer can't even decode it in real-time at lossless 4:2:0 shows that the complexity is quite high.

The lossless 4:4:4 file was nearly 2 GB so I won't upload it (my free dropbox account is limited to 2GB of bandwidth daily), so I arbitrarily compressed it to BD specs. I also had an IT8 target for my flatbed scanner in there. I didn't use it to help with the video or anything like that.
https://dl.dropboxusercontent.com/u/63799907/test%20BD.mkv
Title: Re: 14bit RAW DNG silent pics!
Post by: Andy600 on May 03, 2013, 09:09:15 AM
Very impressive Yoshiyuki. I'm getting similar (but shorter length) results on the 600d. I'm amazed at the color rendition and lack of artifacts
Title: Re: 14bit RAW DNG silent pics!
Post by: sicetime on May 03, 2013, 05:19:23 PM
Quote from: Yoshiyuki Blade on May 03, 2013, 07:14:17 AM

https://dl.dropboxusercontent.com/u/63799907/test%20BD.mkv

This link is dead, anyway we can get that on vimeo somehow? Or some other medium?
Title: Re: 14bit RAW DNG silent pics!
Post by: jonni on May 03, 2013, 05:44:06 PM
Actual link: http://data.cod.ru/170691 (press "скачать файл")
Title: Re: 14bit RAW DNG silent pics!
Post by: EOSHD on May 03, 2013, 05:54:26 PM
Quote from: squig on May 02, 2013, 09:12:14 AM
Looks like the Transcend 1000x is faster than the Lexar. http://www.fredmiranda.com/forum/topic/1202766 They're cheaper too, B&H are selling the 64Gb for $200.

I've had good results with 1000x KomputerBay, even cheaper than Transcend. Was flawless in 1D C at 4K MJPEG, and that was 100MB/s to the card. Not a single dropped frame or buffer full alert. The memory in these cards probably all comes from the same manufacturer anyway!
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 03, 2013, 07:01:04 PM
Quote from: Yoshiyuki Blade on May 03, 2013, 07:14:17 AM
There was some breezy weather a couple days ago so I took some shots of tree leaves swaying around as well as other things with high frequency detail (which would fall apart quickly if the bitrate isn't high enough). I'm still constantly amazed by how much better the quality is! This is the way it was meant to be recorded! The fact that my computer can't even decode it in real-time at lossless 4:2:0 shows that the complexity is quite high.

The lossless 4:4:4 file was nearly 2 GB so I won't upload it (my free dropbox account is limited to 2GB of bandwidth daily), so I arbitrarily compressed it to BD specs. I also had an IT8 target for my flatbed scanner in there. I didn't use it to help with the video or anything like that.
https://dl.dropboxusercontent.com/u/63799907/test%20BD.mkv


Yoshiyuki Blade, I assume that this   is from 5d2 (right?) so how many frames per burst did you captured ?? Was your movie made from 1080p or 720p clips. What card did you use with camera?
Title: Re: 14bit RAW DNG silent pics!
Post by: sicetime on May 03, 2013, 07:50:18 PM
Quote from: jonni on May 03, 2013, 05:44:06 PM
Actual link: http://dl.dropboxusercontent.com/u/72405239/test%20BD.mkv

OH MY GOD THAT LOOKS INCREDIBLE!!!!!!!!!!

What amazing work, seriously breathtaking.

second that on asking what settings you used Yoshiyuki
Title: Re: 14bit RAW DNG silent pics!
Post by: DTSET123 on May 03, 2013, 08:04:14 PM
QuoteI've had good results with 1000x KomputerBay, even cheaper than Transcend. Was flawless in 1D C at 4K MJPEG, and that was 100MB/s to the card. Not a single dropped frame or buffer full alert. The memory in these cards probably all comes from the same manufacturer anyway!

Good price! How much does it bench in ML? How many DNGs did you get with it?
Title: Re: 14bit RAW DNG silent pics!
Post by: dfsgfeiuhd on May 03, 2013, 08:08:06 PM
Wow! I am excited about the feature, the footage looks amazing! This is HUGE!

I downloaded the nightly build and will test the feature on my 600d in the weekend. I briefly tried it yesterday night but the camera recorded 422 files in burst mode. However, one user here mentioned the 600d so I was wondering how to set it up (which settings to choose) to get the raw recording going.

Cheers!
Title: Re: 14bit RAW DNG silent pics!
Post by: sicetime on May 03, 2013, 08:16:32 PM
So i want to test this out on my 5d mk iii, but the nightly builds don't include that in the list of cams, am I missing something, I'm sorry I'm sure this gets asked a lot, should I be using tragic lantern?
Title: Re: 14bit RAW DNG silent pics!
Post by: Yoshiyuki Blade on May 03, 2013, 08:25:54 PM
Quote from: kgv5 on May 03, 2013, 07:01:04 PM

Yoshiyuki Blade, I assume that this   is from 5d2 (right?) so how many frames per burst did you captured ?? Was your movie made from 1080p or 720p clips. What card did you use with camera?

Each segment is about 51 frames, so a little over 2 seconds @ 24 fps. They were made from the raw DNGs so they were originally 1880x1250. I stretched it horizontallly to 1920 and cropped it to 1080. The card is a Sandisk Extreme Pro 16GB 90MB/s, though I doubt that matters much in raw burst mode. It seems that all the frames are captured in memory, then it takes its time writing to the card. Real-time RAW capture seems WAY off in that sense. It takes maybe 20x longer to write all the frames.

Also, my public sharing limit for dropbox was reached again, haha. I guess I can try hosting it somewhere else, like Mega if you guys think it's a good idea. I'm not a fan of uploading to a video service (youtube, vimeo, etc.) since they recompress the files.

Quote from: sicetime on May 03, 2013, 07:50:18 PM
OH MY GOD THAT LOOKS INCREDIBLE!!!!!!!!!!

What amazing work, seriously breathtaking.

second that on asking what settings you used Yoshiyuki

I simply opened the DNGs in photoshop, did the click-to-white-balance on the reference you saw at the end and boosted the saturation a tiny bit. There were also some default options (brightness, contrast, sharpening, color noise reduction) which I left on. I'm not a pro at color grading or post processing so I left a lot of stuff alone.
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 03, 2013, 09:01:21 PM

How did you managed to get over 50 frames in such resolution? With 6d in 1280x720p i am getting 1840x632 (and yes, usually 53 frames), but when i set 1920x1080 i get 1840x1221 and not more than 29 frames. I have sandisk extreme 45 MB/s but i think, that in this moment it doesnt matter, everything probably is about buffer size...
Title: Re: 14bit RAW DNG silent pics!
Post by: Yoshiyuki Blade on May 03, 2013, 09:11:14 PM
As someone mentioned earlier in the thread, if you set the picture quality to JPEG instead of RAW in the default camera menu, you may get some more frames out of it. That's how I went from 36 frames to 51 with the 5D2. Not sure if it works on the 6D too, but it's worth a try!
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 03, 2013, 09:44:18 PM
Thanks,  i have checked and unfortunatelly it doesnt work. No, i said wrong, actually it does work but with RAW  (doesnt matter which raw, mraw, sraw)+jpeg all i have is 15 frames and with RAW off - JPEG only - up to 29 frames  :-\  It seems, that 5d2 has more buffer space or maybe there is some other settings...
Title: Re: 14bit RAW DNG silent pics!
Post by: DTSET123 on May 03, 2013, 10:01:11 PM
QuoteAs someone mentioned earlier in the thread, if you set the picture quality to JPEG instead of RAW in the default camera menu, you may get some more frames out of it. That's how I went from 36 frames to 51 with the 5D2. Not sure if it works on the 6D too, but it's worth a try!

It does! I just jumped from 29 to 53 dng in 720p (1840x632) on 6d. I have the Sandisk Extreme 128Gb 45MB
I set jpeg to S and turned off the RAW in canon menu
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 03, 2013, 10:08:30 PM
Yes, as i said, in 720p i also have 53 frames, unfortunatelly in 1080 those 29 frames are totally unusable...
Title: Re: 14bit RAW DNG silent pics!
Post by: CFP on May 03, 2013, 10:29:35 PM
Quote from: dfsgfeiuhd on May 03, 2013, 08:08:06 PM
I downloaded the nightly build and will test the feature on my 600d in the weekend. I briefly tried it yesterday night but the camera recorded 422 files in burst mode. However, one user here mentioned the 600d so I was wondering how to set it up (which settings to choose) to get the raw recording going.
At first you need to put the files from this link onto your SD Card: https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads

Just replace your current "AUTOEXEC.bin" with that new one, and drag & drop the "MODULES" folder into your ML folder. And rename your "scripts" folder to "SCRIPTS".

Than you can either use the silent pictures to record "RAW video" or the new module - but that crashes all the time on my 600D ...

For the silent pictures, I've got the best results in 720p 60 fps mode with "FPS override" set to 10, 24 or 30 fps and the picture size set to "sRAW" inside the ML menu.

I'm able to get 37 frames in 1740 X 692 and 22 frames in 1740 X 1154 ... Not that much (~ 1s footage), but the image quality is just AMAZING! Way better than H.264.

So, thank you guys! You are amazing! I'm really looking forward to your next progress :)
Title: Re: 14bit RAW DNG silent pics!
Post by: killr0y on May 03, 2013, 10:39:30 PM
Quote from: Yoshiyuki Blade on May 03, 2013, 08:25:54 PM
Also, my public sharing limit for dropbox was reached again, haha. I guess I can try hosting it somewhere else, like Mega if you guys think it's a good idea. I'm not a fan of uploading to a video service (youtube, vimeo, etc.) since they recompress the files.

Please do!  I'd like to see it even if it gets a bit mangled by youtube or vimeo. :P
Title: Re: 14bit RAW DNG silent pics!
Post by: aaphotog on May 04, 2013, 12:29:18 AM
So, I just tried shooting the raw pics(using sd card gets me about 25 frames)
Tried on the small jpeg like posted and got 50 frames. loaded it up, and the preview that the mac shows me is a pink tint(was doing this previously but white balance fixed it) but when I load into after effects or lightroom 4, all i see is black and green and white balance changes NOTHING!
Thinking it was because I was shooting JPG, I went back to small raw(getting 25 frames) and still the same thing(finder shows the image, just a pink cast over it. But when loading into a software, all I see is black and green).
Anyone know what may be causing this?
Title: Re: 14bit RAW DNG silent pics!
Post by: aaphotog on May 04, 2013, 12:30:39 AM
Quote from: aaphotog on May 04, 2013, 12:29:18 AM
So, I just tried shooting the raw pics(using sd card gets me about 25 frames)
Tried on the small jpeg like posted and got 50 frames. loaded it up, and the preview that the mac shows me is a pink tint(was doing this previously but white balance fixed it) but when I load into after effects or lightroom 4, all i see is black and green and white balance changes NOTHING!
Thinking it was because I was shooting JPG, I went back to small raw(getting 25 frames) and still the same thing(finder shows the image, just a pink cast over it. But when loading into a software, all I see is black and green).
Anyone know what may be causing this?

NVM hahaha figured it out. Do not use the flat picture profiles(ie. cinestyle)
Changed the picture profile to neutral and everything loads correctly now.
Wonder why that would be happening... these are supposed to be raw, right?
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 04, 2013, 12:53:10 AM
So you're getting effects from changing raw to jpeg? These things shouldn't affect LV. I get ~50 frames in 720P and ~29 in 1080P normally.

I'll give it a go tho. I'll try to tweak memory some like a1ex did to where I can get it to not freeze but its faster. I think 5 might have been too low.. I got it going with 8. I think backup suite has to be larger than allocated or the other way around.

600D: 26frames 720P with raw Q... 26frames in S3. GD on/off didn't make any difference either for silent pic mode... I think it did using LVrec
Title: Re: 14bit RAW DNG silent pics!
Post by: aaphotog on May 04, 2013, 01:52:55 AM
The messed up color comes from a picture style 'cinestyle'
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 04, 2013, 02:10:17 AM
Picture style shouldn't apply to DNG but it WILL apply to YUV... also black level, put that up too much and it turns pink.
Title: Re: 14bit RAW DNG silent pics!
Post by: aaphotog on May 04, 2013, 05:37:33 AM
Quote from: 1% on May 04, 2013, 02:10:17 AM
Picture style shouldn't apply to DNG but it WILL apply to YUV... also black level, put that up too much and it turns pink.
what is YUV?

All I did was the burst mode, set it to 720p 60fps and over ride the fps to 24 and changed picture style. gave me a horrible image that i couldnt change. when I put picture style back to neutral, everything was ok.

Download cinestyle and try it. Maybe it was because of the curve that they put on the picture which I guess does bring up the black level(but that's baked into the picture style)
Title: Re: 14bit RAW DNG silent pics!
Post by: Africashot on May 04, 2013, 06:03:45 AM
51 frames on the 5D2 and 37 frames on the 600D over here... its getting better by the day! If there was a way of reaching +-240 frames I'd see myself using this very often.
Title: Re: 14bit RAW DNG silent pics!
Post by: Africashot on May 04, 2013, 06:12:20 AM
I have to take that back, turns out the 600D actually acts as if it was taking 37 frames, but then there are no DNGs on the card...
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 04, 2013, 06:32:33 AM
Super neutral and marvel's .. no difference on 600D dng. I haven't tested the 3rd mode: hold/release but it should be working.
Title: Re: 14bit RAW DNG silent pics!
Post by: squig on May 04, 2013, 07:24:20 AM
I'm recording 51 frames @24fps 1080p on the MK2 (Sandisk extreme 60MB/s 8Gb) but I'm only getting 29 frames on the MK3 (with the Sandisk and a 600x Lexar). I'm getting 49 frames @ 720p.
Title: Re: 14bit RAW DNG silent pics!
Post by: Andy600 on May 04, 2013, 11:37:32 AM
I'm getting 16 frames 1080p and 26 frames at 720p with FPS override at 24fps. Picture Styles don't affect the image here either (600d). Exposure and WB settings do seem to affect the image so far as pink casts etc but they are easily adjusted in post. It's a shame the buffer memory is so low because shooting DNG is amazing.

I'm sitting on the fence, waiting to see how things pan out with the 5d3 and raw/DNG shooting before I buy a new camera. The BMCC m4/3 is looking very tempting now.
Title: Re: 14bit RAW DNG silent pics!
Post by: dfsgfeiuhd on May 04, 2013, 12:43:53 PM
Quote from: CFP on May 03, 2013, 10:29:35 PM

For the silent pictures, I've got the best results in 720p 60 fps mode with "FPS override" set to 10, 24 or 30 fps and the picture size set to "sRAW" inside the ML menu.


Thanks for your hints. Installation worked, i like the new layout over the stable 2.3. So far I am happy. Two follow-up questions:

a) Where exactly is the sRAW setting in ML? I have the latest nightly installed and went through every option but cannot find it.
b) Do you use the silent picture workflow in continuous setting to shoot raw? I tried and it only records 422 files, no dngs.
Title: Re: 14bit RAW DNG silent pics!
Post by: Andy600 on May 04, 2013, 12:55:33 PM
Card speed makes no difference to the amount of shots you can shoot with DNG. It only affects the speed at which the buffer dumps DNGs to the card when the buffer is full. I'm getting the same amount of shots with class 6, 10 and UHS-1 cards. (600d)
Title: Re: 14bit RAW DNG silent pics!
Post by: CFP on May 04, 2013, 01:35:48 PM
Quote from: dfsgfeiuhd on May 04, 2013, 12:43:53 PM
a) Where exactly is the sRAW setting in ML? I have the latest nightly installed and went through every option but cannot find it.
b) Do you use the silent picture workflow in continuous setting to shoot raw? I tried and it only records 422 files, no dngs.
I've already told you: To record DNG on the EOS 600D you have to put these files onto your SD card: [600d]LvrecTest.zip (https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads/%5B600d%5DLvrecTest.zip) ;)

Then you'll get DNG files and the option for SRAW. The nightly build itself isn't enough.

Quote from: Andy600 on May 04, 2013, 11:37:32 AM
I'm getting 16 frames 1080p and 26 frames at 720p with FPS override at 24fps.

Then you're doing something wrong ... Like I said, on my 600D I've got 37 frames with a resolution of 1740 X 692 (In 720p) and 22 frames with a resolution of 1740 X 1154 (In 1080p).

Make sure you set the "Pic Quality" inside the ML menu to "SRAW". After that switch the mode with the ontop dial or reboot your camera. Then you should get as many frames as I. (Be carefull, "MRAW" seems to reduce the amount of frames ...)

BTW. ML developers, could you please renable the 3-10x digital zoom on the 600D? I know, it's my fault that you disabled it, but I'm curious which resolution I get with more than 3x zoom ... :D

But I agree with you that the Picture Style doesn't affect the DNG images. But in 5x or 10x zoom mode (Not that one you've disabled, but the normal one) my images are completly purple !? That's no white balance issue, or at least I can't fix it with Lightroom 4. So on my 600D the zoom modes are pretty pointless. I wonder if this is normal?  :(

Edit: And again, I really want to thank you for all these stunning features you've given to us! I love ML!
Title: Re: 14bit RAW DNG silent pics!
Post by: Rush on May 04, 2013, 01:44:31 PM
QuoteBTW. ML developers, could you please renable the 3-10x digital zoom on the 600D? I know, it's my fault that you disabled it, but I'm curious which resolution I get with more than 3x zoom ...
You need to disable 1x-3x shortcut in ML menu to enable 3x...10x mode digital zoom.
Title: Re: 14bit RAW DNG silent pics!
Post by: Andy600 on May 04, 2013, 02:23:34 PM
CFP - Thanks :) I hadn't switched modes but I just tried what you suggested and now getting the same 37 frames in 720p.
Title: Re: 14bit RAW DNG silent pics!
Post by: CFP on May 04, 2013, 02:44:47 PM
Thank you Rush! I didn't know that it is possible to renable the 3-10x zoom inside the ML menu.

But it seems like it doesn't affect the resolution or amount of frames. I get 25 frames with a resolution of 1804 X 1020.

But what's with my problem with the purple frames in 5x and 10x zoom (The focus assist zooms)? Is that normal? Do all 600D users have that? Or am I just doing something wrong? I get 19 frames with 2520 X 1148 in both modes but like I said, they are purple ... Which is sad because that high resolution might be very usefull  :(

@ Andy600: I'm glad that helped you :D
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 04, 2013, 05:15:01 PM
QuoteBTW. ML developers, could you please renable the 3-10x digital zoom on the 600D? I know, it's my fault that you disabled it, but I'm curious which resolution I get with more than 3x zoom

I actually did, just didn't post an update. You can record YUV like this, I think dng isn't affected. Quality above 3x isn't so great but the buffer is small. At least you can tweak it to where you get a bunch more frames or probably even continous if you shrink it down far enough.

I have to check on the zoom frames, post an example. If they are REALLY REALLY pink, I need to alter the size. Mine seem to be coming out tho. Hmm.. maybe sync is a problem here. I get a corrupt frames sometimes that are all green or pink, look like RGB zebras :)

Lol, the sraw trick really seems to work, 26 frames now in crop, 23 in 1080P, 39 on 720P24. So like 10 more frames! 6D that doesn't work unfortunately. But 600D climbing up to 6D levels.
Title: Re: 14bit RAW DNG silent pics!
Post by: aaphotog on May 04, 2013, 05:38:59 PM
So YUV is different from the DNG burst mode? How do I access recording with it?
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 04, 2013, 06:04:48 PM
modules-> load modules on supported cameras, on 600D the DNG size is not set, I can't read the files back with no header... will have to hex edit and copy paste one and see what comes of it. Its hard coded to only 1080P anyway but yield will probably be double when it works. Wonder if SRAW trick will help here too to increase it even more.

I'm thinking sraw lowers the amount of memory some process uses... maybe thats another way to increase yields, kick/shrink canon process memory (that use shoot_malloc).
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on May 04, 2013, 06:30:42 PM
this entire vid is comprised of 5x zoom recordings, 51 frames a piece, on the 5dmkii on a 60mb/s sandisk 16gb 2152x1076 comp size, all at 135mm.

http://www.youtube.com/watch?feature=player_embedded&v=e7Pu6m8GhdM
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 04, 2013, 06:45:51 PM
So I actually have a Q. The header makes the DNG load as 8bits per channel... should it be 16? 3x8 is 24bit total... 3x16 is 48bits... 14bit dng is 14bits all data or per channel?
Title: Re: 14bit RAW DNG silent pics!
Post by: Mati on May 04, 2013, 07:34:27 PM
Canon 5D mark II, Canon 85mm 1.8, 51 frames per sequence, 1s video = 24 frames.
Title: Re: 14bit RAW DNG silent pics!
Post by: Audionut on May 04, 2013, 07:56:50 PM
Quote from: 1% on May 04, 2013, 06:45:51 PM
So I actually have a Q. The header makes the DNG load as 8bits per channel... should it be 16?

As far as I know, all Adobe products default to 8bit/channel.  Generally, 8bits is enough, and it significantly reduces memory/disk usage.


Quote from: 1% on May 04, 2013, 06:45:51 PM3x8 is 24bit total... 3x16 is 48bits... 14bit dng is 14bits all data or per channel?

Per channel.  Good practice is to load your RAW's @ 16bit in ProPhoto RGB color space so as not to clip any information.

http://www.outbackphoto.com/color_management/cm_06/essay.html
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 04, 2013, 08:15:34 PM
So color profile and bits are wrong from the header? I think it picks adobe RGB or something as the profile.

BTW SRAW/MRAW should help increase available buffer for H264 video too since it works for DNG.
Title: Re: 14bit RAW DNG silent pics!
Post by: Andy600 on May 05, 2013, 12:18:49 AM
Quote from: 1% on May 04, 2013, 08:15:34 PM
...
BTW SRAW/MRAW should help increase available buffer for H264 video too since it works for DNG.

I can't say for sure but that seems to be the case. I just had BR up to a constant 130 (GOP3/flush3 using TLv2.0) on a detailed shot with SRAW selected. It falls over at around 90Mbits when RAW is selected on a class 10 SDHC card. Need to do more tests.
Title: Re: 14bit RAW DNG silent pics!
Post by: Andy600 on May 05, 2013, 12:25:10 AM
BTW, I haven't seen a full list of expected frame sizes / frame amounts for DNG shots so here's my list of what I have achieved on the 600d using 1%'s TLv2:

FPS override NOT set / Global draw OFF / PicSize SRAW
1920x1080p 24fps - resulting frame size = 1740x1154 (22 frames)
1920x1080p 24fps x3 Crop - resulting frame size = 1804x1020 (25 frames)
1920x1080p 24fps 5x focus assist - resulting frame size = 2520x1148 (19 frames)
1920x1080p 24fps 10x focus assist - resulting frame size = 2520x1148 (19 frames)
1280x720p 50fps - resulting frame size = 1740x692 (37 frames)

These are maximums. When you change to MRAW/RAW etc the frame count goes down.
Title: Re: 14bit RAW DNG silent pics!
Post by: ideimos on May 05, 2013, 01:23:48 AM
Could double digic processors on the 7D provide any help with this new RAW findings?
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 05, 2013, 03:52:01 AM
Dunno... on EOSM + probably other digic V there is a YUV to JPEG compression path.

FF22FEB0 str_SetJpegYUVBuffer    , etc...
Title: Re: 14bit RAW DNG silent pics!
Post by: Audionut on May 05, 2013, 04:00:21 AM
Quote from: 1% on May 04, 2013, 08:15:34 PM
So color profile and bits are wrong from the header? I think it picks adobe RGB or something as the profile.

There is no wrong, it's a choice.  Personally I think 8bits with adobeRGB is the better option for the header.  It will satisfy a larger percentage of the user base.  Those of us who want different, know it's just a header and will have our RAW editors set to overwrite the header anyway.
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 05, 2013, 04:00:30 AM
Just a quick question on current workflow with what we have working at the moment. How are people identifying which DNG's belong to a particular burst? As in, if I do say 5 bursts, when I come to pull them off a card and into a sequence, does anyone have an easy way of being able to tell where each burst starts and ends? Maybe just me being stupid and there's an obvious answer to this, I just haven't had a great deal of time to play yet.

Also, does anyone know how to correct a frame other than the first frame of a sequence with Adobe RAW once your Raw sequence is imported into AE? Might give Resolve a blast with these badboys in a bit - getting my geek on  8)
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 05, 2013, 04:06:47 AM
Quote from: 1% on May 04, 2013, 08:15:34 PM
So color profile and bits are wrong from the header? I think it picks adobe RGB or something as the profile.

BTW SRAW/MRAW should help increase available buffer for H264 video too since it works for DNG.

Something else I wondered about this...

When opening a DNG in AE you get the Camera Raw options but with no choice on colour space, depth, resolution etc. but in Photoshop, when you look through those options you can choose up to 3072x2039 (6.3MP) . Anyone explain if this is correct and what the actual resolution of the DNG's really is? Cheers  ;)
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 05, 2013, 04:14:02 AM
Quote from: noisyboy on May 05, 2013, 04:06:47 AMAnyone explain if this is correct and what the actual resolution of the DNG's really is?

May have just answered my own question - it's 1840x1221 isn't it?  :P
Title: Re: 14bit RAW DNG silent pics!
Post by: mucher on May 05, 2013, 05:19:37 AM
I can see the color band in the sky in the video captured by 7D, so I am very sure 7D's video capture capacity is 8 bit color depth in the output end, which means that in the footage/clip, the video color depth is 8bit wide(total 256 grade grey tone)

Provided that the LV is 32bit and the hardware h.264 process is not 8 bit wide, anything capture at the LV can pass through into a video will be way better in color depth

I think for now the safest bet will be either: capture can turn it into small jpg files on the fly and jam them through into the CF card, or, jam the capture video through the camera's hardware encoder(get your fingers crossed that the camera's hardware encoder is not 8bit wide, but I dont think that it is likely so)--anything, 4:2:2, 4:4:4 or 4:2:0. Either way you will get better results than it is now.

7D is not UDMA6 bounded (UDMA7 compatible), so its CF writing bandwidth is limited 133mb/s, that should be around 1000x CF, I believe that there is no way one can jam through anything by raw video mode through to the CF card, even the CF card is 4000x CF and I don't think the 5D2 or 5D3 will be able to exceed that CF card performance enough to write video raw into a CF card. I personally see no reason to look into that direction.

Or, one can use a HDMI recorder...
Title: Re: 14bit RAW DNG silent pics!
Post by: DTSET123 on May 05, 2013, 05:23:51 AM
QuoteWhen opening a DNG in AE you get the Camera Raw options but with no choice on colour space, depth, resolution etc. but in Photoshop, when you look through those options you can choose up to 3072x2039 (6.3MP) . Anyone explain if this is correct and what the actual resolution of the DNG's really is? Cheers 
Try using photoshop's image processor. Convert your dng to jpeg and then import to premeire. It will allow you to import as a sequence. I remember Adobe Labs used to have dng importer for premeire but from what I read they discontinued it http://www.youtube.com/watch?v=oALUwhZaK4s
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 05, 2013, 05:30:25 AM
Quote from: DTSET123 on May 05, 2013, 05:23:51 AM
Try using photoshop's image processor. Convert your dng to jpeg and then import to premeire. It will allow you to import as a sequence. I remember Adobe Labs used to have dng importer for premeire but from what I read they discontinued it http://www.youtube.com/watch?v=oALUwhZaK4s

Cheers man but I think I may have caused a confusion here  :P

Importing the DNG's as a raw sequence isn't a problem, I was just curious about how to work on a frame other than the first frame of the sequence once it's opened? As in when I load the Adobe Camera plugin to correct the raw sequence it defaults to the first frame when that might not be the hero frame I want to work with to correct my sequence. Does that make anymore sense? I think I just made it worse didn't I?  :-\
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 05, 2013, 05:34:02 AM
Quote from: mucher on May 05, 2013, 05:19:37 AM
I can see the color band in the sky in the video captured by 7D, so I am very sure 7D's video capture capacity is 8 bit color depth in the output end, which means that in the footage/clip, the video color depth is 8bit wide(total 256 grade grey tone)

Provided that the LV is 32bit and the hardware h.264 process is not 8 bit wide, anything capture at the LV can pass through into a video will be way better in color depth

I think for now the safest bet will be either: capture can turn it into small jpg files on the fly and jam them through into the CF card, or, jam the capture video through the camera's hardware encoder(get your fingers crossed that the camera's hardware encoder is not 8bit wide, but I dont think that it is likely so)--anything, 4:2:2, 4:4:4 or 4:2:0. Either way you will get better results than it is now.

7D is not UDMA6 bounded (UDMA7 compatible), so its CF writing bandwidth is limited 133mb/s, that should be around 1000x CF, I believe that there is no way one can jam through anything by raw video mode through to the CF card, even the CF card is 4000x CF and I don't think the 5D2 or 5D3 will be able to exceed that CF card performance enough to write video raw into a CF card. I personally see no reason to look into that direction.

Or, one can use a HDMI recorder...

Surely if the Blackmagic Pocket Camera is going to be writing 1080p Cinema DNG to an SD card then the write and read speeds of cards we have already shouldn't be a problem?
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 05, 2013, 05:35:39 AM
Unless Blackmagic have a "trick up there sleave" - Eh! Get it!? Blackmag... nevermind... :-X
Title: Re: 14bit RAW DNG silent pics!
Post by: DTSET123 on May 05, 2013, 05:37:58 AM
QuoteImporting the DNG's as a raw sequence isn't a problem, I was just curious about how to work on a frame other than the first frame of the sequence once it's opened? As in when I load the Adobe Camera plugin to correct the raw sequence it defaults to the first frame when that might not be the hero frame I want to work with to correct my sequence. Does that make anymore sense? I think I just made it worse didn't I?

It does make sense but I dont think there is a work around that, at least not until someone will figure out how to put them in a container (like MOV wrapper or something).
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 05, 2013, 06:13:53 AM
Quote from: DTSET123 on May 05, 2013, 05:37:58 AM
It does make sense but I dont think there is a work around that, at least not until someone will figure out how to put them in a container (like MOV wrapper or something).

Cool - cheers :)

Lightroom with Autosync does it for now then I guess!
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 05, 2013, 07:58:18 AM
Quote from: noisyboy on May 05, 2013, 04:00:30 AM
Just a quick question on current workflow with what we have working at the moment. How are people identifying which DNG's belong to a particular burst? As in, if I do say 5 bursts, when I come to pull them off a card and into a sequence, does anyone have an easy way of being able to tell where each burst starts and ends? Maybe just me being stupid and there's an obvious answer to this, I just haven't had a great deal of time to play yet.

Also, does anyone know how to correct a frame other than the first frame of a sequence with Adobe RAW once your Raw sequence is imported into AE? Might give Resolve a blast with these badboys in a bit - getting my geek on  8)

Hi
in a 6D in the canon menu there is an option to make a new folder, after one burst i am just creating and choosing new destination folder, after shooting i have my bursts in separate places. It helps a lot. I don't know if other cameras have such feature but it is worth to check.
Title: Re: 14bit RAW DNG silent pics!
Post by: mucher on May 05, 2013, 09:23:11 AM
Quote from: noisyboy on May 05, 2013, 05:34:02 AM
Surely if the Blackmagic Pocket Camera is going to be writing 1080p Cinema DNG to an SD card then the write and read speeds of cards we have already shouldn't be a problem?

as far as I know, that BPC uses a lossless compressed format:CinemaDNG 12bit RAW, that will take a processor that can compress the file in real-time, obviously 5d2, 5d3 and 7d or likes do not have that ability built-in.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 05, 2013, 03:45:45 PM
Well if that YUV to jpeg path could be used somehow then we'd get YUV 4:2:2 jpeg as video at least. Can't compress the DNG with CPU so would need to user parts of the camera designed to this to shrink it unless there is some CPU easy way to compress it you could recover on the computer.
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 05, 2013, 05:46:25 PM
Quote from: kgv5 on May 05, 2013, 07:58:18 AM
Hi
in a 6D in the canon menu there is an option to make a new folder, after one burst i am just creating and choosing new destination folder, after shooting i have my bursts in separate places. It helps a lot. I don't know if other cameras have such feature but it is worth to check.

Lucky for me I have a 6D :) That's a good idea - thanks!

Quote from: mucher on May 05, 2013, 09:23:11 AM
as far as I know, that BPC uses a lossless compressed format:CinemaDNG 12bit RAW, that will take a processor that can compress the file in real-time, obviously 5d2, 5d3 and 7d or likes do not have that ability built-in.

Ah - I see :) Yeah, that would make sense!
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 05, 2013, 05:52:55 PM
All cameras can create dir... I don't know why it isn't used.
Title: Re: 14bit RAW DNG silent pics!
Post by: stevefal on May 05, 2013, 06:24:32 PM
Quote from: 1% on May 05, 2013, 03:45:45 PM
Well if that YUV to jpeg path could be used somehow then we'd get YUV 4:2:2 jpeg as video at least. Can't compress the DNG with CPU so would need to user parts of the camera designed to this to shrink it unless there is some CPU easy way to compress it you could recover on the computer.

If that was possible, and you had access to the upstream color quantization parameters, it might be possible to generate two JPEGs per frame, one for 8 high bits and one for the 6 low bits. Then generate 14/16-bit images through an HDR workflow and you'd have full sensor range.
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 05, 2013, 07:45:11 PM
Quote from: 1% on May 05, 2013, 05:52:55 PM
All cameras can create dir... I don't know why it isn't used.

If all cameras can do this is there a way you wizards can make the camera create a new dir after each burst or is that just crazy talk?
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 05, 2013, 08:16:31 PM
Possible, just tried but no workee... I think it has to pass the directory and most of the functions are set to just pass the file name.
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 05, 2013, 08:32:29 PM
Quote from: 1% on May 05, 2013, 05:52:55 PM
All cameras can create dir... I don't know why it isn't used.

550D doesn't have such option, everything goes to one dir.
Title: Re: 14bit RAW DNG silent pics!
Post by: Rush on May 05, 2013, 08:43:55 PM
Quote from: kgv5 on May 05, 2013, 08:32:29 PM
550D doesn't have such option, everything goes to one dir.
Check Canon Menu->Select Folder (first page of settings)
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 05, 2013, 09:03:21 PM
FIO_CreateDirectory is on all cameras I've looked at. Just how to integrate it....
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 05, 2013, 09:08:13 PM
I have it (550D) in my hand right now and i cannot find it anywhere in a canon stock menu. I saw this feature first time when i bought 6D.
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 05, 2013, 09:12:57 PM
Quote from: 1% on May 05, 2013, 08:16:31 PM
Possible, just tried but no workee... I think it has to pass the directory and most of the functions are set to just pass the file name.

Cool! This would make life a whole lot easier if you figure it out. Not the end of the world if you can't though :)
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 05, 2013, 09:27:24 PM
550D its just not in GUI... canon still makes those DCIM folders.
Title: Re: 14bit RAW DNG silent pics!
Post by: vstrglv on May 06, 2013, 09:47:07 AM
Quote from: CFP on May 03, 2013, 10:29:35 PM
and the picture size set to "sRAW" inside the ML menu.

I can not find this in ML menu. MKIII
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 06, 2013, 11:46:09 AM
Me neither. I don't know where it is. 6D
Title: Re: 14bit RAW DNG silent pics!
Post by: Alia5 on May 06, 2013, 02:29:17 PM
sRAW in ML-Menu is for 600D
I guess you should choose sRAW in normal canan-dialog
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 06, 2013, 04:35:01 PM
Sraw trick doesn't work for 6D or any digicV I think. I tried raw, L, S3... when i started switching around I got more frames. When I tried sraw it only allocated 83MB and got less frames. Maybe there is  a difference between those but not sure.
Title: Re: 14bit RAW DNG silent pics!
Post by: Primary on May 06, 2013, 07:24:01 PM
Quote from: stevefal on May 05, 2013, 06:24:32 PM
If that was possible, and you had access to the upstream color quantization parameters, it might be possible to generate two JPEGs per frame, one for 8 high bits and one for the 6 low bits. Then generate 14/16-bit images through an HDR workflow and you'd have full sensor range.

It sounds good. Nice idea.
Could we solve this problem? I think it's betten than the original video quality. I wait for the solution.
Title: Re: 14bit RAW DNG silent pics!
Post by: CFP on May 06, 2013, 07:36:10 PM
Quote from: 1% on May 04, 2013, 05:15:01 PM
I have to check on the zoom frames, post an example.
Sorry for the late reply. Here's an example as well as two normal images. I hope that helps you:

- ML DNG with zoom (https://docs.google.com/file/d/0B1Hu7J2NCF4va25UYzNLWFMtdTg/edit?usp=sharing)
- ML DNG without zoom (https://docs.google.com/file/d/0B1Hu7J2NCF4vQ2dKMGlwTUUyZWs/edit?usp=sharing)
- Normal CR2 (https://docs.google.com/file/d/0B1Hu7J2NCF4vNGNSRXNvQXIzZEU/edit?usp=sharing)

All images were taken with the same settings in 'M' mode. I've used yesterday's nightly build. I took about 200 frames, but all of them had that strange color cast. So I'm quite sure that they aren't just corupted.

I've changed a lot of settings but I haven't found any option that may cause this. But nobody else has that problem, right? So I guess I'm doing something wrong. Any idea what I could try to get rid of that ... purpleness?  :D

Btw. the 720p DNGs look stretched, but this is normal, isn't it? Or is anything wrong with my camera?
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 06, 2013, 07:51:31 PM
I updated the sizes so that shouldn't happen on latest build.

The stretching does seem to happen, I think that's how it looks. Both on 6D and 600D and M there was some stretch.
Title: Re: 14bit RAW DNG silent pics!
Post by: Rush on May 06, 2013, 08:04:59 PM
QuoteBtw. the 720p DNGs look stretched, but this is normal, isn't it? Or is anything wrong with my camera?
It is not stretching but horizontal line skipping of 1080p
Title: Re: 14bit RAW DNG silent pics!
Post by: CFP on May 06, 2013, 08:13:53 PM
Yeah but it looks stretched because of the line skipping. But if that is normal I don't care. I just wanted to make sure my 600D is okay.

I'll try the latest build and see if that helps. Thank you guys  :)
Title: Re: 14bit RAW DNG silent pics!
Post by: AlexVakulov on May 06, 2013, 08:23:40 PM
-
Title: Re: 14bit RAW DNG silent pics!
Post by: vstrglv on May 06, 2013, 10:00:11 PM
Is the MAX frames only 29 for 1920p MKIII? Too small.
Title: Re: 14bit RAW DNG silent pics!
Post by: Rush on May 06, 2013, 10:38:26 PM
Quote from: vstrglv on May 06, 2013, 10:00:11 PM
Is the MAX frames only 29 for 1920p MKIII? Too small.
I think that 1920x720 RAWs can be 60p - it is better than H.264 1080p in terms of quality (correct me if I am wrong).
(But for video - required card speed is double over 30p, so I am not sure that it can be recorded continuous...)
Title: Re: 14bit RAW DNG silent pics!
Post by: dfsgfeiuhd on May 07, 2013, 12:07:02 AM
I tested the feature in the weekend on my 600d. I used different settings an am amazed by the quality.

At the moment, the camera fills the buffer and writes to card after the burst is completed i.e. the buffer is full. So no matter how fast the SC-Dard in the camera is, you always get a constant frame number depending on the settings chosen initially. I used two fast San Disk SDHC cards (Extreme II and Extereme HD) as well as a very slow 512 MB SD card. The results are the same.

Is there a way to write from the buffer to the card while filling it in parallel? This should squeeze some more frames out. I'll take even 10 more frames :) And for time lapse photography, that should be the way to go.

I also used 12 fps and tried to double that in after effects by frame interpolation. Sometimes, that works but not for fast movement. It looks super-weird then...
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 07, 2013, 12:10:01 AM
LV_REC will do this (well already does if you can process the files).... I think we need to find out how to sync 600D properly, first. Somehow burst mode doesn't have this problem but lvrec does..
Title: Re: 14bit RAW DNG silent pics!
Post by: dfsgfeiuhd on May 07, 2013, 12:15:01 AM
Awesome! I'll patiently await this feature then. So far, I am psyched: This is huge and I am even starting to like my 600d again as a video camera.

I forgot: Using Adobe Camera Raw 7.2 the white balance could be fixed. I had quite a bit of purple, but the fix was easy so I do not mind at all.

And: With the nightly and the raw add-on can I still format the card in camera? I haven't tried that yet since I did not want to corrupt the card.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 07, 2013, 12:35:37 AM
I don't think anything changed with 600D in regards to formatting.

We might get lucky and the raws won't have the sync problem... right now its for YUV... frames 1-24 ok, frames 25-1XX get all wonky like the example posted.
Title: Re: 14bit RAW DNG silent pics!
Post by: duncanidaho25 on May 07, 2013, 03:32:21 AM
I'v used burst mode, but what is "LV_REC" how do I do that?

I'm using a 5D mark ii

Thanks
Title: Re: 14bit RAW DNG silent pics!
Post by: CFP on May 07, 2013, 01:30:16 PM
Quote from: 1% on May 07, 2013, 12:35:37 AM
I don't think anything changed with 600D in regards to formatting.
Well, but something did change  ;)

If you format your SD-card inside the 600D and chose to 'keep ML', your 'MODULES' folder will disapear. So if you try to load the modules after that you'll get an error ...

I would definitely call that a change "in regards to formatting"  ;D

duncanidaho25: LV_REC is a module to record RAW and YUV video but it's still far away from being perfect and I think it is not yet available for 5DII?
Title: Re: 14bit RAW DNG silent pics!
Post by: Andy600 on May 07, 2013, 01:47:42 PM
I'm not sure if this has any relevance but I was reading John Brawley's overview of the new BMCC cameras and when he talks about the pocket camera he says:

"In fact, there has apparently always been an allowance for compressed RAW in the DNG specifications, it's just very few of the existing DNG applications have implemented it."

from this article: http://johnbrawley.wordpress.com/2013/04/09/the-pocket-rocket-blackmagic-downsizes-the-bmcc-and-does-a-4k-upsize-of-the-orginal/

Are the DNG files we get from the Canon already compressed?

Disclaimer: I know absolutely nothing about this stuff :)
Title: Re: 14bit RAW DNG silent pics!
Post by: g3gg0 on May 07, 2013, 02:30:11 PM
not canon makes the DNG, but ML does it (using CHDK code)
Title: Re: 14bit RAW DNG silent pics!
Post by: Michael Zöller on May 07, 2013, 05:58:06 PM
For what it's worth, here are some specs for lossy DNG from the official 1.4 DNG docs:
QuoteCompression

Two Compression tag values are supported in DNG versions before 1.4.0.0:
• Value = 1: Uncompressed data.
• Value = 7: JPEG compressed data, either baseline DCT JPEG, or lossless JPEG compression.If PhotometricInterpretation = 6 (YCbCr) and BitsPerSample = 8/8/8, or if PhotometricInterpretation = 1 (BlackIsZero) and BitsPerSample = 8, then the JPEG variant  must be baseline DCT JPEG.

Otherwise, the JPEG variant must be lossless Huffman JPEG. For lossless JPEG, the internal width/length/components in the JPEG stream are not required to match the strip or tile's width/length/components. Only the total sample counts need to match. It is common for CFA images to be encoded with a different width, length or component count to allow the JPEG compression predictors to work across like colors.

DNG Version 1.4.0.0 adds support for the following compression codes:
• Value = 8: Deflate (ZIP)
• Value = 34892: Lossy JPEG
Deflate (8 ) compression is allowed for floating point image data, 32-bit integer image data and transparency mask data. Lossy JPEG (34892) is allowed for IFDs that use PhotometricInterpretation = 34892 (LinearRaw) and 8-bit integer data. This new compression code is required to let the DNG reader know to use a lossy JPEG decoder rather than a lossless JPEG decoder for this combination of PhotometricInterpretation and BitsPerSample.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 07, 2013, 06:29:58 PM
Zip? Wouldn't that be easy on the proc... as in no real image processing.
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on May 07, 2013, 07:08:58 PM
Back from a short holiday, still catching up with the news.

I've just commited some things I've been tinkering with, while I was offline. Some of the neat stuff:

- Memory benchmarks (memcpy & friends). On 5D3, newlib memcpy does around 75 MB/s and dma_memcpy does around 100.

- We now know how to allocate large contiguous memory blocks (up to 32 MB).

- All cameras now have full exmem support, so porting this DNG stuff should be easier.

- A burst mode that only saves the best shots to the card. Works great on 5D3, but I doubt it will work well on 5D2. Try it e.g. on a dog running towards you, it should autodetect the focused shots.

- Revisited the slit-scan mode (now you can get distorted DNGs like this (http://a1ex.magiclantern.fm/bleeding-edge/raw/slitscan.dng)).

(http://a1ex.magiclantern.fm/bleeding-edge/raw/slitscan.jpg) (http://a1ex.magiclantern.fm/bleeding-edge/raw/slitscan-big.jpg)

That's it for now; I thought you were already recording 3K footage at 24p :P
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 07, 2013, 09:45:11 PM
67 frames now in 720P24 so new memory changes are good...

Also... real easy way to see whats going on... change pic quality and go look at shoot malloc total in memory info... 6D is 83M (mraw/sraw/raw+anything) or 152mb (raw, any jpeg). 600D goes from like 70mb to 103mb if you put the sraw hack on.
Title: Re: 14bit RAW DNG silent pics!
Post by: Andy600 on May 07, 2013, 09:48:15 PM
Quote from: 1% on May 07, 2013, 09:45:11 PM
67 frames now in 720P24 so new memory changes are good...

Waits patiently for the TL2 build ;)
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 07, 2013, 09:50:49 PM
Heh, well I'll up new 6D and 600D soon... with fixed MZ for 24p... still no go on the YUV tho.
Title: Re: 14bit RAW DNG silent pics!
Post by: Andy600 on May 07, 2013, 09:53:54 PM
Quote from: 1% on May 07, 2013, 09:50:49 PM
Heh, well I'll up new 6D and 600D soon... with fixed MZ for 24p... still no go on the YUV tho.

Cheers :) TBH I'm having more fun with DNGs so YUV isn't a problem for me. Nice tip about memory info!
Title: Re: 14bit RAW DNG silent pics!
Post by: dfsgfeiuhd on May 07, 2013, 09:57:44 PM
Quote from: 1% on May 07, 2013, 09:45:11 PM
67 frames now in 720P24 so new memory changes are good...

getting there, awesome! Do I need the new nightly build for that or just the new extra file?
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 07, 2013, 10:01:55 PM
Anything with LVrec needs new MAGIC.SYM (sometimes new lv_rec.mo) and autoexec
Title: Re: 14bit RAW DNG silent pics!
Post by: dfsgfeiuhd on May 07, 2013, 10:09:36 PM
Noted. Where can I get these? Last time, a user pointed me towards a download location but I could not find any general link to the latest files. I'd like to help development by testing.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 07, 2013, 10:12:23 PM
https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads
Title: Re: 14bit RAW DNG silent pics!
Post by: dfsgfeiuhd on May 07, 2013, 10:33:44 PM
Bookmarked! The version with the increased frame number is easy cut?
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 07, 2013, 10:41:37 PM
Top is latest upload.
Title: Re: 14bit RAW DNG silent pics!
Post by: dfsgfeiuhd on May 07, 2013, 10:47:51 PM
Can't wait to test it. Those 5 downloads, are the all for the Raw-feature and different (=updated) versions or also unrelated hacks? The filenames are a bit confusing and hint in another direction i.e. different features.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 07, 2013, 10:53:26 PM
Just get the latest... previous builds are just there if there is a regression. It adds on top of itself, so you don't want older builds where features weren't developed yet.

Aaaaand.. 6D has 256MB shoot malloc not in LV.. has to be  a way to pull a 600D and increase... would get quite a few more pics.

Title: Re: 14bit RAW DNG silent pics!
Post by: duncanidaho25 on May 07, 2013, 10:59:31 PM
Is there a link like this for 5D mark ii?

https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads
Title: Re: 14bit RAW DNG silent pics!
Post by: dfsgfeiuhd on May 08, 2013, 03:07:47 AM
Quote from: 1% on May 07, 2013, 10:53:26 PM
Aaaaand.. 6D has 256MB shoot malloc not in LV.. has to be  a way to pull a 600D and increase... would get quite a few more pics.

Wow, it's getting there. If one can get it to record around 4s of 24 fps DNG Video (fps override active or not), the feature is powerful enough for ~95% of my applications. No more h264, I am super excited!!!
Title: Re: 14bit RAW DNG silent pics!
Post by: vfx_junkie on May 08, 2013, 08:40:26 AM
(http://s12.postimg.org/632s74gn1/Untitled.jpg)

hey guys with what do you open this raw files I can't open them with anything

I am new here, but I have been using ML for a long time now :D
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on May 08, 2013, 09:06:51 AM
After Effects CS3 and up. Others say Lightroom Havent tried that, only AE.
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 08, 2013, 09:14:51 AM
Quote from: dfsgfeiuhd on May 08, 2013, 03:07:47 AM
Wow, it's getting there. If one can get it to record around 4s of 24 fps DNG Video (fps override active or not), the feature is powerful enough for ~95% of my applications. No more h264, I am super excited!!!


I think we now should find some good frame blending solution. I was testing LV_rec feature with different framerates (FPS override) on 6D (with sandisk extreme 64GB 45mbit/s)

LV_rec (single file, raw recording 1820x720p)
15 fps - 517 frames ~22 sec of movie at 24 fps
17 fps - 201 ~ 8 sec
18 fps - 152 ~ 6,5 sec
19 fps - 131 ~ 5,5 sec
20 fps - 116 ~ 5 sec
21 fps - 108 ~ 4,5 sec
22 fps - 97 ~ 4 sec
24 fps - 85 ~ 3,5 sec

I use HDR function a lot, with 35 fps override I have 17,5 fps (dark and light frames) and make frame blending with twixtor. When there is no fast action (I am using it mostly with slider motions) it looks very very nice, no visible ghosting. I believe, that we could achieve similar results with RAW. You said, that 4 seconds is enough  so framerate would be around 22-21 fps which I think should be good even for faster motion.
With burst no matter what fps settings are it gives me 67 DNG frames  - with slow movement, 16-17fps override and good frame blending we could have ~ 4 seconds of footage.

Unfortunatelly I couldn't make twixtor work with DNG's but maybe I was doing something wrong or maybe twixtor cannot handle RAW files. We have to wait also for the moment when someone find a way to process those Lv_rec RAW recording.
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 08, 2013, 09:41:38 AM
Quote from: ted ramasola on May 08, 2013, 09:06:51 AM
After Effects CS3 and up. Others say Lightroom Havent tried that, only AE.

You mean RAW single file recording from Lv_rec module? I tried and  AE CS6 doesn't work for me.
1% said before , that it is not contenerized yet or something (am I right?)
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on May 08, 2013, 09:43:23 AM
oooh. I meant the DNGs from burst mode. I did not know that the raw files from the LVrec is a different beast.
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 08, 2013, 09:47:53 AM
I don't know exactly if this is the same but I hope so.

So maybe someone could write with a simple language what is the main difference between RAW DNG burst and RAW recording mode in LV_rec module? Thanks
Title: Re: 14bit RAW DNG silent pics!
Post by: vfx_junkie on May 08, 2013, 10:07:49 AM
AE cs6 can't open single RAW file not even PS or PR
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 08, 2013, 03:18:51 PM
DNGs from Lvrec are packed and don't have a header. I think to open them you need to see what it does in the source when it saves them in burst mode. Nobody has made a script or processor for them yet. Will get you double yields once there is a way to use them.

Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 08, 2013, 04:48:24 PM
Quote from: vfx_junkie on May 08, 2013, 10:07:49 AM
AE cs6 can't open single RAW file not even PS or PR

Hey :)

You can open in AE, Lightroom or PS so take your pic :) AE is the obvious choice and if you select the first one and check the "raw sequence" checkbox it'll import them all as a sequence for you. The problems you are facing might be due to not having the latest Adboe Camera Raw plugin installed. A quick google search might help you find it (sorry, hate it when people don't just link to the actual thing they are talking about but Im a bit short on time).
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 08, 2013, 04:51:47 PM
That was assuming you mean captured from silent pic burst mode? I probably should have bothered reading the thread first  :P
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 08, 2013, 04:53:12 PM
Quote from: noisyboy on May 08, 2013, 04:48:24 PM
Hey :)

You can open in AE, Lightroom or PS so take your pic :) AE is the obvious choice and if you select the first one and check the "raw sequence" checkbox it'll import them all as a sequence for you. The problems you are facing might be due to not having the latest Adboe Camera Raw plugin installed. A quick google search might help you find it (sorry, hate it when people don't just link to the actual thing they are talking about but Im a bit short on time).

I am not saying about RAW DNG sequence friom burst mode. I am talking about RAW which is recorded in Lvrec module, these are completely different things. When you select "one file" you get SINGLE file (with say 150 frames inside) with .raw extension.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 08, 2013, 04:54:05 PM
Yep... LVrec .raw vs .dng from burst.
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 08, 2013, 04:58:08 PM
Quote from: kgv5 on May 08, 2013, 04:53:12 PM
I am not saying about RAW DNG sequence friom burst mode. I am talking about RAW which is recorded in Lvrec module, these are completely different things. When you select "one file" you get SINGLE file (with say 150 frames inside) with .raw extension.

Ah - sorry! Well that's me firmly confused. I don't even know what Lvrec is (keep reading about it but still thouroughly puzzled). From your description I get what it is but have no idea how to use it or even where it is so it't probably best for all involved if I quietly walk away  :-X

Anyone got a brief synopsis to fill a friendly novice in? :)
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 08, 2013, 05:10:37 PM
No problem  :D When someone will figure out how to use recorded .raw files we all will be extremely  happy.
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on May 08, 2013, 05:21:32 PM
This is the LVREC thread: http://www.magiclantern.fm/forum/index.php?topic=5247

In this thread we are talking about DNG silent pics (which you can open). This method is limited to small bursts (50 frames or so) and it's not suitable for continuous recording, because creating the DNGs in camera is quite slow.

LVREC is research code for continuous recording, and it's not yet ready to use for beginners. You get raw image dumps and you have to do all the cooking yourself, in post. Right now the only way is to modify chdk-dng.c, compile it for the PC, and tweak everything manually. Or, write your own converter. If you don't know how, just wait.

And g3gg0 said it very clearly: "This code will only work when compiled from repository (there is no release yet)". Please respect that.
Title: Re: 14bit RAW DNG silent pics!
Post by: wolf on May 08, 2013, 06:16:40 PM
I'm just curios.
If the RAW DNG silent pic would work on a legendary 550D, would it be larger than 1056x704 or would it only differ in codec?
Edit:Grammar
Title: Re: 14bit RAW DNG silent pics!
Post by: kgv5 on May 08, 2013, 06:23:58 PM
Quote from: a1ex on May 08, 2013, 05:21:32 PM
This is the LVREC thread: http://www.magiclantern.fm/forum/index.php?topic=5247

In this thread we are talking about DNG silent pics (which you can open). This method is limited to small bursts (50 frames or so) and it's not suitable for continuous recording, because creating the DNGs in camera is quite slow.

LVREC is research code for continuous recording, and it's not yet ready to use for beginners. You get raw image dumps and you have to do all the cooking yourself, in post. Right now the only way is to modify chdk-dng.c, compile it for the PC, and tweak everything manually. Or, write your own converter. If you don't know how, just wait.

And g3gg0 said it very clearly: "This code will only work when compiled from repository (there is no release yet)". Please respect that.

Very sorry for messing in this topic. We are respecting your outstanding work guys, really, you are awesome. ML became so popular that there are many many people who knows nothing about programming (including me in the first place) and we are just curious how things works.
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on May 08, 2013, 06:46:12 PM
Quote from: wolf on May 08, 2013, 06:16:40 PM
If the RAW DNG silent pic would work on a legendary 550D, would the be larger than 1056x704 or would the only differ in codec?

Try to guess with the edmac tool (hit don't click me to enable the raw flag, then compare the edmac status before and after).
Title: Re: 14bit RAW DNG silent pics!
Post by: noisyboy on May 08, 2013, 07:30:58 PM
Quote from: a1ex on May 08, 2013, 05:21:32 PM
This is the LVREC thread: http://www.magiclantern.fm/forum/index.php?topic=5247

In this thread we are talking about DNG silent pics (which you can open). This method is limited to small bursts (50 frames or so) and it's not suitable for continuous recording, because creating the DNGs in camera is quite slow.

LVREC is research code for continuous recording, and it's not yet ready to use for beginners. You get raw image dumps and you have to do all the cooking yourself, in post. Right now the only way is to modify chdk-dng.c, compile it for the PC, and tweak everything manually. Or, write your own converter. If you don't know how, just wait.

And g3gg0 said it very clearly: "This code will only work when compiled from repository (there is no release yet)". Please respect that.

Woah, I meant no disrespect a1ex, I just wondered what it was dude. I have no interest in meddling with things I don't understand. No offence intended  :)
Title: Re: 14bit RAW DNG silent pics!
Post by: duncanidaho25 on May 09, 2013, 04:38:40 AM
Thanks for this explanation.  I'm a beginner and have also been trying to figure out the LV_REC.  I will wait patiently (as patiently as I can anyway), and continue to play with the DNG files. 

Thanks for all your hard work Magic Lantern!
Title: Re: 14bit RAW DNG silent pics!
Post by: Africashot on May 09, 2013, 08:26:24 AM
I used the silent pics function yesterday during a shoot for a client, rather accidentally as the initial idea was to do a motion controlled HDR timelapse.
My intention was to shoot the TL in sRAW2 but the CF card I had in my 5D2 at the time happened to be the one with the nightly built on it, thus when I had it all set up and hit the shutter button and it started recording silent pics instead of the regular shots I had initial expected.
Generally it worked well, except for the fact that roughly 3-5% of the DNGs are corrupted (it looks as if it was a shot between frames of which one portion is completely magenta, I'll post samples if necessary although I assume anyone who has played with this function has seen those frames) In this case I was able to fix it in post but I could see this being a real challenge in other situations.
Another thing I have noticed is that due to the extra resolution the Moire gets extremely bad, I re-shot the same scene in h.264 HDR video and due to its softness it shows no moire whatsoever on places where  the DNG version shows almost fluorescent rainbow patterns, I guess the only way to fix this is an optical filter though.
I am not sure if this helps at all but since I can't help with the coding I am trying to at least assist with some feedback. And of course I can't thank all the developers enough for the amazing work they are doing!
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on May 09, 2013, 09:38:25 AM
which nightly build was it? Because I recalled the magenta'pinkish cast was somewhat addressed in later builds.
Title: Re: 14bit RAW DNG silent pics!
Post by: Rush on May 09, 2013, 10:51:54 AM
Quote from: Africashot on May 09, 2013, 08:26:24 AM
Another thing I have noticed is that due to the extra resolution the Moire gets extremely bad, I re-shot the same scene in h.264 HDR video and due to its softness it shows no moire whatsoever on places where  the DNG version shows almost fluorescent rainbow patterns, I guess the only way to fix this is an optical filter though.
Check that your RAW import software don't raise sharpness. And you can get rid of most color patterns with Lightroom:
(http://s2.ipicture.ru/uploads/20130501/nMA6pk6V.png)
Title: Re: 14bit RAW DNG silent pics!
Post by: Africashot on May 09, 2013, 11:47:10 AM
Quote from: ted ramasola on May 09, 2013, 09:38:25 AM
which nightly build was it? Because I recalled the magenta'pinkish cast was somewhat addressed in later builds.

Not really 100% sure what the nightly built was, I am guessing anytime between 1 - 3 May, is there a way to check?
It is also not a 'cast' what I am talking about (Lightroom imports with magenta levels somewhere at around 45 or so but this is simply adjusted by bringing it back to 0 or whatever works) I mean actually broken frames as you can see here:
Normal:
(http://africashot.com/wp-content/uploads/2013/05/normal.jpg)
Broken:
(http://africashot.com/wp-content/uploads/2013/05/broken1.jpg)
Or another one:
(http://africashot.com/wp-content/uploads/2013/05/broken2.jpg)
I see it more frequently when using the zoom function, nevertheless it randomly appears on 3 - 5 of the shots out of 100 in 1880 - 1250 too...
Title: Re: 14bit RAW DNG silent pics!
Post by: Africashot on May 09, 2013, 11:56:00 AM
Quote from: Rush on May 09, 2013, 10:51:54 AM
Check that your RAW import software don't raise sharpness. And you can get rid of most color patterns with Lightroom:
(http://s2.ipicture.ru/uploads/20130501/nMA6pk6V.png)

Yup, I have noticed that; Lightroom adds sharpness and I did defringe, nevertheless it is areas like this I find hard to deal with:
(http://africashot.com/wp-content/uploads/2013/05/00000174-2.jpg)
Nevertheless, what Lightroom version are you using, I think I am on 4.3 and can't find the advanced defringe option I see on your screenshot but kinda remember seeing it on maybe LR3?
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on May 09, 2013, 12:55:44 PM
Quote from: Africashot on May 09, 2013, 11:47:10 AM
Not really 100% sure what the nightly built was, I am guessing anytime between 1 - 3 May, is there a way to check?
It is also not a 'cast' what I am talking about (Lightroom imports with magenta levels somewhere at around 45 or so but this is simply adjusted by bringing it back to 0 or whatever works) I mean actually broken frames as you can see here:

Update with the latest build. If you have read this thread it has been fixed waaay back.
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 09, 2013, 05:04:53 PM
What camera? On 600D and 6D this does not happen for burst pics. There is a pink cast on *SOME* correctly sized DNGs... you could fix this pre photo with black level or in post with AWB. When they're all cut like that its a miss sync.
Title: Re: 14bit RAW DNG silent pics!
Post by: Rush on May 09, 2013, 07:08:40 PM
Quote from: Africashot on May 09, 2013, 11:56:00 AM
Yup, I have noticed that; Lightroom adds sharpness and I did defringe, nevertheless it is areas like this I find hard to deal with:
(http://africashot.com/wp-content/uploads/2013/05/00000174-2.jpg)
Nevertheless, what Lightroom version are you using, I think I am on 4.3 and can't find the advanced defringe option I see on your screenshot but kinda remember seeing it on maybe LR3?
It is LR5 beta. Try to increase defringe amount. But be care - it affects whole image.
Title: Re: 14bit RAW DNG silent pics!
Post by: duncanidaho25 on May 09, 2013, 07:23:34 PM
Africashot:  Do you have global draw on?  I had some similar results on my 5d mark ii however when I turned global draw off all my shots cleared up.
Title: Re: 14bit RAW DNG silent pics!
Post by: exflyer on May 09, 2013, 11:44:03 PM
Is there a way to download DNG files directly from the camera to the computer?
Title: Re: 14bit RAW DNG silent pics!
Post by: ted ramasola on May 10, 2013, 10:36:53 AM
On a 5D mk2, Turning global draw on or off has no impact on color aliasing or moire as shown on res chart tests I did. Only way to remove/lessen it is by mosaic VAF filter.

taken from DNG files from burst mode. Nightly build version May 10.

Global draw off - no VAF filter
(https://fbcdn-sphotos-d-a.akamaihd.net/hphotos-ak-ash4/964239_512365845487152_739804021_o.jpg)

Global draw on - no VAF filter
(https://fbcdn-sphotos-f-a.akamaihd.net/hphotos-ak-ash4/919378_512365852153818_857696967_o.jpg)

Global draw on - with VAF filter
(https://fbcdn-sphotos-b-a.akamaihd.net/hphotos-ak-ash4/459818_512365848820485_323822676_o.jpg)
Title: Re: 14bit RAW DNG silent pics!
Post by: ifkey on May 10, 2013, 05:08:59 PM
Quote from: kgv5 on May 08, 2013, 09:14:51 AM

I think we now should find some good frame blending solution. I was testing LV_rec feature with different framerates (FPS override) on 6D (with sandisk extreme 64GB 45mbit/s)

LV_rec (single file, raw recording 1820x720p)
15 fps - 517 frames ~22 sec of movie at 24 fps

you can put in the Internet? 15 fps - 517 frames ~ 22 sec of movie at 24 fps
Title: Re: 14bit RAW DNG silent pics!
Post by: Primary on May 12, 2013, 09:50:06 AM
How can i use silent pics? I downloaded a nightly biuld, but i found only the 1100D-105.fir, there isn't for 5D2.
Where can i download?
Title: Re: 14bit RAW DNG silent pics!
Post by: Malcolm Debono on May 12, 2013, 10:48:48 AM
Quote from: Primary on May 12, 2013, 09:50:06 AM
How can i use silent pics? I downloaded a nightly biuld, but i found only the 1100D-105.fir, there isn't for 5D2.
Where can i download?

You need the 2.3 build for the nightly to work (i.e. use 2.3 build files and replace the files with the nightly)
Title: Re: 14bit RAW DNG silent pics!
Post by: Primary on May 12, 2013, 12:01:34 PM
Thank you!
Title: Re: 14bit RAW DNG silent pics!
Post by: squig on May 13, 2013, 03:00:47 AM
Title: Re: 14bit RAW DNG silent pics!
Post by: Kim.dh on May 14, 2013, 09:01:26 PM
Does the RAW silents work on a 550d yet? Thanks.
Title: Re: 14bit RAW DNG silent pics!
Post by: wolf on May 14, 2013, 09:33:32 PM
The last couple of days silent pics didn't work proper at all on a 550D, but I didn't try it out today.
Title: Re: 14bit RAW DNG silent pics!
Post by: dym on May 16, 2013, 08:20:02 AM
Is there any tutorial how to do timelipse using 14bit raw?

I have 5d2 and last stable version - should i turn on silent pics and intervalometer for exmple for few seconds and thats all or should i do something more? How to have/change resolution to 2040x1428?

Regards and great job :)
Title: Re: 14bit RAW DNG silent pics!
Post by: dlrpgmsvc on May 16, 2013, 07:58:31 PM
News for 550d ?  ???
Title: Re: 14bit RAW DNG silent pics!
Post by: tihon on May 16, 2013, 09:19:39 PM
http://www.magiclantern.fm/forum/index.php?topic=5446.0
Title: Re: 14bit RAW DNG silent pics!
Post by: marekk on May 16, 2013, 11:40:56 PM
Is there a method to do it on 60D ?
Title: Re: 14bit RAW DNG silent pics!
Post by: OrhCh on May 18, 2013, 07:11:23 PM
Quote from: marekk on May 16, 2013, 11:40:56 PM
Is there a method to do it on 60D ?

I guess it is not implemented yet. At least in the last nightly build. I am interested in this function too.
Title: Re: 14bit RAW DNG silent pics!
Post by: 3pointedit on May 20, 2013, 06:27:28 AM
dlrpgmsvc sure grab a nightly build (550D is supported) and turn on silent pic. Set dial to movie (forces liveview on) and half press shutter button. Get like 2 fps or so. Drawback is you must turn off global draw and don't use FPS override (my frames seem to be corrupted).
Title: Re: 14bit RAW DNG silent pics!
Post by: dlrpgmsvc on May 20, 2013, 07:24:01 PM
Many thanks 3pointedit !

You, and all the people that have a 500D and a 550D, you can take a peek here ! --> http://www.magiclantern.fm/forum/index.php?topic=5446.75

just to contribute and stay on the bleeding edge of silent pics DNG RAW for 500d and 550d !
Title: Re: 14bit RAW DNG silent pics!
Post by: scrax on May 24, 2013, 03:23:45 PM
Added a pull request for the 600D, based on 1% fork.
It works on my 600D but not sure if all what I've added is needed and also here:


/*********************** Camera-specific constants ****************************/

/**
* LiveView raw buffer address
* To find it, call("lv_save_raw") and look for an EDMAC channel that becomes active (Debug menu)
**/

#if defined(CONFIG_5D2) || defined(CONFIG_600D)
#define RAW_LV_EDMAC 0xC0F04508
#endif

#if defined(CONFIG_500D) || defined(CONFIG_550D)
#define RAW_LV_EDMAC 0xC0F26008
#endif

#if defined(CONFIG_5D3) || defined(CONFIG_6D) || defined(CONFIG_650D) || defined(CONFIG_600D) || defined(CONFIG_60D) || defined(CONFIG_EOSM)
/* probably all new cameras use this address */
#define RAW_LV_EDMAC 0xC0F26208
#endif


What is the right edmac for 600D:
RAW_LV_EDMAC 0xC0F04508
RAW_LV_EDMAC 0xC0F26208

I've kept the second one since seems the one is using also TL and removed the first to avoid warning in compilation, but maybe someone else can assure me that it's right like this
Title: Re: 14bit RAW DNG silent pics!
Post by: 1% on May 24, 2013, 04:13:06 PM
That was a bug.. same raw photo edmac as 5d2 but same LV edmac as all the other cameras.
Title: Re: 14bit RAW DNG silent pics!
Post by: bhursey on May 26, 2013, 02:10:05 PM
So here is my question. I can not seem to be able to be able to do it. So say I set silent picture. However I set say the shutter at 3 seconds. I noticed when I was trying to create blur it does not.?  I want to sorta be able to catch car streaks and stars if possible. To do this the exposure needs to be long.  Basically what are the limitations? I think I am out of luck because it pulls the image from the live view correct? And if live view can not see it then it will not work.
Title: Re: 14bit RAW DNG silent pics!
Post by: GregoryOfManhattan on May 30, 2013, 04:42:18 PM
note:
lv_rec not working on 50D

undefined errors on
DeleteMemorySuite
PackMem_StartEDmac

these are not in the platform/50D.109/stubs.S so the module will throw undefined errors when attempting to load

interest is on raw video - so i'm just letting the community know.
Title: Re: 14bit RAW DNG silent pics!
Post by: shuggyrasklat on June 01, 2013, 11:30:21 PM
has there been any updates, on the buffer aize, to get more pictures out od the silent burst mode, on the 5dmkii?  I'm getting 55, for the moment..
Title: Re: 14bit RAW DNG silent pics!
Post by: Marsu42 on June 06, 2013, 05:53:41 PM
Quote from: a1ex on May 07, 2013, 07:08:58 PM
- A burst mode that only saves the best shots to the card. Works great on 5D3, but I doubt it will work well on 5D2. Try it e.g. on a dog running towards you, it should autodetect the focused shots.

Any experiences with "best shots" - I understand you only expect it to work on newer cameras like 5d3, what about digic4? It certainly does select some shots on my 60d, but I don't quite understand how the selection does it and if it really works reading about your doubts?
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on June 06, 2013, 05:56:25 PM
On 60D I expect it to work quite well, because focus estimation updates quickly. It worked great on 5D3, I just move the focus ring back and forth a few times, and then it saves the sharpest pics only.

Post some samples.
Title: Re: 14bit RAW DNG silent pics!
Post by: chmee on June 13, 2013, 10:01:53 AM
Hey Guys. for actual Discussion i thought about a A/B-Comparison of CR2 and lv_rec-RAW. Is there any chance to read out the whole sensor? Fundamentally we discuss about "how equal are both outputs"..

regards chmee
Title: Re: 14bit RAW DNG silent pics!
Post by: SoulState on July 13, 2013, 07:37:43 PM
What is best way to convert silent pic dng's to use in Lightroom or AE ACR plugin?
I have Lightrom 3.4 and After Effects 5.5 with ACR plugin, but both programs can't open RAW DNG. I tryed to use latest abobe dng converter v8.1.043, but no success(((
DNG silent pics from 60d camera.
Title: Re: 14bit RAW DNG silent pics!
Post by: SoulState on August 06, 2013, 03:21:19 PM
Just installed latest lightroom 5.2 RC, but it's all the same crap.
WTF? How you people open silent dng's?
Title: Re: 14bit RAW DNG silent pics!
Post by: Can on August 06, 2013, 03:39:21 PM
Quote from: SoulState on August 06, 2013, 03:21:19 PM
Just installed latest lightroom 5.2 RC, but it's all the same crap.
WTF? How you people open silent dng's?

I use ACR (opens with PS), or some folks use ufraw. It's in the beginning of the thread.
Title: Re: 14bit RAW DNG silent pics!
Post by: SoulState on August 08, 2013, 02:32:16 AM
Quote from: Can on August 06, 2013, 03:39:21 PM
I use ACR (opens with PS), or some folks use ufraw. It's in the beginning of the thread.

What version of ACR & PS do you use?
Can you try to open this file with your ACR, please?...
http://www40.zippyshare.com/v/56214344/file.html
Title: Re: 14bit RAW DNG silent pics!
Post by: SoulState on August 09, 2013, 09:11:28 PM
Ok, tryed latest PS with ACR 8.1 with no result. Not supported format again.
What's wrong with 60D dng raws?
Title: Re: 14bit RAW DNG silent pics!
Post by: Franzkekko on September 09, 2013, 10:19:45 PM
Hi folks!  :)
I did a timelapse test this afternoon to try capturing a sunset, silent Pictures + ETTR + Deflicker with XMP sidecar.
I set 400 shots and all seemed ok, except that then in post with lightroom it imported just first 212 shots and gave me an error opening last 187 pics.

Same error trying to open DNGs of the second part of tl with UFRaw, ACR or RawTherapee.

Do anyone know what may be the trouble? :)

Working DNG and XMP:
00000009.DNG  (https://docs.google.com/file/d/0B2YCVBuZF75bczk5ZmFlUnZldE0/edit?usp=sharing)
00000009.XMP  (https://docs.google.com/file/d/0B2YCVBuZF75bUXd3dnRIWjMzX0U/edit?usp=sharing)

Don't Working DNG with related xmp
00000335.DNG  (https://docs.google.com/file/d/0B2YCVBuZF75beUhTVG1EMHhnaGc/edit?usp=sharing)
00000335.XMP  (https://docs.google.com/file/d/0B2YCVBuZF75bZDV4REhpVUFHYXc/edit?usp=sharing)

P.S. don't look at the pole, i'm planning to take it down with a saw  8) :D ;D
Thanks in advance! ;)
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on September 19, 2013, 09:56:35 AM
Update: silent pics code is now a module:

https://bitbucket.org/hudson/magic-lantern/src/tip/modules/silent
Title: Re: 14bit RAW DNG silent pics!
Post by: Marsu42 on September 24, 2013, 01:54:39 PM
Quote from: Francis on April 26, 2013, 03:47:38 PM
Where does the 2040x1428 res come from on the 5d2?

On 6D, all I get is 1824x1224 - am I doing something wrong, or is the 6D lv res really inferior to the 5d2 and this is the max. it delivers (unless of course the recently discussed update is applied)?
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on September 24, 2013, 02:02:45 PM
That's all it delivers, silent pics are not cropped at all. 2040 includes the optical black areas (develop the silent pics with dcraw -E to see them; this switch is undocumented, but I need it for cr2hdr).
Title: Re: 14bit RAW DNG silent pics!
Post by: akumiszcza on September 26, 2013, 07:43:26 PM
I've tried using Best Shots options in Silent Pics module (latest nightly, Canon 50D). Each series ended up with first photo OK, all the rest just noise.

Example: https://mega.co.nz/#!IUlBhSCA!f6-9zAjzOfSmSGTGp0jqxHuT4ztIpY-xqwta3rZ-IZU

Burst mode worked fine.
Title: Re: 14bit RAW DNG silent pics!
Post by: a1ex on September 27, 2013, 12:03:44 PM
IIRC, the 50D does not output any focus quality metric, so "best shots" will not work. I should remove it from menu.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: far.in.out on October 04, 2013, 01:56:16 PM
Hi. Is there version for 600D?
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Audionut on October 15, 2013, 06:18:43 AM
Burst mode is only working when using the back button.  Half shutter reports that frames are being buffered but only 1 image is being saved.
Wasn't there discussion about moving focus to back button so it didn't move focus while firing off shots?
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Marsu42 on October 20, 2013, 05:15:00 PM
Fyi: This is currently broken on 60d (works on 6d), at least I couldn't get it to work anymore. I'm cross-posting this information here since it *used* to work and this is a regression´... people might end up like me with unexpected and unfortunate dead shots :-\

https://bitbucket.org/hudson/magic-lantern/issue/1719/silent-pictures-broken-on-60d
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: escho on October 20, 2013, 05:42:33 PM
Quote from: Marsu42 on October 20, 2013, 05:15:00 PM
Fyi: This is currently broken on 60d (works on 6d), at least I couldn't get it to work anymore. I'm cross-posting this information here since it *used* to work and this is a regression´... people might end up like me with unexpected and unfortunate dead shots :-\

https://bitbucket.org/hudson/magic-lantern/issue/1719/silent-pictures-broken-on-60d

600d: no problems

Edgar
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: a1ex on October 20, 2013, 05:59:28 PM
Same here on 5D3 (all OK).

@Marsu42: try recompiling the module. If that fixes the problem, we should bump the module API number.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Marsu42 on October 20, 2013, 06:44:01 PM
Quote from: a1ex on October 20, 2013, 05:59:28 PM
@Marsu42: try recompiling the module. If that fixes the problem, we should bump the module API number.

Ugh? I just recompiled the whole thing inc. make clean from unmodified trunk, and I've even reset the ml config and enabled silent.mo as the only module - it just produces black dng files.

As I wrote, it's also working fine on 6d for me using the same compilation method, so this seems to be a 60d-specific regression, the first time I noticed it was a couple of days ago when I came home with broken silent focus stacks.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: a1ex on October 20, 2013, 06:44:58 PM
Can't check on 60D right now; can you upload a DNG?
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Marsu42 on October 20, 2013, 06:47:54 PM
Quote from: a1ex on October 20, 2013, 06:44:58 PM
Can't check on 60D right now; can you upload a DNG?

https://bitbucket.org/Marsu42/ml-aiso/downloads/61540001.DNG
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: a1ex on October 20, 2013, 06:51:47 PM
Humm... it's empty; is raw video working?
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Marsu42 on October 20, 2013, 06:55:39 PM
Quote from: a1ex on October 20, 2013, 06:51:47 PM
Humm... it's empty

Agreeing on that is a good start :->

Quote from: a1ex on October 20, 2013, 06:51:47 PM
; is raw video working?

Speaking of which ... I just tried the new mlv_rec and mlv_dump was also complaining empty frames or something like this (I don't have deleted the raw file since as I never do raw video anyway - I can redo it again if it helps). What's up with your 60d - bricked again :-p ?
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: a1ex on October 20, 2013, 06:57:08 PM
I don't have it with me right now.

Empty frames?! sounds like something in the raw backend...
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Marsu42 on October 20, 2013, 07:06:51 PM
Quote from: a1ex on October 20, 2013, 06:57:08 PM
I don't have it with me right now.

it would be really nice to get it working again as it's now mushroom time and I'd like to shoot some silent focus stacks with it, but now that I know it's broken it's hardly a matter of hours... unless it's really necessary I'd prefer not to revert-search for the last ml build that worked as I have no idea how long it's been broken. Raw zebras/histogram are working btw.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: a1ex on October 20, 2013, 07:21:32 PM
If you or any other 60D owner can do a hg bisect and find the changeset that broke it, it will be very helpful.

The cool thing about hg bisect is that you only need log(n) tests, so if you want to locate a bug in 1024 changesets, you need only 10 test runs on the camera to find the changeset that broke it.

I've recompiled the nightly builds roughly one hour ago; is it also broken with these builds? Just to rule out toolchain problems or custom modifications.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: a1ex on October 21, 2013, 03:34:20 PM
Just tried on my 60D and it's working fine...

Maybe you can upload your config files? (the settings folder)
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Marsu42 on October 21, 2013, 07:02:47 PM
Quote from: a1ex on October 21, 2013, 03:34:20 PM
Just tried on my 60D and it's working fine... Maybe you can upload your config files? (the settings folder)

I don't understand this at all ... sure I can upload my current config, but I even tried it with an unmodified trunk with default settings.

Do I have some special video mode enabled for silent pictures to work, or are there any other potential problems that could trigger this behavior on the 60d while the 6d works fine with me doing absolutely the same things? I'll try an official nightly build and then report back if I got lucky, maybe it's my current compiler setup ... but thanks for helping me debug this!
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: a1ex on October 21, 2013, 09:31:16 PM
I tried in photo mode (don't remember which one). Again, can't try it until I get back home (later this week).
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Marsu42 on October 23, 2013, 11:21:11 PM
Quote from: a1ex on October 21, 2013, 03:34:20 PM
Just tried on my 60D and it's working fine...

Ok, I've found the cause - it's a compiler issue. I tried both Windows (cygwin x64) and Linux (Ubuntu x64) with both the newest arm gcc from Launchpad as well as the 2012-q4 that's in ml's Makefile.user) and it works in -Os mode, but produces the problems described in -O1/2/3... what I still don't understand why these obviously work for Nightly and not for me - but the important thing for me is that at least I'm able to compile it flawlessly, though not at high optimization atm.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: marekk on June 19, 2014, 07:41:49 PM
How to remove a vertical stripes from a DNG file captured using silent pics feature ?? I would like to prepare for some trip with my 5D and timelapses in lv mode. I would like to take shoots every 10-20 seconds so 0.5 fps override with mlv_rec is to fast for me..
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: garry23 on June 19, 2014, 07:56:08 PM
If it helps I use silent DNG with the intervalometer and get good results. That is LV mode, not 'movie' mode.

I add back the missing EXIF data using ExifToolGUI.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: marekk on June 19, 2014, 08:01:52 PM
ok but vertical stripes are removed by raw2dng and mlv_dump.. DNG files captured by silenc pics module are not processed by vertical stripes corrector..
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: garry23 on June 19, 2014, 08:09:16 PM
My experience is that the DNGs are clean and usable, ie no artifacts.

BTW I'm using a 5DIII.

My workflow for a fixed exposure timelapse is:
- a-ETTR to set exposure, ensure in manual mode and take note of Exposure values
- set intervalometer
- capture DNGs
- do holy grail adjustments if require. I only adjust my 1 stop so I know what the jumps are
- correct EXIF data in EXIFToolsGUI, having noted the key frame changes. This step is a little labour intensive if you used holy grail adjustments
- upscale if you wish to, ie more pixels for virtual panning
- use LR, LRTimelapse and Panolapse

Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: a1ex on June 19, 2014, 08:16:11 PM
The easiest workaround would be to convert the DNG to RAW (compile dng2raw.c), and then back to DNG with raw2dng.

(of course, you only need to do this if you actually get the artifact)

Side note: if you use photo mode LiveView (instead of movie mode), you may want to use a manual lens, or enable exposure override; otherwise, all your images will be taken wide-open.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: marekk on June 20, 2014, 11:41:05 AM
I will try to add some code to include exif data in dng files captured by silent pics feature..


Quote from: garry23 on June 19, 2014, 08:09:16 PM
My experience is that the DNGs are clean and usable, ie no artifacts.

BTW I'm using a 5DIII.

My workflow for a fixed exposure timelapse is:
- a-ETTR to set exposure, ensure in manual mode and take note of Exposure values
- set intervalometer
- capture DNGs
- do holy grail adjustments if require. I only adjust my 1 stop so I know what the jumps are
- correct EXIF data in EXIFToolsGUI, having noted the key frame changes. This step is a little labour intensive if you used holy grail adjustments
- upscale if you wish to, ie more pixels for virtual panning
- use LR, LRTimelapse and Panolapse
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: marekk on June 20, 2014, 07:39:20 PM
What do you think about option in MLV_REC to capture frames using intervalometer settings instead of fps ? 
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: garry23 on June 20, 2014, 09:14:26 PM
Marekk

If you could get the basic EXIF working in silent DNG that would really make this feature a timelapsers friend, ie no shutter action, a-ETTR to handle the holy grail cases, and DNG use straight in LR  and LRT, ie where you need the EXIF data.

I'm afraid I'm not a coder so can not offer any help, 'only' a tester and user :-o)
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: marekk on June 23, 2014, 11:43:19 PM
It looks like we can simply modify silent_pic_save_dng in silent.c

static void silent_pic_save_dng(char* filename, struct raw_info * raw_info)
{
    dng_set_shutter(1, (int)(1000000.0f/(float)(1000.0f * (1000000.0f / (float)get_current_shutter_reciprocal_x1000()))));
    dng_set_aperture(lens_info.aperture * 10, 100);
    dng_set_lensmodel((char*)lens_info.name);
    dng_set_focal(lens_info.focal_len, 1);


    if(lens_info.iso == 0)
    {
        dng_set_iso(lens_info.iso_auto);
    }
    else
    {
        dng_set_iso(lens_info.iso);
    }
   
    dng_set_camname((char*)camera_model);
    save_dng(filename, raw_info);
}


Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: garry23 on June 24, 2014, 01:08:48 AM
Matekk

Looks logical even to me non-coder eyes :-)

Have you tried if?

Is it a simple matter of dropping your code somewhere? Or do you need someone to compile it in a module?

Cheers

Garry
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: marekk on June 24, 2014, 11:42:38 AM
Yes, I tested it on 5D3 1.2.3 and it works... but my dng files still need vertical stripes correction..
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Audionut on June 24, 2014, 11:49:10 AM
Quote from: marekk on June 24, 2014, 11:42:38 AM
but my dng files still need vertical stripes correction..

Is this in movie mode?  I never noticed stripes in my photo mode silent DNGs.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: marekk on June 24, 2014, 11:51:14 AM
Quote from: Audionut on June 24, 2014, 11:49:10 AM
Is this in movie mode?  I never noticed stripes in my photo mode silent DNGs.

Yes, in movie mode. I have to check it in photo mode and let you know.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Audionut on June 24, 2014, 11:53:33 AM
Ok, I'll compile your code and confirm it working if you like.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: marekk on June 24, 2014, 12:04:19 PM
You should to add this at the beginning of silent.c

#include <lens.h>
#include <fps.h>

Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Audionut on June 24, 2014, 12:39:57 PM
edit:  Solved, needed #include "../../src/chdk-dng.h" also.

And confirmed working.
https://dl.dropboxusercontent.com/u/34113196/ML/61760002.DNG
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: marekk on June 24, 2014, 01:27:18 PM
Could you add a commit ?

Quote from: Audionut on June 24, 2014, 12:39:57 PM
edit:  Solved, needed #include "../../src/chdk-dng.h" also.

And confirmed working.
https://dl.dropboxusercontent.com/u/34113196/ML/61760002.DNG
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Audionut on June 24, 2014, 01:41:19 PM
Done:  https://bitbucket.org/hudson/magic-lantern/pull-request/524/silent-pics-enable-exif-info-in-dng/diff
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: garry23 on June 24, 2014, 02:27:00 PM
Marekk

I'm also using a 5DIII, but have never seen any artifacts, including vertical stripes.

I have taken many DNG timelapses and not one stripe.

Thanks for the EXIF fix, looking forward to an a-ETTR based holy grail timelapse test.

Cheers

Garry
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: gerk.raisen on June 24, 2014, 02:33:21 PM
If confirmed working could someone please submit a pull request? :)
I support too the idea of Marekk to have an option in MLV_REC to capture frames using intervalometer settings instead of fps.
Have all the timelapses in only one file, and you can use chmee software to create CDNG it will be very comfortable...
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: marekk on June 24, 2014, 09:27:17 PM
Could you check this dng file ? Do I need vertical stripes correction or not ? It was taken in photo mode.

https://www.dropbox.com/s/5syx42bpujqkumq/61800000.DNG
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: a1ex on June 24, 2014, 09:30:00 PM
If you see them, you need the correction. If not, you don't.

I don't see any on this particular image, but others (from the same camera) may need it.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: garry23 on June 24, 2014, 10:03:48 PM
Marekk

I also looked at the file and, other than post-process lens vignetting correction, it looks fine.

Cheers

Garry
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: dubzeebass on June 26, 2014, 05:40:44 AM


Quote from: a1ex on June 24, 2014, 09:30:00 PM
If you see them, you need the correction. If not, you don't.

Hahahahahahahhaha dying
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: barepixels on June 27, 2014, 07:53:21 AM
A1ex wrote

The easiest workaround would be to convert the DNG to RAW (compile dng2raw.c), and then back to DNG with raw2dng.

(of course, you only need to do this if you actually get the artifact)

===================================

I look and look but cant figured how to convert DNG to RAW.  Help, I want to fix those DNG that looks corupted.

(http://199.167.201.88/screencaptures/6T4Tt.png)
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: barepixels on June 27, 2014, 06:16:19 PM
can someone please show me where i can download  dng2raw.exe   

thnx

canon 5D2  magiclantern-Nightly.2014May13.5D2212
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: walter_schulz on June 27, 2014, 11:51:15 PM
http://www.magiclantern.fm/forum/index.php?board=54
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: a1ex on June 27, 2014, 11:54:12 PM
Where exactly? I'm not aware of any binaries for this tool.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: garry23 on June 28, 2014, 03:22:18 AM
Marekk

Just checking as I downloaded the latest 5DIII nightly and I see that the silent DNG fix you submitted hasn't been included yet.

I'm afraid I'm ignorant of the process to get changes made.

Do you know when your fix is likely to apper?

Cheers
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: a1ex on June 28, 2014, 06:32:16 AM
Here you go: https://bitbucket.org/hudson/magic-lantern/pull-request/524/silent-pics-enable-exif-info-in-dng/diff#comment-2221983

I'm not sure if Marekk is going to implement the last change I've asked for (people started arguing about other stuff), so others with coding skills are welcome to give it a try.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: marekk on June 28, 2014, 10:48:05 AM
:)
http://www.magiclantern.fm/forum/index.php?topic=12479.0
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: garry23 on June 28, 2014, 04:53:19 PM
Marekk

Thanks for the link and the module.

I will try it out in timelapse mode this weekend and give feedback.

BTW I still would like some kind sole to put the EXIF write code in the silent dng module :-)

Cheers
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: dxotic on June 28, 2014, 11:39:38 PM
I heartily agree. exif data would be immense in silent DNG as it would mean longer durations in exposure therefore more flexibility. But thanks goes to the team for an awesome addition to the tools  :D
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: babarasghar on November 09, 2014, 07:49:01 AM
this thread is silent for a very long time
i just tried silent pictures n compared them with raw
i find the shadow recovery suffers greatly in DNG files
the dynamic range is also very bad
am i doing something wrong here?
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: dmilligan on November 09, 2014, 03:27:30 PM
Post a silent DNG and regular CR2 that use the same exposure settings and are of the same scene. Also make sure you have expo override enabled or are in manual movie mode when taking silent pics, the LV picture might be "simulated" if you are not.

Also please include basic info like camera and ML build date, and all of the settings you are using.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: florinfl on November 12, 2014, 01:18:52 PM
Hi,
Is there a way in nightly builds to take full res pictures in 600d? I'm not sure but it seems that in 550d is already implemented. I was already told that it is in development, I just want to be sure that I don't miss something. Or maybe is already implemented in one form or another in earlier version of nightly bulds. All I cand do now is to make videos at low fps but of course at low res too. Maybe is there an unofficial build that has implemented already?
Thanks
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: dmilligan on November 12, 2014, 04:28:40 PM
You can already take raw silent pictures with the current nightly builds, but they are not full resolution (that is what this thread is about), they are video (LV) resolution. The full resolution silent pictures are not available (and haven't ever been previously) in the nightly builds because this functionality is not very stable and has lots of quirks. If you want to use it now, you need to compile ML yourself. Some people have posted unofficial builds that include this functionality, but I do not recommend using them (because IMO, if compiling ML is beyond your skill level, you probably shouldn't be using untested, experimental, and potentially dangerous code like this).
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: florinfl on November 12, 2014, 06:36:17 PM
Thank you for your advice, indeed I would be tempted to try what other tried and succeded (on their risk) but I'll wait for future updates, maybe at some point full resolution silent pictures will be part of the nightly builds.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: babarasghar on November 17, 2014, 05:54:22 PM

Quote from: dmilligan on November 09, 2014, 03:27:30 PM
Post a silent DNG and regular CR2 that use the same exposure settings and are of the same scene. Also make sure you have expo override enabled or are in manual movie mode when taking silent pics, the LV picture might be "simulated" if you are not.

Also please include basic info like camera and ML build date, and all of the settings you are using.


@dmilligan here is the link of DNG, CR2 and JPEG
i tried same thing on 6D and same issues, DNG is not even 1080p :(
DNG File: https://dl.dropboxusercontent.com/u/106218119/98910000.DNG
JPG File: https://dl.dropboxusercontent.com/u/106218119/IMG_9890.JPG
CR2 File: https://dl.dropboxusercontent.com/u/106218119/IMG_9891.CR2
Camera settings
(http://tapatalk.imageshack.com/v2/14/11/17/bf2f143dfb5812c8449ec6e90b60c3b8.jpg)

Magic lantern version is latest nightly build available
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Audionut on November 20, 2014, 03:37:19 AM
You should provide an CR2 direct from the camera in large raw.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: babarasghar on November 20, 2014, 01:57:01 PM
Check the links bro!!! Can u open these links?
One of them is cr2 :)
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Audionut on November 20, 2014, 09:04:40 PM
Some low resolution CR2 that's had the optical black areas removed.

Quote from: Audionut on November 20, 2014, 03:37:19 AM
You should provide an CR2 direct from the camera in large raw.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: dmilligan on November 20, 2014, 10:33:00 PM
The silent DNG appears to have more noise than the small CR2 (but without you supplying a full size CR2, there's no OB areas to get an exact amount). This makes sense. The silent DNG came from video mode where resolution is reduced by line skipping, while the small CR2 likely benefited from proper downsampling (when downsampling resolution, neighboring pixels sort of get merged together in a sense, which has the effect of reducing noise, in other words there is a trade off between noise and resolution that we are taking advantage of).

There are additional reasons silent DNGs would have more noise: the sensor is in a different mode where it is continuously read out, this causes the sensor to heat up and the heat increases noise (there may also be other effects on noise from this mode besides just heat, IDK).

If you want a more fair comparison, use crop mode for the silent DNG, and compare it to a full size CR2.

You can compare the noise yourself in camera using raw_diag (from here: http://www.magiclantern.fm/forum/index.php?topic=10111.0).

If you need to capture more DR then: ETTR properly, use dual ISO (should work with silent pics, but you might have to be in movie mode), and/or consider actually using your shutter.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: babarasghar on November 24, 2014, 01:44:41 PM
The why we r even calling it a 14bit dng when it don't even have as much dynamic range as much even a jpeg!
:(
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: walter_schulz on November 24, 2014, 02:06:57 PM
You're mixing things up. Dynamic range and bit depth are different terms.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: dmilligan on November 24, 2014, 02:15:15 PM
1. They are called that because that's what they are.

2. You haven't offered any definitive proof that they have less DR than a JPEG

3. DR is not necessarily related to bit depth or compression => I can capture 20 EVs of DR with a large bracket, process and tone it, and output the result into a JPEG that has 20 EVs of DR.

4. DR isn't the only important thing.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: babarasghar on November 29, 2014, 09:02:48 AM
Can't u guys open the links I have attached?
What other proof u need?
Why don't u tell me how should I take pictures n then post so that you see them and then give a verdict.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Walter Schulz on November 29, 2014, 09:12:01 AM
Mostly because you ignored Audionut's request. Twice.
http://www.magiclantern.fm/forum/index.php?topic=5240.msg134060#msg134060
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: astro2012 on December 01, 2014, 03:34:33 PM
Hello! share please the working module for canon 600d. I want to do full-res pictures with exposure to 10 seconds. I will be very grateful!
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: dmilligan on December 01, 2014, 04:39:34 PM
That feature still has some issues and is not ready for "prime time". You also need more than just the module, there are some requisite changes to the ML core. You can compile it yourself if you like, otherwise you need to wait until devs have time to fix the issues and merge it into the nightly builds.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: astro2012 on December 01, 2014, 05:10:41 PM
Thanks! I will think.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: babarasghar on February 08, 2015, 07:40:31 PM
So do we have better dynamic range dng now from this?
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Walter Schulz on February 08, 2015, 07:50:08 PM
It has nothing to do with better DR. ML has a silent pic function right now but output is cropped/LV resolution. Full Resolution Silent Pic (FRSP) is able to use all the sensor's pixels.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: babarasghar on February 08, 2015, 08:01:18 PM
Quote from: Walter Schulz on February 08, 2015, 07:50:08 PM
It has nothing to do with better DR. ML has a silent pic function right now but output is cropped/LV resolution. Full Resolution Silent Pic (FRSP) is able to use all the sensor's pixels.

yes i just tried Silent pictures option and max resolution i am getting is 1824x1224 on 6d (thats what it shows on screen in live view)
so how to enable FRSP on 6D, im using 29 Jan 2015 Nightly build
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Walter Schulz on February 08, 2015, 08:02:48 PM
Read reply #503
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: babarasghar on February 08, 2015, 08:40:09 PM
Quote from: Walter Schulz on February 08, 2015, 08:02:48 PM
Read reply #503

okay here are files, difference of resolution and one can further see in light room how less details one can extract from shadows (isnt it dynamic range) i think at the moment the info is gathered from live view (movie mode) n this is max info a DNG of silent pic can retain. For comparison i have uploaded full resolution CR2 n Silent pic DNG.

Silent Pic DNG: https://www.dropbox.com/s/gzwtt1u9itlx729/Ipad%20SIlent%20pic.DNG?dl=0

Normal CD2: https://www.dropbox.com/s/u69rfn51u61j13x/Ipad%20CR2.CR2?dl=0
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Walter Schulz on February 08, 2015, 09:05:30 PM
Both loaded in Photoshop. White balance adjusted and exposure +2 EV.
Which problem do you have here?

(http://picload.org/image/cipwlcp/bild3.jpg)
(http://picload.org/image/cipwlwg/bild4.jpg)
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: babarasghar on February 08, 2015, 09:56:47 PM
Recovering shadows not exposure boost over all
Try to lower highlights to -100 n boost shadows to +100 u will see which has better recovery

CR2
(http://i.imgur.com/EiB2zKf.jpg)

Silent pic
(http://i.imgur.com/erXh57u.jpg)
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Walter Schulz on February 08, 2015, 10:29:56 PM
And here we are running in circles: http://www.magiclantern.fm/forum/index.php?topic=5240.msg134066#msg134066
When you talk about DR you are actually referring to SNR (signal to noise ratio).
As dmilligan pointed out you have no proof of less DR delivered but shown what he admitted: Binning Skipping is diminishing image quality. If you want to compare DR you have to use silent pic in crop mode.
Title: 14bit RAW DNG silent pics! (silent.mo)
Post by: babarasghar on February 08, 2015, 10:42:00 PM
Quote from: Walter Schulz on February 08, 2015, 10:29:56 PM
And here we are running in circles: http://www.magiclantern.fm/forum/index.php?topic=5240.msg134066#msg134066
When you talk about DR you are actually referring to SNR (signal to noise ratio).
As dmilligan pointed out you have no proof of less DR delivered but shown what he admitted: Binning is deminishing image quality. If you want to compare DR you have to use silent pic in crop mode.
Thanks for ur time brother I'm bothering u for last two days
So whatever it is if I'm naming it wrongly, I don't mean to offend you or team of developers. So is there any way to get rid of this SNR? Especially I'm talking with reference to Milky Way shots. Can we even shoot long exposures in silent picture (I've read max is 15 sec) if not then ISO will have to be high meaning very little recovery from shadows/foreground.
Same goes to day light shots, which can be addressed with using ND filters but night gonna be an issue especially Milky Way
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Walter Schulz on February 08, 2015, 10:49:25 PM
SNR: Dmilligan and I already pointed that out. Use crop mode. No binning skipping. Or wait for FRSP.
15 sec: At time of writing you have to live with that limitation.

EDIT: And if you do longer exposures you are not that prone to mirror/shutter wear. 8 h session will require 960 shots and tearing down the mechanics at that rate will take some time.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: babarasghar on February 08, 2015, 10:51:34 PM

Quote from: Walter Schulz on February 08, 2015, 10:49:25 PM
SNR: Dmilligan and I already pointed that out. Use crop mode. No binning. Or wait for FRSP.
15 sec: At time of writing you have to live with that limitation.

Plz pardon me what's is crop mode n what's binning? (Plz don't ban me if its very basic question)
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: dmilligan on February 08, 2015, 10:54:43 PM
press the zoom button
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Walter Schulz on February 08, 2015, 11:01:02 PM
Binning: Sorry, my wrong. Mixed up terms badly.
Actually all cams (but 5D3 and 7D2. Don't know about 750/760D yet) use line skipping. When doing native Canon video mode they will only record every third line and skip the rest. Canon uses some denoising algorithm and sharpening and up/down scaling. Leading not to the best results compared to other brands.

5D3 is using binning (or better: There are strong indications for this). Lines are not skipped but put together and processed in some way. Causing much less moiré -> praise from video people.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: singlag on April 11, 2015, 04:40:21 PM
Full Resolution Silent Pic work on my Eos-M, but I have some question
1)To take a Full Resolution Silent Pic, I have to half press shutter to focus, and then press * to take a shot, is there any simple way, like Hold half press shutter for 3 second and auto take a shot ?

2) Full Resolution Silent Pic need Manual mode, but the shutter speed seem locked at 1/30sec ? Can I increase the shutter speed ?
I have tried to change FPS override to 70, but the actual shutter speed still show 30 at ML menu

Thank you for your help
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: Licaon_Kter on April 12, 2015, 01:46:46 AM
Quote from: singlag on April 11, 2015, 04:40:21 PM
1)To take a Full Resolution Silent Pic, I have to half press shutter to focus, and then press * to take a shot, is there any simple way, like Hold half press shutter for 3 second and auto take a shot ?
2) Full Resolution Silent Pic need Manual mode, but the shutter speed seem locked at 1/30sec ? Can I increase the shutter speed ?
1. Set up C.FnIV to 1 or 3 then you *-hold to focus and half-shutter press or *-press to take a pic.
2. The actual speed is 1/4 ( ~238 ms ) or worse on the M1, not sure where you see 1/30 though.
BTW, the main FRSP thread is this: http://www.magiclantern.fm/forum/index.php?topic=12523.675 (http://www.magiclantern.fm/forum/index.php?topic=12523.675)
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: chross on August 07, 2015, 01:35:39 PM
I must say, I love this module, especially the slit cam mode.
One question:
Is there any way another slit cam mode could be added? I'm talking about taking the SAME line every x secs and placing it left to right, or top-to-bottom. Right now, as I understand it, the camera will scan across the image.


Regards,
chross
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: chross on October 12, 2015, 10:21:47 PM
Hi, any news on this front?
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: hypeastrum on December 16, 2015, 09:36:30 PM
There is slit-scan mode "Horizontal" which uses single center horizontal line to build the picture. Vertical slit seems to be not supported.
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: EDGERIDER on February 26, 2019, 11:28:03 AM
hi all!

how hard would it be to do a silent mode sequence like 4K at 30fps ala go pro burst mode?
Title: Re: 14bit RAW DNG silent pics! (silent.mo)
Post by: masc on February 26, 2019, 12:41:48 PM
Quote from: EDGERIDER on February 26, 2019, 11:28:03 AM
hi all!

how hard would it be to do a silent mode sequence like 4K at 30fps ala go pro burst mode?
Already done. It is called RAW Video.