Magic Lantern Forum

Developing Magic Lantern => General Development => Topic started by: g3gg0 on April 27, 2013, 12:07:12 AM

Title: uncompressed 14-bit RAW video recording
Post by: g3gg0 on April 27, 2013, 12:07:12 AM
UPDATE:
Initially this thread was about my lv_rec module that allowed recording YUV422 and RAW video on an experimental basis.
In this module I found out a lot about the EDMAC (DMA controller) and wrote it down in our wiki. (http://magiclantern.wikia.com/wiki/Register_Map)
Since then we were able to use this high speed engine to copy portions of the image into our own buffers.

Meanwhile alex refactored all the code and optimized buffering, so that we are able to record 14 bit raw bayer data.
the result is a module named 'raw_rec' which he highly optimized to get the maximum out of our beloved canon cameras.

Since then we are constantly trying to improve the usability.
Our focused target is the 5D Mark III, but the devs are porting it to other models as you can see (thanks 1%, coutts, nanomad)

Yet this code is EXPERIMENTAL. It will cause any random failures that lead from data loss to crashing cameras.
As you know, ML is very stable, but sometimes code at this early stage causes unforseen problems.
Prepare yourself for that before you go shooting. (a backup CF card, ML-free SD card)

Key ingredients:
- canon has an internal buffer that contains the RAW data
- we understand the high speed DMA controller "EDMAC" a lot better now and know how to crop areas out of an image
- we know how to get the maximum rate out of the CF card and so achieve to get up to 90MiB/s
- we provided a reference tool that converts the Magic Lantern .RAW movie into single .DNG frames plus a MJPEG script

All together sums up to the most advanced 14-bit RAW recording system people can get for less than 3 kEUR.
We will prepare a full article as soon we see this code being stable enough for public testing.


Again. This is EXPERIMENTAL, so:
- bloody beginners and non-geeks should not touch the whole thing. wait until it is "beginner-proof". we will tell you on the website.
- you know that you are a bloody beginner, when you read the whole thread and you still cannot get it to work.
- DON'T be disappointed if it doesn't work or we figure out the whole thing is unstable and/or unusable
- NO, there is no manual yet
- NO, there is no all-in-one tool that fits every use case
- NO, we don't have tutorials how to use it
- NO, not all models are supported yet ;)
- we are just at the moment testing how good it works and what we have missed and what to improve
- you are welcome to post comparisons, experiences (both good and bad), or even deep analysis or just cool videos
- if you are a programmer and you see potential for improvements, grab the source and support :)

As always we want to remind you of these things:
- Magic Lantern Team will not be responsible or liable for any kind of direct or indirect damage to your camera. (nothing new anyway)
- The software provided for download is not related to Canon in any way
- Do not contact Canon about issues related with this software
- Do not blame Canon for not-implementing this feature! Why? This is no feature that is stable enough so that a company like Canon would ever release.
- Prepare for footage loss due to frame drops, tinted frames, corrupted frames etc.
- THIS IS EXPERIMENTAL. Deal with it. Don't blame anyone.

About sensor heating rumors:
The only thing that could get warmer is DIGiC and the CF circuitry, but i am sure that the power dissipation that reaches the sensor
through all that plastic housing will not have any noticeable temperature raise.

detailed: when doing that much DMA transfers and CF writing, we may cause a bit more current drain (which causes squared power dissipation)
but we do not encode any H.264 while recording, so we use less power there.
its *possible* that the CF writing will consume less energy than the encoding with H.264, which will result in *less* power consumption.
raw is being produced by the DIGiC for every single frame anyway. we "just" save it away.

still this is a *theory*, but i expect the consumption and the temperatures not to raise at all.




old post:
Currently i am working on a module that records YUV422 data to card.
This code will only work when compiled from repository (there is no release yet)

5D3: can record 1904x1274 @ 12.5 fps

here some example video:
https://docs.google.com/file/d/0BwQ2MOkAZTFHdU1tR1pITXFVVXM/edit?usp=sharing
(not sure how to make it look better and not take 600MiB)

here some sample images:
https://docs.google.com/file/d/0BwQ2MOkAZTFHdFFsV1BGU0Nmd2s/edit?usp=sharing

there are three major options
- Frame skipping: record every n-th frame. choose 2 on 5D3 in 25 fps mode to record with 12.5 fps *continuously*
- Single file: save some processing time by writing a single file. you have to split it later on your computer. (maybe the 422 converters will somewhen support this?)
- RAW mode: not working yet, just saving gibberish ;)

right now the module is not user-friendly. press start and it will record 2000 frames.
it will abort if the buffers are exhausted.
you can also abort by removing battery ;)

All questions regarding RAW VIDEO POST PROCESSING on any OS
HERE: http://www.magiclantern.fm/forum/index.php?topic=5404.0

All questions regarding RAW_REC MODULE COMPILATION/INSTALLATION/USAGE
HERE: http://www.magiclantern.fm/forum/index.php?topic=5405.0

And this thread here should from now on just be used for video results, and open discussion.
NOT for any installation or processing help.
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 27, 2013, 12:14:24 AM
Working here... stops after the buffer fills. I dunno how FPS is yet or how to split up the 422 files. Got 39 frames as single, 33 as multiple.
Title: Re: uncompressed YUV422 video recording
Post by: Yoshiyuki Blade on April 27, 2013, 01:23:26 AM
Quote from: g3gg0 on April 27, 2013, 12:07:12 AM
here some example video:
https://docs.google.com/file/d/0BwQ2MOkAZTFHdU1tR1pITXFVVXM/edit?usp=sharing
(not sure how to make it look better and not take 600MiB)

You can try compressing with x264. I compressed the 45 frames in the other zip file and it came out to about 25MB lossless.
Title: Re: uncompressed YUV422 video recording
Post by: g3gg0 on April 27, 2013, 01:31:40 AM
for the posted video i used these settings (x264 in virtualdub):
--profile high444 --qp 0 --preset placebo --tune stillimage --demuxer raw --input-csp i420 --input-res %(width)x%(height) --fps %(fpsnum)/%(fpsden) -o "%(tempvideofile)" -

edit: damn, i used i420 instead of i422. retrying
edit2: causes image distortion :
Title: Re: uncompressed YUV422 video recording
Post by: Yoshiyuki Blade on April 27, 2013, 01:51:40 AM
Here's an output I made at 4:2:2, PC levels (0-255)
....

The image will probably look darker than it should because the levels aren't rescaled to TV (16-235), similar to how a clip from a Canon camera would look unprocessed.

Used these settings from MeGUI: --preset veryslow --crf 0 --colorprim bt709 --transfer bt709 --colormatrix bt709 --output-csp i422

Here's a "standard" 4:2:0 at TV levels. it should look as intended:
....

Edit: (Links removed due to excessive dropbox traffic causing suspension. :o)

Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 27, 2013, 05:25:07 PM
for the old school 422 single frames I just threw them into after effects as an image sequence and exported to whatever... cineform, h264, dnxhd, animated gif, lol.
Title: Re: uncompressed YUV422 video recording
Post by: g3gg0 on April 27, 2013, 11:27:40 PM
added crop modes to record 1904x720 @ 25 fps.

the module is recording into a single file where all frames are concatenated.
use a splitting tool to split to single files. the frame size is displayed in menu.

maybe a popular .422 converter will add a feature to split the files automatically?
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 27, 2013, 11:42:39 PM
With crop and some frame skip it keeps going with only 1 frame buffered.

There needs to be a way to stop this thing tho because when it keeps going I have to turn the camera off.

I can split on linux with HJsplit but need to know the frame size.
Title: Re: uncompressed YUV422 video recording
Post by: g3gg0 on April 27, 2013, 11:53:41 PM
> There needs to be a way to stop this thing tho because when it keeps going I have to turn the camera off.
>> stop recording when leaving LV and movie mode
>> https://bitbucket.org/hudson/magic-lantern/commits/a0b2ac64c625356f618757f1d48d938406f5e770

> I can split on linux with HJsplit but need to know the frame size.
>> the frame size is displayed in menu.
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 27, 2013, 11:56:32 PM
So just shut down LV?


Ha, looks like anamorphic. Splitting first one..

Heh, frame size is only in WxH... i went into a hex editor and found start then start again and took 1 frame out to get size.
Title: Re: uncompressed YUV422 video recording
Post by: g3gg0 on April 28, 2013, 12:02:47 AM
Quote from: 1% on April 27, 2013, 11:56:32 PM
Heh, frame size is only in WxH... i went into a hex editor and found start then start again and took 1 frame out to get size.

no just see the help when you are over the resolution.
it tells you the frame size.

Quote from: 1% on April 27, 2013, 11:56:32 PM
So just shut down LV?
yep, just leave LV or video mode.
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 28, 2013, 12:09:28 AM
Ah... I see it. Totally usable... wonder if it will roll with audio. Maybe would be in sync if lv_rec threw it the semaphore when writing the first frame. Also then you'd have FPS, get X frames over the lenght of the audio.

Heh, need more frame skip when audio is recording... maybe for 5d3 wav can go to SD and video to CF... but 6D is fucked. At least the audio didn't skip.
Title: Re: uncompressed YUV422 video recording
Post by: g3gg0 on April 28, 2013, 12:21:00 AM
hmm then we would have to lower the resolution a bit more.
with 720 lines its at the limit already.

yeah, writing to SD is a good idea.
thought about doing that for higher fps or resolution, but didnt think about audio :)
Title: Re: uncompressed YUV422 video recording
Post by: squig on April 28, 2013, 12:33:51 AM
Is the YUV422 displaying the same level of detail as the RAW video?

If so and if you're able to lower the vertical resolution a little it's not going to be a drama to upscale the image a little in post. Cropping it horizontally to 500 pixels for a 1.5x anamorphic mode is gonna produce better resolution.
Title: Re: uncompressed YUV422 video recording
Post by: g3gg0 on April 28, 2013, 12:44:08 AM
cropping horizontally isnt feasible as it will reduce write speed a lot.

well, it will be possible if EDMAC is understood better.
but for now its not doable.
Title: Re: uncompressed YUV422 video recording
Post by: squig on April 28, 2013, 12:52:37 AM
Can you get the audio recording 8bit?
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 28, 2013, 01:15:18 AM
Sure, you don't save that much though and it sounds like shit.
Title: Re: uncompressed YUV422 video recording
Post by: squig on April 28, 2013, 01:24:47 AM
8bit doesn't actually sound that bad, we used 8bit samplers back in the day. 22khz sounds like shit. I'm just thinking in terms of a sync track not audio you'd actually use. Is there a way you could just record the first 10 seconds of audio for sync?
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 28, 2013, 03:19:21 AM
Probably, you can start/stop it at will.

I bet tho 8 bit vs 16 bit won't matter. The write rate just doesn't seem to be there. I only have ~150MB of buffer... maybe 5d3 has more.
Title: Re: uncompressed YUV422 video recording
Post by: squig on April 28, 2013, 07:33:01 AM
If you guys need help testing builds on the MK3 I'm up for it. I don't know jack about code but I'm a fully qualified crash test dummy. I've got a 1000x Lexar card.
Title: Re: uncompressed YUV422 video recording
Post by: Kabuto1138 on April 28, 2013, 08:44:47 AM
I'm with squig, I have a 5d2(don't know if this patch is working for the 5d2) and I also have a 1000x Lexar card
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 28, 2013, 08:57:01 AM
Should it write start time and end time to the file name?
Title: Re: uncompressed YUV422 video recording
Post by: Michael Zöller on April 28, 2013, 01:46:21 PM
Quote from: 1% on April 28, 2013, 08:57:01 AM
Should it write start time and end time to the file name?
Its nice to have. Audio wise many use an external recorder and sync with Pluraleyes. Just buffer-recording the first (and last?) two seconds might be enough for it to sync. But we wouldn't store audio into the resulting video file anyway, would we?
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 28, 2013, 06:02:17 PM
When I record wav with it, run time is reduced. 
Title: Re: uncompressed YUV422 video recording
Post by: N/A on April 28, 2013, 06:32:27 PM
Uncompressed video with audio would be ok BUT I'd sacrifice audio to have better video. And besides, you can still do the clapboard trick if absolutely necessary.

Edit: First page!  ;D
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 28, 2013, 07:49:24 PM
Hehehe, well then don't record audio :)

I need to try recording zoom image too... bigger width, lower height.
Title: Re: uncompressed YUV422 video recording
Post by: N/A on April 28, 2013, 09:13:12 PM
As in uncompressed video AND 3x zoom? Hmmmm....

Oh and once you record audio through Apogee/RME converters, nothing quite compares  ;)
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 28, 2013, 09:46:26 PM
Yea, dunno what it will look like. I'm trying to get 6D under control.
Title: Re: uncompressed YUV422 video recording
Post by: wolf on April 28, 2013, 10:57:26 PM
I wonder if Black and White could lower the amount of data?
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 28, 2013, 11:00:52 PM
Like throw all the chroma away? I dunno what does it do to silent pics on the cameras where that digic register for B/W works.
Title: Re: uncompressed YUV422 video recording
Post by: N/A on April 28, 2013, 11:12:19 PM
Still working on your bitrate hack version or has this taken priority? Or shall you be combining them, if possible?
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 29, 2013, 12:36:49 AM
Still working on everything. Just takes a while... 3 cameras at once.
Title: Re: uncompressed YUV422 video recording
Post by: N/A on April 29, 2013, 01:04:54 AM
No doubt, keep up the awesome work.
Title: Re: uncompressed YUV422 video recording
Post by: g3gg0 on April 29, 2013, 01:49:26 AM
added code to save RAW files on 5D3 now ;)
you have to call("lv_save_raw", 1) before running my code. (place it in run_test for example)

you should save it into separate files, not one single file.
there are some problems that cause file corruption. (if someone finds it, please fix ;) )

how it works:
i am using two extra EDMAC channels that are not used.
one is set up to read data from lv_save_raw buffer and write to connection #6
and the other is set up to get data from that connection and write it to my ringbuffer.
Title: Re: uncompressed YUV422 video recording
Post by: Rush on April 29, 2013, 07:03:08 AM
Quote from: g3gg0 on April 29, 2013, 01:49:26 AM
added code to save RAW files on 5D3 now ;)
Do you mean you had implemented "continuous" RAW recording?  ???
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 29, 2013, 07:14:16 AM
The lv rec writes faster so it will be better I guess. I don't see code for it yet
Title: Re: uncompressed YUV422 video recording
Post by: g3gg0 on April 29, 2013, 04:25:56 PM
oh, true. thought i committed it already.
will do later.
Title: Re: uncompressed YUV422 video recording
Post by: g3gg0 on April 29, 2013, 07:10:50 PM
Commited to repo
Title: Re: uncompressed YUV422 video recording
Post by: nanomad on April 29, 2013, 08:49:54 PM
Thanks :D

Gonna play with it tonight
Title: Re: uncompressed YUV422 video recording
Post by: g3gg0 on April 29, 2013, 09:55:07 PM
didnt implement crop profiles yet, you have to set it manually.
about 600 should result in 720p raw video.
as said, writing into a single file causes some data corruption. didnt investigate it yet.
Title: Re: uncompressed YUV422 video recording
Post by: g3gg0 on April 30, 2013, 12:51:22 AM
okay should work now :)

720p, 400 frames, raw @ 24fps should be no problem with a fresh formatted 1000x card.
maybe i can improve it a bit, but for now it is quite useable.
Title: Re: uncompressed YUV422 video recording
Post by: 1% on April 30, 2013, 12:59:19 AM
I have to find stubs and ifdef the sizes. I shot some today, looked good, most 720P takes 100fps+

Shot this with LVrec today:

Title: Re: uncompressed YUV422 video recording
Post by: Rush on April 30, 2013, 08:51:06 AM
Quote from: g3gg0 on April 30, 2013, 12:51:22 AM
720p, 400 frames, raw @ 24fps should be no problem with a fresh formatted 1000x card.
Great! g3gg0, you are genius!
Is it 1280x720 or 1920x720?
Title: Re: uncompressed YUV422 video recording
Post by: a1ex on April 30, 2013, 09:35:17 AM
Since there's no 1280 raw in 5D3, it's 1920 (well, 2080).

I'd like to test a hypothesis that might speed up the writing speed a little bit, can you try these benchmarks?
https://bitbucket.org/hudson/magic-lantern/commits/2a8021e59953

It's important to run them in LiveView (photo mode) with GlobalDraw OFF, otherwise we'll probably compare apples with oranges.
Title: Re: uncompressed YUV422 video recording
Post by: squig on April 30, 2013, 09:48:35 AM
You've already tested the Lexar 1000x yeah?

I haven't done the benchmark tests but back when I was doing the VBR testing the Lexar beat the fastest Sandisk extremes.

I'm gonna get one of those new Toshiba 1066x cards as soon as I can find somewhere that sells them (there's some listed on ebay outta HK but it looks a bit dodgy).

From the footage I've seen the 720p blows the H.264 1080p away for resolution, and with anamorphic glass that 720p IS 1080p. I'm just not clear on which footage is YUV and which is RAW. Can everyone please specify when they post footage.
Title: Re: uncompressed YUV422 video recording
Post by: a1ex on April 30, 2013, 09:51:35 AM
I've only tested the 266x card and got 20MB/s.

QuoteI'm just not clear on which footage is YUV and which is RAW

This means there isn't any noticeable difference, so why bother? ;)
Title: Re: uncompressed YUV422 video recording
Post by: squig on April 30, 2013, 09:59:52 AM
Quote from: a1ex on April 30, 2013, 09:51:35 AM
I've only tested the 266x card and got 20MB/s.

This means there isn't any noticeable difference, so why bother? ;)

Well I don't know that yet unless you're saying that's true. What I'm saying is I like to be Walter White with these things and if people don't specify what their recording settings are I can't analyse their tests, but they are nice to look at.
Title: Re: uncompressed YUV422 video recording
Post by: Pelican on April 30, 2013, 10:03:22 AM
Quote from: a1ex on April 30, 2013, 09:35:17 AM
I'd like to test a hypothesis that might speed up the writing speed a little bit, can you try these benchmarks?
https://bitbucket.org/hudson/magic-lantern/commits/2a8021e59953

Could you please write the camera model, card maker and card model on the screen so easier to identify which test result belongs to which camera and card?
Title: Re: uncompressed YUV422 video recording
Post by: a1ex on April 30, 2013, 10:06:20 AM
Will do tonight.
Title: Re: uncompressed YUV422 video recording
Post by: a1ex on April 30, 2013, 12:12:51 PM
Just as I thought - writing is a bit faster for round buffer sizes - by around 10MB/s. Not bad.

http://i.imgur.com/TBBxVJj.png
Title: Re: uncompressed YUV422 video recording
Post by: squig on April 30, 2013, 12:18:41 PM
What's that the Lexar 1000x on the MK3?
Title: Re: uncompressed YUV422 video recording
Post by: a1ex on April 30, 2013, 12:38:11 PM
Yes. Just added card/camera info to the screenshots, but not tested, no camera with me right now.
Title: Re: uncompressed YUV422 video recording
Post by: Audionut on April 30, 2013, 01:07:30 PM
Sandisk 60MB/s 16GB CF - 5D3

(https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/more/VRAM2.BMP)
Title: Re: uncompressed YUV422 video recording
Post by: a1ex on April 30, 2013, 01:40:35 PM
If my math is not broken, you should be able to 576 lines of raw data with a 2MB buffer, at 24/25p.

2*1024*1024 / (2080*14/8) => 576
2*1024*1024 * 24 => 50331648 < 51.2 * 1024 * 1024
Title: Re: uncompressed YUV422 video recording
Post by: Pelican on April 30, 2013, 02:28:42 PM
Quote from: a1ex on April 30, 2013, 12:38:11 PM
Yes. Just added card/camera info to the screenshots, but not tested, no camera with me right now.
Thanks!
Title: Re: uncompressed YUV422 video recording
Post by: RenatoPhoto on April 30, 2013, 03:25:37 PM
Results with 1000x

Lexar Professional 1000X 32GB with 5D3

2048K 85.3
2048K 128
1953K 85.3
1953K 128
3072K 93
3072K 128
4096K 113.7
4096K 146.2
3906K 93
3906K 146.2
16384K 113.7
malloc allocation error
15625K 113.7
malloc allocation error
128K 33
128K 37.9


Title: Re: uncompressed YUV422 video recording
Post by: a1ex on April 30, 2013, 03:28:51 PM
Can you post the screenshot? it's on the card.
Title: Re: uncompressed YUV422 video recording
Post by: Greg on April 30, 2013, 05:18:26 PM
500D + Sandisk Extreme 45MB/s 16GB

(http://s1.postimg.org/89upt8d1r/TEST.png)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on April 30, 2013, 06:44:13 PM
Patriot EP card GD OFF:

(http://i.imgur.com/G8fJN6Vl.jpg) (http://imgur.com/G8fJN6V)

Looks like on SD you want to use the sizes the card performs best at. CF doesn't have this problem? Like I'm better off writing 4 frames at once? I did another test with GD on 1080P and seemed way slower on the smaller buffers (lost from auto-power off). Should it be turning GD off before taking the pics or is freezing LV enough?





Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on April 30, 2013, 06:57:38 PM
To write 4 frames at once, you need to put them in a contiguous block. That's hard, because you also want to crop them, and right now the EDMAC writes full frames only.

Memcpy is way too slow. You would have to configure the cropping directly in the EDMAC (no idea how).
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on April 30, 2013, 07:08:12 PM
Why isn't it using DMA memcpy? That fixed audio for me.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on April 30, 2013, 07:22:09 PM
Try it, but you would have one copy from edmac and another from dma_memcpy (which may slow down the write speed, compared to just copying from edmac). Plus non-trivial task synchronization.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on April 30, 2013, 07:28:54 PM
I only see memcpy for the header and then I see edmac redirected to write to the buffer and then buffer written with fio write file?

*So can speed up header writing for sure but dunno how slow that is.

Sped up the header writing.. maybe slightly faster to save the files once the buffer is full.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on April 30, 2013, 07:36:50 PM
I don't see any memcpy in lv_rec, and save_dng was never intended for real-time recording.

For real-time, all you have to do is to dump the image data without any processing (and that's what lv_rec does).
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on April 30, 2013, 07:41:19 PM
I'm getting better sync from your dng writer.. the video looks more fluid so i was messing with that. I have to try the rewrite g3ggo did to lv_rec... it writes with the DMA directly I think, no way to speed that up.

Ok, tested the module with raw... into a single file getting pretty much double rate. I have to look at the files. Almost 60 frames of "hd". Also turning off GD seems to help a bit.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on April 30, 2013, 09:04:45 PM
you can skip frames (half or third frame rate) to have continuous video recording.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on April 30, 2013, 10:25:03 PM
I dunno how frame skips will play out in terms of motion quality. I guess I'll see.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: duncanidaho25 on May 01, 2013, 03:43:32 AM
Is this helpful?  If not, what kind of test can I run to help out?

https://vimeo.com/65192979
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ozcancelik on May 01, 2013, 01:07:07 PM
Kingston Ultimate 600x CF Card

(http://i.imgur.com/xZYvXDw.gif)

Product specs say "90 Mb/s write and read speed" But seems like it's not
http://www.bhphotovideo.com/c/product/738080-REG/Kingston_CF_32GB_U3_32GB_Ultimate_CompactFlash_600x.html

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: RenatoPhoto on May 01, 2013, 03:41:50 PM
Lexar 1000x  32 gig with 5D3

(http://s20.postimg.org/fkjz1vfe5/5_D3_Lexar_1000_X.jpg)


Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kfprod on May 01, 2013, 04:17:32 PM
I'm new to all this but couldn't you use the second card slot of the camera and write every second frame to that, sort of as a RAID config?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 01, 2013, 04:19:15 PM
How do I open the .raw files from lvrec? They have no headers. Photoshop wants a size but when I put in the dimensions and 16bit 3 channels it says file too large.

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: RenatoPhoto on May 01, 2013, 04:48:37 PM
It seems that digital crop video would be one step closer with the new zoom+silent pictures?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Mati on May 01, 2013, 05:59:17 PM
I'm new here, hello everyone :).
My test conducted on the SanDisk Extreme 60MB/s UDMA 16GB, Canon 5D Mark II.
I do not know how umieściś same picture so here 's the link to the test : http://www.sendspace.com/file/q5i5yl
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: HugoFilipe on May 01, 2013, 08:33:51 PM
This will sound completly noob, i know, i am one.
I've put the 29th April firmware on my 5D3 but i only manage to get a couple of frames (below 50) with 720p and fps overwrite to 24.
I'm using a Sandisk Extreme Pro 90MB/s 32GB.

What i'm doing wrong? 400 frames would do it fine por me!

Thank you in advance!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 01, 2013, 09:29:14 PM
Try global draw off. 400 frames = good luck. I get a little over 50 on sd card. Almost twice the amount with new lv_rec but dunno how to containerize the files so something will open them.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: CaptainHook on May 01, 2013, 10:37:52 PM
Is it just not possible to get the 14-bit Raw feed sent uncompressed to the HDMI out? Or the uncompressed YUV422 to the HDMI out?
Would the YUV422 to HDMI out (if even possible) look better than canon's version?

Sorry to cross-post this, but if you look at the BMCC's Prores recording done on a hyperdeck shuttle (via SDI out of the camera), i imagine IF the 14-bit Raw feed could be sent out the HDMI it would be so much better (than current 5D3 options) and you wouldn't have to worry about write speeds of CF/SD cards:

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 01, 2013, 10:58:18 PM
HDMI chip is probably an off the shelf solution. I doubt its so easy to just feed it a different buffer when it does processing on the data.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: HugoFilipe on May 01, 2013, 10:58:53 PM
Quote from: 1% on May 01, 2013, 09:29:14 PM
Try global draw off. 400 frames = good luck. I get a little over 50 on sd card. Almost twice the amount with new lv_rec but dunno how to containerize the files so something will open them.

I've tried some combinations, the best i get is 29 frames at 1080 or 49 frames at 1920x670 (image looks stretched!)
Whatever the settings i make, the camera only starts to write to the card after the buffer is full, so whatever fps I set, i only get those frames.
Is this normal?

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: CaptainHook on May 01, 2013, 11:07:40 PM
Quote from: 1% on May 01, 2013, 10:58:18 PM
HDMI chip is probably an off the shelf solution. I doubt its so easy to just feed it a different buffer when it does processing on the data.

I see. Be good to find out either way. Thanks! :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: VectorZ on May 02, 2013, 02:25:11 AM
Does SRaw mode do anything to help this situation?

Theoretically it could reduce your resolution sampling, or perhaps offer some alternate compression scheme?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 02, 2013, 02:53:45 AM
I think those settings don't affect the DNG or YUV image.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: duncanidaho25 on May 02, 2013, 02:58:58 AM
I'm sorry, I know YUV is a colorspace, but what do you mean YUV as opposed to the DNG files? 
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 02, 2013, 03:03:09 AM
There is a YUV buffer there and the buffer where DNG is copied from. Neither should be affected by mraw or sraw selected for photos.
Title: Re: uncompressed YUV422 video recording
Post by: RenatoPhoto on May 02, 2013, 03:08:44 AM
Can anyone explain how to get this speed?  Has anyone attained it?

Quote from: g3gg0 on April 30, 2013, 12:51:22 AM
okay should work now :)

720p, 400 frames, raw @ 24fps should be no problem with a fresh formatted 1000x card.
maybe i can improve it a bit, but for now it is quite useable.
Title: Re: uncompressed YUV422 video recording
Post by: P337 on May 02, 2013, 07:29:59 AM
Quote from: RenatoPhoto on May 02, 2013, 03:08:44 AM
Can anyone explain how to get this speed?  Has anyone attained it?

Wait, by "RAW" do you mean "14bit 4:4:4" or "8bit 4:2:2 uncompressed"?
Title: Re: uncompressed YUV422 video recording
Post by: Francis on May 02, 2013, 07:38:38 AM
Quote from: P337 on May 02, 2013, 07:29:59 AM
Wait, by "RAW" do you mean "14bit 4:4:4" or "8bit 4:2:2 uncompressed"?

By RAW he means RAW. As in RAW image sensor feed. Prior to any color space or compression. 14bit data
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 02, 2013, 08:05:41 AM
Lvrec is working on 600D... only allocating like 70MB though, shoot malloc is supposed to be 104mb? Or is this more of that wasted memory a1ex was mentioning? Only 16 frames == boooooo

Ok, 80 frames of 1080pX480

Around 120frames in 720P

I guess numbers will be the same with DNG?

You can play yourself:

https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads
Title: Re: uncompressed YUV422 video recording
Post by: P337 on May 02, 2013, 08:58:03 AM
Quote from: Francis on May 02, 2013, 07:38:38 AM
By RAW he means RAW. As in RAW image sensor feed. Prior to any color space or compression. 14bit data

Are you sure?  Because this is the "uncompressed YUV422 video recording" thread and it's a very common mistake to make.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 02, 2013, 09:36:40 AM
It saves both dng and yuv... a little bit hard-coded for 5d3 and you have to enable lv debug... I haven't been able to process any of the "dng" files yet as they have no header, nothing wants to open them.

YUV and DNG file sizes are similar.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 02, 2013, 10:18:54 AM
Hi 1%,

How are you recording DNG at 720p on your 600d? Nothing I'm doing seems to alter the frame size.

All my DNGs are above 2k and buffer maxes out at 14-18 shots. Not sure if my settings are correct or if it's card speed related.

BTW F**king huge thanks for this :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: P337 on May 02, 2013, 10:21:16 AM
Quote from: 1% on May 02, 2013, 09:36:40 AM
It saves both dng and yuv... a little bit hard-coded for 5d3 and you have to enable lv debug... I haven't been able to process any of the "dng" files yet as they have no header, nothing wants to open them.

YUV and DNG file sizes are similar.

Cool, can a post a YUV and DNG file so we can try to open them?  If Photoshop can't open them I take it they aren't normal DNG files.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 02, 2013, 10:34:33 AM
Quote from: Andy600 on May 02, 2013, 10:18:54 AM
Hi 1%,

How are you recording DNG at 720p on your 600d? Nothing I'm doing seems to alter the frame size.

All my DNGs are above 2k and buffer maxes out at 14-18 shots. Not sure if my settings are correct or if it's card speed related.


BTW F**king huge thanks for this :)

Doh, I hadn't changed to small/medium. Still only getting 22 frames before buffer fills though. Must be the card.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 02, 2013, 10:41:56 AM
i am very busy right now, but let me explain.

this module can record two things:
- YUV422
- 14 bit RAW bayer RGBG

both can be recorded either in
- single file (splits at around 2GiB)
- separate files

both formats have different resolutions, come from different sources and have different format.

the YUV422 format can be read by popular 422-to-X tools like 422ToImg etc.

the RAW format can not be processed yet, except if you have a tool that alex coded for PC.
it is not in a release-able state yet i think. we are working on letting you process RAW frames soon.
Title: Re: uncompressed YUV422 video recording
Post by: P337 on May 02, 2013, 11:04:54 AM
Quote from: g3gg0 on April 30, 2013, 12:51:22 AM
okay should work now :)

720p, 400 frames, raw @ 24fps should be no problem with a fresh formatted 1000x card.
maybe i can improve it a bit, but for now it is quite useable.

Wow, so you really did mean 400 14bit files then?  That's all I'd need :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 02, 2013, 11:20:19 AM
Although my first attempts at shooting DNG with the 600d are limited to around 1 second of footage, I think this development is simply amazing.

If nothing else, it gives us Canon users a little taste of shooting and working with DNG. That in itself might put a few people off when considering moving up/over to a RAW shooting workflow (i.e. the BMCC) but for me, this little insight is making me hungry for it.

I understand the limitations (if not the complexities of the coding) but I'd be more than happy with getting a few seconds at 720p DNG.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N.Mendes on May 02, 2013, 11:31:29 AM
Here is a great new article from eoshd about "raw video using Magic Lantern"..
http://www.eoshd.com/content/10294/3-5k-canon-5d-mark-iii-raw-video-with-magic-lantern-and-latest-updates

He seems to be hopeful about the soon future ..

With all improvements you made these days, are you (the ML team) optimistic about seeing a real 2k raw @24 fps on the 5d3? (when i say real, i mean a continuous recording video, or at least 1mn..)

Strength and courage my friends..
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N/A on May 02, 2013, 02:51:48 PM
With single file YUV, 720p, FPS override on 24p, Large, I'm managing 127 frames on the 600D with the Sandisk 95.

Next step. Figuring out how to look at the damn things. On a Mac, and I'm too slow to get the 422 windows program running on here.

Blah.

Oh and with the same settings into separate files, looks like 98 frames.

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Alia5 on May 02, 2013, 03:08:01 PM
on my 600D raw-recording crashes
Log:

mode:20


Magic Lantern version : v2.3.NEXT.2013May02.600D102
Mercurial changeset   : 6f2b596d2b19+ tip
Built on 2013-05-02 05:58:15 by user@D610.
Free Memory  : -1733647K + 1255K




Edit: Crash only appears in crop modes
Getting 16Frames normally
and 26 in 720p mode


720p modes are stretched, but edit normally in adobe camera-raw
Setting fps-override to low, will result in coruppt files :(


Getting about 30frames in yuv, single file
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: dude on May 02, 2013, 03:08:46 PM
ot:
eoshd is one of the worst sites ever... absolutely no idea what he is talking about.
seems like panasonic does not sponsor him any more, becaus he was against canon all the time.
and people who believe in tests lie "gh2 vs arri alexa" are not professionell at all
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 02, 2013, 05:26:23 PM
Raw recording needs lv debug enabled... Also its still hardcoded to 5d3 for the 600D build.

Any chance to manually put the container on the raw files? I assume thats why they won't open.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N.Mendes on May 02, 2013, 05:37:36 PM
Can we translate your "technical words" by, "any chance to obtain a real 2k raw @24 fps on the 5d3"?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 02, 2013, 05:44:11 PM
Its not that good yet... although the 720P isn't far off from 1080P and can be stretched quite far.

You're getting like 2s at 24fps... maybe 1 on "2K" 1080 or zoom mode. CF card cameras will have more luck here, also depending on how much memory they have. 5d3 probably getting best frame rates so far with 6D close second and maybe 50D when it works will have enough memory to match or beat it since it has CF.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N.Mendes on May 02, 2013, 06:04:05 PM
The only thing i can say is, "good luck my friends", you're doing your best, and everyone knows that..

I'm french, and here in Paris, everybody is talking about what you're doing, about the Bmcc too, because your exploit could change the game..

Some are saying that the maximum you can obtain is a few seconds @ 24fps 2K raw, some others are saying that you can do that

Let's the time tell us who is right..

Again, Good luck from Paris
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 02, 2013, 06:06:48 PM
Also for 600D how is the raw histogram looking... I'm not sure that its accurate. It looks squished and sometimes black level detects as 0.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Alia5 on May 02, 2013, 06:08:55 PM
Quote from: 1% on May 02, 2013, 05:26:23 PM
Raw recording needs lv debug enabled... Also its still hardcoded to 5d3 for the 600D build.

Any chance to manually put the container on the raw files? I assume thats why they won't open.

The RAWs are also corrupt in camera (when it saves them), i guess they really are broken, havent looked to deep into it though
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 02, 2013, 06:31:24 PM
Most work ... I opened one 1740x1154 I think thats crop mode... also 1804x1020 works. I did not check with fps override yet... I basically compiled the bin and shot a few pics then went to sleep.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 02, 2013, 07:00:48 PM
I think there's a bug when changing pic size in the Canon menu (I'm using Tragic Lantern 2 on a 600d)

(http://t.imgbox.com/advgVi86.jpg) (http://imgbox.com/advgVi86)

I can still change pic size in the ML menu but the Canon menu seems to be stuck like this now. Any ideas?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 02, 2013, 07:07:38 PM
Change to valid pic size, not MRAW/SRAW from ML menu. Also change won't accept until you switch modes so switch to raw only and flip between the 2 modes you were in. Menu should go back to normal.

Plus still other bugs, like bit rate stuff isn't all done.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 02, 2013, 07:16:10 PM
Quote from: 1% on May 02, 2013, 07:07:38 PM
Change to valid pic size, not MRAW/SRAW from ML menu. Also change won't accept until you switch modes so switch to raw only and flip between the 2 modes you were in. Menu should go back to normal.

Plus still other bugs, like bit rate stuff isn't all done.

Thanks! That sorted it :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N/A on May 03, 2013, 09:29:12 PM
Something I did on the 600D yesterday resulted in the camera never buffering out with yuv, got up to 400 or so frames and had to end up doing a batt pull lol. Think it had something to do with frame skip...
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Rush on May 03, 2013, 09:36:45 PM
Quote from: N/A on May 03, 2013, 09:29:12 PM
Something I did on the 600D yesterday resulted in the camera never buffering out with yuv, got up to 400 or so frames and had to end up doing a batt pull lol. Think it had something to do with frame skip...
You need a lot of frame skipping to make 600D to write continuously. It is sad, that 600D SD is limited to only 21 mb/s. So - only very short bursts (or timelapse) are usable.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 03, 2013, 09:54:46 PM
Is it possible to try YUV422 on 6d? How much those 600d yuv422 frames weights?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N/A on May 03, 2013, 10:37:57 PM
If each frame could be truncated in-camera to ~1 MB, would still be better than h.264.

Easier said than done I suppose.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 04, 2013, 01:09:13 AM
Yep, if they could be compressed to JPEG all problems would be solved.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: EOSHD on May 04, 2013, 01:27:00 AM
Quote from: 1% on May 04, 2013, 01:09:13 AM
Yep, if they could be compressed to JPEG all problems would be solved.

Is it a code problem or a CPU time issue? Is debayering and compressing to JPEG at 24fps just not possible?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 04, 2013, 01:40:05 AM
A1ex + G3ggo said it wouldn't work real time... nobody has tried to compress it in any way tho so who knows.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N/A on May 04, 2013, 02:55:24 AM
Random thought for music videos... Play the song back at half the bpm, have the singer perform along while recording raw/yuv at 12fps, and play it back in post at 24p. Feasible? Or just a choppy fuckin mess?

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: shuggyrasklat on May 04, 2013, 04:20:19 AM
this entire vid is comprised of 5x zoom recordings, 51 frames a piece, on the 5dmkii on a 60mb/s sandisk 16gb 2152x1076 comp size, all at 135mm.

http://www.youtube.com/watch?feature=player_embedded&v=e7Pu6m8GhdM
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: P337 on May 04, 2013, 05:54:27 AM
Quote from: shuggyrasklat on May 04, 2013, 04:20:19 AM
this entire vid is comprised of 5x zoom recordings, 51 frames a piece, on the 5dmkii on a 60mb/s sandisk 16gb 2152x1076 comp size, all at 135mm.

http://www.youtube.com/watch?feature=player_embedded&v=e7Pu6m8GhdM

Hmm, still see some exposure hick-ups (or was that something actually in "the scene" or added in post processing during the 0:18-0:19 clip?) 

Was this filmed with the new "YUV/DNG recorder"?  I assume it was filmed with the older 14bit "silent burst pic" DNGs. 

Has anyone got the DNGs from the new recorder open yet? (and are the new DNGs still 14bit or just referenced from the 14bit raw image then compressed?  I don't understand yet why the DNGs from the new recorder won't open)

btw thanks for the test shuggyrasklat (looks sharp)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: P337 on May 04, 2013, 06:12:36 AM
Quote from: 1% on May 04, 2013, 01:09:13 AM
Yep, if they could be compressed to JPEG all problems would be solved.

True for bandwidth, but from a purely image quality perspective wouldn't even the "Highest Quality" JPEG (8bit, 4:2:2, 2.6:1 compression) produce slightly lower quality than the YUV422 files or HDMI out?  If we had to resort to JPEG, is there even a point to use it over YUV422 or HDMI out? (for 5D3 users especially)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 04, 2013, 06:20:47 AM
I'm tried to aggregate FIO_WriteFile , like this ..
Because, I think contenious sequential access is most faster when we write file to filesystem.

@@ -416,8 +417,12 @@

         if(!skip_saving)
         {
-            FIO_WriteFile(save_file, UNCACHEABLE(&save_data->chunkData.chunkAddress[save_data->chunkData.chunkOffset]), avail);
+            memcpy(shmem+written,UNCACHEABLE(&save_data->chunkData.chunkAddress[save_data->chunkData.chunkOffset]),avail);
             written += avail;
+            if(written == length)
+            {
+                FIO_WriteFile(save_file, shmem , length);
+            }
+
         }

         length -= avail;
@@ -451,12 +456,20 @@
     strcpy(&data->fileSuffix[1], tmp);
}

void lv_rec_start()
{
     int yPos = 3;
     lv_rec_data_t data;
     lv_rec_save_data_t save_data;

+    shmem = shoot_malloc(8000000);
+    if(shmem == NULL)
+    {
+        NotifyBox(3000,"8M shoot memory allocate failed");
+        msleep(3000);
+        return;
+    }
+
     /* set all values to zero */
     memset(&data, 0x00, sizeof(lv_rec_data_t));
     memset(&save_data, 0x00, sizeof(lv_rec_save_data_t));



But memcpy from DMA mem to shoot mem is really slow. Let me know more faster memcpy method, If you know.
Also , shoot mem is only allocated about 8MB. where located more fast and bigger memory?

Modified
5D3 has no dma_memcpy
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 04, 2013, 06:23:18 AM
dma memcpy didnt' work? shoot_malloc(0) should be the big memory
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 04, 2013, 06:31:46 AM
Quote from: 1% on May 04, 2013, 06:23:18 AM
dma memcpy didnt' work? shoot_malloc(0) should be the big memory

Anybody didn't find it. and I did a actual test like this

shoot_malloc(4M)
shoot_malloc(8M)
shoot_malloc(16M)

and check these returns. Then , I can find only 8M to use from ML.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 04, 2013, 06:36:28 AM
Specific amount won't let you allocate the big chunk, its a new process.

dma mecpy is there, its called aj blt dma or something like that, just not renamed.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 04, 2013, 07:06:08 AM
I see thanks. I will try it when I going back to home.
BTW, Alex is already suggested "memcpy is slow" , So I don't know we can get good result .
It's my hobby.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 04, 2013, 07:30:47 AM
Dma memcpy is faster but I don't know what it will do while using the dma for other things.

Also can mess with exmem.c make the step bigger than 5.. the backup has to be > than the step or it will freeze the second time you shoot.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 04, 2013, 10:20:13 AM
(not very active this weekend due to family events)

there are 3 possibilits for memcpy:
- CPU: Load and Store words using more or less fast opcodes (LDMIA, STMIA with many registers)
- DMA: simple DMA controller
- EDMAC: special SDRAM DMA controller

the last one (EDMAC) seems to be the fastest, but is a bit complex to set up as it supports line skipping and stuff.
it is used also for transferring image (raw, yuv, jpeg) data between hardware like compressors, converters etc and RAM

DMA is a simple one and seems to be slower than EDMAC but faster than CPU.

CPU is the slowest of course ;)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 04, 2013, 06:06:44 PM
Weird that dma copy == corrupt headers with dng burst. All 3 are separate, thats good at least.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: shuggyrasklat on May 04, 2013, 06:21:25 PM
P337, I shot it with the burst feature of silent picture on the 5dmkii. With the "YUV/DNG recorder," you mean with the new mkiii firmware, or is it a newer feature, within ML, that I am unaware of?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 04, 2013, 06:44:11 PM
5dII modules aren't enabled.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: shuggyrasklat on May 04, 2013, 08:45:47 PM
1%, is that being worked on?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 04, 2013, 08:58:12 PM
Hopefully... I don't have 5d2... I did 600D/EOSM (console broken) and 6D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 05, 2013, 03:02:41 AM
Quote from: g3gg0 on May 04, 2013, 10:20:13 AM
(not very active this weekend due to family events)
Enjoy with your family.

Quote from: g3gg0 on May 04, 2013, 10:20:13 AM
there are 3 possibilits for memcpy:
- CPU: Load and Store words using more or less fast opcodes (LDMIA, STMIA with many registers)
- DMA: simple DMA controller
- EDMAC: special SDRAM DMA controller
Let me confirm.
CPU
-malloc & memcpy()
DMA
-shoot_malloc & dma_memcpy()
EDMAC
-CreateMemorySuite & (not found yet)

Correct?

And now I finding address of dma_memcpy, but I can't find it yet.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 05, 2013, 03:43:03 AM
It should be
NSTUB(0xFF9EB650 - RAM_OFFSET, AJ_HP_Copy_n_DMA_channel_n_BLTDMA)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 05, 2013, 04:13:49 PM
Quote from: 1% on May 05, 2013, 03:43:03 AM
It should be
NSTUB(0xFF9EB650 - RAM_OFFSET, AJ_HP_Copy_n_DMA_channel_n_BLTDMA)

Thanks . But not enough speed..
Result is more slow with g3ggo's original method.

Test code is here.

diff -r 99bd629f7c57 modules/lv_rec/lv_rec.c
--- a/modules/lv_rec/lv_rec.c   Tue Apr 30 22:18:21 2013 +0200
+++ b/modules/lv_rec/lv_rec.c   Sun May 05 23:09:11 2013 +0900
@@ -416,8 +417,12 @@

         if(!skip_saving)
         {
-            FIO_WriteFile(save_file, UNCACHEABLE(&save_data->chunkData.chunkAddress[save_data->chunkData.chunkOffset]), avail);
+            dma_memcpy(shmem+written,UNCACHEABLE(&save_data->chunkData.chunkAddress[save_data->chunkData.chunkOffset]),avail);
             written += avail;
+            if(written == length)
+           {
+                FIO_WriteFile(save_file, shmem , length);
+            }
+
         }
diff -r 99bd629f7c57 platform/5D3.113/stubs.S
--- a/platform/5D3.113/stubs.S  Tue Apr 30 22:18:21 2013 +0200
+++ b/platform/5D3.113/stubs.S  Sun May 05 23:09:11 2013 +0900
@@ -69,7 +69,7 @@
NSTUB(0xFF9E9AF0 - RAM_OFFSET, GetMemoryAddressOfMemoryChunk)
NSTUB(0xFF9E9ED8 - RAM_OFFSET, GetNumberOfChunks)
NSTUB(0xFF9E9DF8 - RAM_OFFSET, GetSizeOfMemorySuite)
-
+NSTUB(0xFF9EB650 - RAM_OFFSET, dma_memcpy)
NSTUB(0xff9f3900 - RAM_OFFSET, AcquireRecursiveLock) // AJ_KernelDry_KerRLock.c
NSTUB(0xff9e88e8 - RAM_OFFSET, AllocateMemory)
NSTUB(0xff44c8c4, CreateDialogBox)

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 05, 2013, 04:19:26 PM
So the only other thing to try will be an equivalent to memcpy for edmac? So something like copy memory suite?

Also there are a couple of different memcpy around that one, on 6D i found a faster one but the files were screwed up for some reason (some kind of data was copied tho). Maybe poke around there and try the others for fun. Maybe the parameters are different.

There is also HPcopy.

int (*HPCopyAsync) (unsigned char *dst, unsigned char *src, int length, void (*cbr)(unsigned int), int ctx) = (int (*) (unsigned char *dst, unsigned char *src, int length, void (*cbr)(unsigned int), int ctx))0xCB80;


What is allocateHPmemory?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 05, 2013, 05:04:02 PM
Quote from: minimimi on May 05, 2013, 03:02:41 AM
Enjoy with your family.
Let me confirm.
CPU
-malloc & memcpy()
DMA
-shoot_malloc & dma_memcpy()
EDMAC
-CreateMemorySuite & (not found yet)

Correct?

And now I finding address of dma_memcpy, but I can't find it yet.

nearly :)

for CPU copy it is better to work on cached memory (address & ~0x40000000)
whereas for (E)DMAC its important to work on uncached memory (address | 0x40000000).
so it is not important if you use malloc, alloc_dma_memory or  shoot_malloc.

but whenever you work with cached/uncached you have to ensure that the caches for this areas are flushed.
so when using DMA on memories that you accessed with CPU in a cached way, you have to flush before and after CPU access.


some example code for EDMAC memcpy (didnt test it, just wrote down right now):
see wiki page Register Map for some details on EDMAC. copy size granularity for EDMAC is 4096, so sizes must be a mutiple of 4096.
in this example i chose to copy from a continuous memory block to a newly allocated memSuite buffer.
you can change it to set up two memSuites from memory blocks or whatever. its just an example.


/* pick some free (check using debug menu) EDMAC channels write: 0x00-0x06, 0x10-0x16, 0x20-0x21. read: 0x08-0x0D, 0x18-0x1D,0x28-0x2B */
dmaChannelRead = 0x19
dmaChannelWrite = 0x11

/* both channels get connected to this... lets call it service. it will just output the data it gets as input */
dmaConnection = 6

/* see wiki, register map, EDMAC what the flags mean. they are for setting up copy block size */
dmaFlags = 0x20001000


/* create a memory suite from a already existing (continuous) memory block with given size. */
struct memSuite *memSuiteSource = CreateMemorySuite(<addr>, <size>, 0);
/* allocate a memory suite that is of given size (potentially fragmented) */
struct memSuite *memSuiteDest = shoot_malloc_suite(<length>);

/* only read channel will emit a callback when reading from memory is done. write channels would just silently wrap */
PackMem_RegisterEDmacCompleteCBRForMemorySuite(dmaChannelRead, &complete_cbr, 0);

/* connect the selected channels to 6 so any data read from RAM is passed to write channel */
ConnectWriteEDmac(dmaChannelWrite, dmaConnection);
ConnectReadEDmac(dmaChannelRead, dmaConnection);

/* setup EDMAC driver to handle memory suite copy. check return codes for being zero (OK)! if !=0 then the suite size was not a multiple of 4096 */
err = PackMem_SetEDmacForMemorySuite(dmaChannelWrite, memSuiteDest , dmaFlags);
err = PackMem_SetEDmacForMemorySuite(dmaChannelRead, memSuiteSource , dmaFlags);

/* start transfer. no flags for write, 2 for read channels */
PackMem_StartEDmac(dmaChannelWrite, 0);
PackMem_StartEDmac(dmaChannelRead, 2);
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 05, 2013, 06:19:10 PM
Quote from: g3gg0 on May 05, 2013, 05:04:02 PM
see wiki page Register Map for some details on EDMAC. copy size granularity for EDMAC is 4096, so sizes must be a mutiple of 4096.

Thankyou for your kind explanation.
I think If we use DMA memory , we can't write single operation to write files.(can't aggregate FIO_WriteFile)
If we use another memorySuite instance, WriteFile operation is same as current codes.
It means not speed up .....


But I'm learned memory things, It's really helpful. Thankyou again.

Modified
Just understood your code.
Will try it.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 05, 2013, 08:25:48 PM
you are welcome.
it would be cool to see performance results of those three methods.

e.g. let them copy 2 MiB of data and check how long it took.
for EDMAC it would be better to copy 20 MiB as i guess it is very fast.

how to get timer ticks:

static uint32_t tskmon_get_timer_reg()
{
    return *(uint32_t*)0xC0242014;
}
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 05, 2013, 11:19:14 PM
Hi

is it possible to change crop amount top/bottom in LV rec? There is such option but when I am changing crop size it returns to "default" number (it doesn't save entered values). It would be nice to set such a crop to get 544 lines (1280x544 or 1840x544) and 800 lines (1840x800). It will corespond with 2.35:1 aspect ratio so one doesn't have to make black bars in post and it will save some bitrate.

Thanks
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 06, 2013, 12:29:13 AM
The non presets don't seem to work.

g3ggo just added footer data, I have to merge it and rebuild, so that solves some problems.

sRaw trick didn't work for EOS-M :(... actually Sraw didn't work at all on EOSM, just dies when you take a pic. Maybe 650D will have better luck.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 06, 2013, 05:41:31 AM
Quote from: g3gg0 on May 05, 2013, 08:25:48 PM
you are welcome.
it would be cool to see performance results of those three methods.

e.g. let them copy 2 MiB of data and check how long it took.
for EDMAC it would be better to copy 20 MiB as i guess it is very fast.

hmm, I'm now porting your pseude-code to lv_rec.c, but shoot_malloc_suite is allocating chunked memory using memSuite.
So I think , the code makes a faster memory copy , but it needs a lot of FIO_WriteFile . It means a slowing down to writing a file I guess.

So we need to find how to write a file with DMA, I think..


Or , my old school experience,.. When we write a file , make a 1Gig or 2gig file with padding 0 before writing a file.
The write () don't need to allocate  disk spaces (sector) realtime, so it's a little bit faster.
In my experience, the file created by dd on UNIX at that time. So we need to implement the function or , find fseek IMHO.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 06, 2013, 01:35:29 PM
LV rec set to YUV recording creates single *.yuv file. 422toIMg accepts *.422 files.
How are you guys processing *.yuv files to use them in premiere etc ?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: RenatoPhoto on May 06, 2013, 01:47:53 PM
Quote from: 1% on May 06, 2013, 12:29:13 AM
The non presets don't seem to work.

g3ggo just added footer data, I have to merge it and rebuild, so that solves some problems.

sRaw trick didn't work for EOS-M :(... actually Sraw didn't work at all on EOSM, just dies when you take a pic. Maybe 650D will have better luck.

Is it possible to add g3ggo work to 5D3 so I can compile it and tested?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 06, 2013, 04:59:39 PM
Main repo has it, just enable TCC and modules.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: sicetime on May 06, 2013, 05:16:19 PM
I can't find it now but I read that card speed is not the issue so would this work with sd cards as well? I am having trouble with my slow non code oriented mind, following along with the YUV 422 along side the DNG threads. Are both available for the 5d3? How come I can not find the cam on the nightly builds? tragic lantern 2.0 doesn't have a list of compatible cameras, is TL available for the 5d3? 

Also, unrelated question, has anyone used komputerbay 1000x cards? I want to pick up a few because of the amazing advancements here, but now am thinking that it would be fruitless... thoughts?

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: AlexVakulov on May 06, 2013, 05:35:04 PM
Hello to everyone!
From 600D, quality of succesful shots is amasing, but about a half of all series is broken. Why?
https://docs.google.com/file/d/0B4Dxk-qc9EApcjZpem5mbE5FRVE/edit?usp=sharing
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 06, 2013, 06:01:09 PM
Sync on 600D is not so good. It looks like its catching it in the middle of the frame. I'll do a test again now that cutting from one file is convenient.

Quotetragic lantern 2.0 doesn't have a list of compatible cameras

2.0 is 600D
6D is 6D + EOSM (needs more fixed)... you could also put back the raw size for lvrec and compile 5d3 off of it. Haven't recently though.

Yea, just checked, have this problem too. It sucks because I got over 120 frames on 720P with fps override. Ugh... 150 frames now and sync goes away after like 24. Tried display state and all parts of evf state.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Rush on May 06, 2013, 07:34:58 PM
Quote from: sicetime on May 06, 2013, 05:16:19 PMI can't find it now but I read that card speed is not the issue so would this work with sd cards as well?
Card speed is not issue for RAW 1-2 sec bursts or RAW timelapse. But if you aim to record RAW continuously (which I hope will be available) - you need super duper fast CF at least 1000x.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Michael Zöller on May 07, 2013, 09:33:56 AM
I've moved the posts regarding compilation and TCC to a new thread (http://www.magiclantern.fm/forum/index.php?topic=5324.0).
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 07, 2013, 11:18:41 AM
I made a quick comparison (with my 6D)  test in AE using linear wipe: 1080p h.264 IPB encoder VBR bitrate 10, qp 6 24fps (bitrate about 80 mbit/s) vs 1080p yuv422 24fps processed with 422toimg -single file yuv to jpeg sequence - quality 100%.
IMHO there is almost no visible difference, maybe very very small improvement only visible on 400% crop. Did you guys make such comparisons?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 3pointedit on May 07, 2013, 12:51:29 PM
A good way to detect artefacts is to mix between identical shots with a subtraction function. This way the only image data left will be the difference.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 07, 2013, 12:59:22 PM
thanks for your comparison.
i think JPEG might be a bad choice when you want to compare image quality, even with factor set to 100 :)
i am no video guy, but maybe you should compare fine details with high contrast. like grass or trees.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 07, 2013, 03:45:54 PM
Low QPs are supposed to be close to "lossless".. but I see a difference in the 4:2:0 to 4:2:2 right away. I got MZ sync'd on 600D for 24P/30P and even works when 720P is overwritten to 24 frames... I still get shifted frames tho :(

Weird... I think burst + modules use the same EVF sync... when I tried MZ sync for evf sync it produced bad burst pics.. but normally it does not... so wtf is happening to the YUV frames if burst frames are coming out perfect. Almost 160 YUV frames on 720P today... I'm hoping when raw recording is decodable it won't be doing this. I think frame sizes are similar so the yield should be too.


Ok, double weird.... if you look at the sequences with moving subjects... after 21 frames the corruption starts, first at the bottom. Then it appears after a few frames the sequence is repeating. i.e I see the same truck that passed me in a good frame in a corrupt frame further down the line. WTF, indeed.

Issue remains, sraw trick or not.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 07, 2013, 04:51:42 PM
huh?
when recording with lv_rec you get corrupted frames?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 07, 2013, 05:06:14 PM
Yea, on 600D. Just like the guy posted. After 21 frames starts to look like mis-aligned old silent pic and repeats images. 6D I haven't noticed any issues yet. Someone was complaining about something similar with 5dII, maybe affects digic IV only?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 07, 2013, 05:16:29 PM
memory copy test results(without memSuite)
I'm checked it but chunk copy is fast . It's misteirous.
and, I don't know uint32_t timer counter is how many cycled.So I don't know the result is correct or not.


The code is here, and these codes located at  run_test()
    void *m1 = malloc(20*1000*1000);
    void *m2 = malloc(20*1000*1000);
    void *sm1 = shoot_malloc(20*1000*1000);
    void *sm2 = shoot_malloc(20*1000*1000);
    void *ss1 = shoot_malloc_suite(20*1000*1000);
    void *ss2 = shoot_malloc_suite(20*1000*1000);

    uint32_t t1 = tskmon_get_timer_reg();
    uint32_t t2,tmp;
    int i,off;

    console_printf("malloc-ed 20M memcpy\n");
    memcpy(m1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc-ed 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(m1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc-ed 20M dma_memcpy\n");
    dma_memcpy(m1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc-ed 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(m1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M memcpy\n");
    memcpy(sm1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(sm1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M dma_memcpy\n");
    dma_memcpy(sm1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc-ed 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(sm1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc to shoot_malloc-ed 20M memcpy\n");
    memcpy(sm1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc shoot_malloc-ed 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(sm1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M memcpy\n");
    memcpy(m1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            memcpy(m1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            t2 = ~tmp - t1 + t2;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);





    t1 = tskmon_get_timer_reg();
    console_printf("malloc to shoot_malloc-ed 20M dma_memcpy\n");
    dma_memcpy(sm1,m2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("malloc shoot_malloc-ed 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(sm1+off,m2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M dma_memcpy\n");
    dma_memcpy(m1,sm2,20*1000*1000);
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);

    t1 = tskmon_get_timer_reg();
    console_printf("shoot_malloc to malloc 20M dma_memcpy 2m each\n");
    off=0;
    for(i=0;i<20;i++)
        {
            dma_memcpy(m1+off,sm2+off,2*1000*1000);
            off = off + 2*1000*1000;
        }
    t2 = tskmon_get_timer_reg();
    if(t2<t1)
        {
            tmp= 0;
            tmp = ~tmp - t1;
            t2 = t2 + tmp;
        }
    else
        {
            t2 = t1 -t2;
        }
    console_printf("%d\n",t2);




And result is:  (printf has no %u so I used %d(signed))
malloc-ed 20M memcpy
-485806
malloc-ed 20M memcpy 2m each
-71147
malloc-ed 20M dma_memcpy
-272364
malloc-ed 20M dma_memcpy 2m each
-524067
shoot_malloc-ed 20M memcpy
-484381
shoot_malloc-ed 20M memcpy 2m each
-81456
shoot_malloc-ed 20M dma_memcpy
-768859
shoot_malloc-ed 20M dma_memcpy 2m each
-527604
malloc to shoot_malloc-ed 20M memcpy
-561515
malloc shoot_malloc-ed 20M memcpy 2m each
-76210
shoot_malloc to malloc 20M memcpy
-494179
shoot_malloc to malloc 20M memcpy 2m each
-77837
malloc to shoot_malloc-ed 20M dma_memcpy
-266705
malloc shoot_malloc-ed 20M dma_memcpy 2m each
-513931
shoot_malloc to malloc 20M dma_memcpy
-268545
shoot_malloc to malloc 20M dma_memcpy 2m each
-537939
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 07, 2013, 06:50:24 PM
Sorry for the dumb question. I have no problem working with DNGs but I haven't a clue how to convert YUV files. I downloaded 422ToImage but can't get it to work. Are there any tuts?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: AlexVakulov on May 07, 2013, 06:58:12 PM
Quote from: Andy600 on May 07, 2013, 06:50:24 PM
Sorry for the dumb question. I have no problem working with DNGs but I haven't a clue how to convert YUV files. I downloaded 422ToImage but can't get it to work. Are there any tuts?
Hello! Change your file extension from YUV to 422 and use the same program.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 07, 2013, 07:07:49 PM
This is what is happening to 600D sequences:

http://imgur.com/a/fNUme#0
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 07, 2013, 07:12:45 PM
I've noticed the same distortion on 5D3 with raw mode (480p at 12fps). First frames are pretty much OK, then it loses sync.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: mindogas on May 07, 2013, 07:16:33 PM
Quote from: AlexVakulov on May 07, 2013, 06:58:12 PM
Hello! Change your file extension from YUV to 422 and use the same program.
Bad answer. *.YUV extension is for YUV422 videos and *.422 for still pictures. *.YUV files contains some extra information about its frames so  this extra data is like trash for .422 conversion. converting *.YUV files is quite easy but you must know that this is experimental feature. ML and 422ToImage sources changes everyday so you should use the last versions then you experimenting with YUV videos.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 07, 2013, 07:46:52 PM
hmm, interesting.
made a few videos in .422 video mode and got no distortion at all.
this looks like there is some data lost in the beginning of frame number 21.

will look into it tonight.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: deleted.account on May 07, 2013, 08:57:45 PM
Quote from: mindogas on May 07, 2013, 07:16:33 PM
Bad answer. *.YUV extension is for YUV422 videos and *.422 for still pictures. *.YUV files contains some extra information about its frames so  this extra data is like trash for .422 conversion. converting *.YUV files is quite easy but you must know that this is experimental feature. ML and 422ToImage sources changes everyday so you should use the last versions then you experimenting with YUV videos.

Is it really a bad answer? I rename the .422 files to .yuv, import into avisynth, rescale chroma as the source yuv is JFIF chroma over full 8bit range and encode to h264 without creating any intermediate image sequences at all. :-)

We already know images sizes for .422 and can specify them in Avisynth so no big deal.

Its raw 8bit yuv, whether we call it image or video, whether we name it .422 or yuv.

Its down to how the host application handles it?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 07, 2013, 09:47:25 PM
Haven't seen any LV_rec issues on 6D yet... but I can't play back the raw :( (yea i know, bitch bitch bitch ) :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 07, 2013, 10:14:57 PM
Quote from: Andy600 on May 07, 2013, 06:50:24 PM
Sorry for the dumb question. I have no problem working with DNGs but I haven't a clue how to convert YUV files. I downloaded 422ToImage but can't get it to work. Are there any tuts?

Do you have the newest 1.9 version? it handles single movie files *.yuv very well, i tried today and had no problem with converting yuv to jpeg.
ver 1.8 didn't have such a feature
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 07, 2013, 10:31:07 PM
weird. tried now with 1700 frames again and no problem.
maybe its due to a slow card?
i have a lexar 1000x. do you have a slower one?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: mindogas on May 07, 2013, 10:31:46 PM
Quote from: y3llow on May 07, 2013, 08:57:45 PM
Is it really a bad answer? I rename the .422 files to .yuv, import into avisynth, rescale chroma as the source yuv is JFIF chroma over full 8bit range and encode to h264 without creating any intermediate image sequences at all. :-)

We already know images sizes for .422 and can specify them in Avisynth so no big deal.

Its raw 8bit yuv, whether we call it image or video, whether we name it .422 or yuv.

Its down to how the host application handles it?
Your post should start from last line i think. The thing is that question was about 422toimage software and for it the answer is clear - renaming files extension messing things up.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 07, 2013, 10:36:10 PM
6D, no issues SD card
600D, same make/model sd = issues.

theoretically raw on 6D should be really out of sync if its bad on 5d3 with "slower" card.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 07, 2013, 11:07:05 PM
about the EDMAC performance:
(http://upload.g3gg0.de/pub_files/32a85708bcf3d15e0a8078805e2b32cf/VRAM2.BMP)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: deleted.account on May 07, 2013, 11:16:43 PM
Quote from: mindogas on May 07, 2013, 10:31:46 PM
Your post should start from last line i think. The thing is that question was about 422toimage software and for it the answer is clear - renaming files extension messing things up.

Of coarse you're correct as the author of the application. It 'messes things up' due to the way 422toimage handles the source, my apologies.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 07, 2013, 11:57:56 PM
Edmacs from 6D.


http://imgur.com/a/u3v6q#0

http://imgur.com/a/u3v6q#1


EOS-M

http://imgur.com/a/SP2jG
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: shuggyrasklat on May 08, 2013, 05:26:18 AM
why cant we get this for the mkii?!?!?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 08, 2013, 07:50:28 AM
Oh come on. How often did we explain that?
Let us finish the stuff so it is working at least on one model before we port it to the other models.
....
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: arrinkiiii on May 08, 2013, 01:21:47 PM
YES!!!  YOU GO g3gg0 !!!   ;D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 08, 2013, 04:46:19 PM
600D: http://imgur.com/a/KE9RY#0


With global draw off... memcpy64 starts winning. So as long as you don't use it for anything else... I guess memcpy is faster. Even with zebras/overlay peaking, dma_memcpy still has speed to spare while memcpy bogs down to single digits... so I guess dma scales better...
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Greg on May 08, 2013, 08:53:45 PM
500D :
(http://s10.postimg.org/87i3b8ou1/photo.png)

(http://s13.postimg.org/pt8bjju7r/TEST.png)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 08, 2013, 09:18:31 PM
How are these so fast with GD on? Not a lot of indicators?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Greg on May 08, 2013, 09:25:56 PM
Maybe it's a bug they were setting the default ML.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 08, 2013, 09:34:05 PM
Default ML doesn't have a lot of indicators enabled. I guess I'll try turning things on/off and find whats slowing it down.

Found a big hit to everything is the new peaking/display filter.. without dma_memcpy the hit is even bigger... but still not trivial.. hit to EDMAC is fairly huge too. I guess this is all the side effect of really high CPU usage. Sucks.. the blinking dots peaking is a bunch less useful for me. Anamorphic probably does it too... I guess defish doesn't anymore since I saw it was sped up in some commits.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 09, 2013, 01:29:01 AM
Do you think a bare bones "Magic Blondie" build with nothing but LV_rec would get us closer to 24p?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 09, 2013, 02:00:32 AM
Won't make that much of a difference. Still limited by card write speed and shoot_malloc total size. Only possibility is to shrink reserved memory for other canon things.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 09, 2013, 04:34:53 AM
Solved compilation error. Because ,CreateMemorySuite is imported to exmem.h

diff -r 22fd8107540b modules/lv_rec/lv_rec.c
--- a/modules/lv_rec/lv_rec.c   Thu May 09 00:10:56 2013 +0200
+++ b/modules/lv_rec/lv_rec.c   Thu May 09 11:32:32 2013 +0900
@@ -9,7 +9,6 @@

unsigned int exmem_clear(struct memSuite * hSuite, char fill);
unsigned int exmem_save_buffer(struct memSuite * hSuite, char *file);
-struct memSuite *CreateMemorySuite(unsigned int address, unsigned int size, unsigned int flags);

/* file footer data */
typedef struct
@@ -593,7 +592,7 @@
         data.dmaFlags = 0;

         /* create a memory suite that consists of lv_save_raw raw buffer */
-        data.memCopySuite = CreateMemorySuite(shamem_read(RAW_LV_EDMAC), save_data.frameSize, 0);
+        data.memCopySuite = CreateMemorySuite((void *)shamem_read(RAW_LV_EDMAC), save_data.frameSize, 0);
         PackMem_RegisterEDmacCompleteCBRForMemorySuite(data.dmaCopyChannel, &complete_cbr, 0);
         PackMem_RegisterEDmacPopCBRForMemorySuite(data.dmaCopyChannel, &pop_cbr, 0);
     }
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 09, 2013, 02:18:13 PM
Are these numbers real or there's a bug in my benchmark code?!

(http://www.trackshotsphotos.com/photos/i-3Xz9DZ9/0/M/i-3Xz9DZ9-M.jpg)
(http://www.trackshotsphotos.com/photos/i-RjCcR9d/0/M/i-RjCcR9d-M.jpg)

139MB/s for a 2GB file CF+SD or 118 CF only?!

(source: https://bitbucket.org/hudson/magic-lantern/commits/7e2bd05de88e752eca98b3e8e36a46f23db02968 )
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 09, 2013, 04:59:25 PM
Leaps and bounds ahead of SD.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N.Mendes on May 09, 2013, 11:20:30 PM
Quote from: a1ex on May 09, 2013, 02:18:13 PM
Are these numbers real or there's a bug in my benchmark code?!
139MB/s for a 2GB file CF+SD or 118 CF only?!



Wow... F***ck**ing Impressive! So, if my english doesn't suck too much, the 1% last comment says that "Yes, it's real and not a bug"
Really crazy...

Also, someone already asked before but i'v seen no clear answer:

In term of quality, is there a difference between "Yuv422" and the 14bit raw from burst mode?

And why "Yuv422", is it prores 422 10bits? (so not as good as the 14 bit raw from the burst mode right?)

Thanks for what you do fellas, you're really crazy..
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 09, 2013, 11:43:07 PM
6D with new timing:

(http://i.imgur.com/zMhQ02ul.png)


Buffer size is fairly important. 4MB drop for ~2M
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: hirethestache on May 10, 2013, 01:46:42 AM
Quote from: 1% on May 09, 2013, 11:43:07 PM
6D with new timing:

(http://i.imgur.com/zMhQ02ul.png)


Buffer size is fairly important. 4MB drop for ~2M

So what does this mean for us non coders(other than more of a reason to eagerly wait a release!)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 10, 2013, 02:00:13 AM
this means we are working and testing
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: arrinkiiii on May 10, 2013, 02:02:59 AM
...dont stop !!! ...testing for having a good release !!!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: mindogas on May 10, 2013, 06:22:42 AM
 Everyone who is using 422ToImage please update to 1.9.1 version. There was made some corrections regarding .YUV files ordering, decoding then in threads simultaneously, fixed HDR processing for single shoots and some other little fixes.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 10, 2013, 06:53:37 PM
CF LEXAR 1000X  in my 5D3. I am able to record continues 1904x 960 at 24 fps or 1904x1080 x 20 fps for single file 422 YUV with lv_rec.

I am thinking if we were able to record to SD and CF, I would be able to obtain 1904x1080 x 24 fps. I noticed the CF card benchmark is slower in liveview vs in playback mode. If it is possible to speed up the CF card in liveview mode, it might be possible to do 24fps with just the CF card. 
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 10, 2013, 10:02:56 PM
Live view is using other edmac channels + more memory and CPU. Only way to speed things up is to turn off some canon stuff or shrink its memory. You'd get more frames too with bigger shoot malloc.

What is shoot malloc on 5d3 anyways?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 10, 2013, 10:19:11 PM
244MB in photo mode (RAW quality), 168 in LV. Changing pic quality results in even less free RAM (e.g. only 88 with SRAW, 168 with jpeg).

The total RAM seems to be 512MB. Where did it go?!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 10, 2013, 10:25:00 PM
Heh... so you just have ~10 more MB than I do... same story... raw is the best (thankfully) on memory. The other cameras are stuck with sraw and L.

So we have disableaewb.. I also found DisableWBdetection and DisableFaceCatch... maybe there are more like the AEWB.. would maybe free up some CPU and hopefully memory.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 10, 2013, 10:39:09 PM
I would try to stop somehow the edmacs that bring LiveView data (or alter their configuration to output one line only). Not sure how... should work by writing to edmac registers from some vsync spot (similar to redirecting the lv_save_raw buffer in the burst dng code).

What about running the benchmarks with most overlays active? (focus peaking and such). If the numbers are similar, CPU usage doesn't matter; it's just the data bus being overloaded by all those image buffers.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 10, 2013, 11:00:21 PM
I can't say CPU doesn't matter... running those benchmarks with new peaking enabled absolutely kills memcpy. Unless its all from creating another image buffer to actually apply the peaking. Also GD on slows everything down.

seems I gained 1MB after I ran that aewb disable, hasn't gone away yet, have 31 and 104MB now on 600D

600D Edmacs:

(http://i.imgur.com/hYQoduYl.png)

6D Edmacs - Both of these are 720P60

(http://i.imgur.com/5W9VRf4l.png)

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N.Mendes on May 10, 2013, 11:32:06 PM
Quote from: lourenco on May 10, 2013, 06:53:37 PM
CF LEXAR 1000X  in my 5D3. I am able to record continues 1904x 960 at 24 fps or 1904x1080 x 20 fps for single file 422 YUV with lv_rec.

In raw mode?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 10, 2013, 11:36:15 PM
Quote from: N.Mendes on May 10, 2013, 11:32:06 PM
In raw mode?

It was in 422 YUV. Not in Raw mode.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: CFP on May 10, 2013, 11:41:00 PM
Quote from: 1% on May 10, 2013, 11:00:21 PM
I can't say CPU doesn't matter... running those benchmarks with new peaking enabled absolutely kills memcpy.
I can confirm this. I've just tried the card benchmark with Global Draw on (All features enabled) and off like you suggested, a1ex. And with GD I've got about 1/3 of the writing speed that I've got without GD. At the end my 600D actually overheated!  :(

I hope that helps you a little bit, because I'd really like to see how much DNG frames you'll get out of our cameras in the future.

May I ask if you have already found a way to edit the LV_Rec images? I know that things like this need time and I don't want to push you. I am just curious. And I'm still very excited about this feature. Are you more pessimistic or optimistic regarding the development?

I apologize if I'm bothering you. Just don't answer if you don't want to :)

Anyway, thanks a lot for what you've already done. I really appreciate your work! ML is fantastic  :D

Also I would like to help you, but since I'm only familiar with Java, I guess there's no way I can help you apart from testing?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N.Mendes on May 10, 2013, 11:52:39 PM
Quote from: lourenco on May 10, 2013, 11:36:15 PM
It was in 422 YUV. Not in Raw mode.

Thanks, did you try to enable the raw mode (by hitting "don't click me")?

Some guys tested it (CF+SD) and i don't know if they were able to obtain a continuous recording, it would be awesome..
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 11, 2013, 12:07:57 AM
Quote from: 1% on May 10, 2013, 11:00:21 PM
I can't say CPU doesn't matter...

i think he meant EDMAC performance which is decoupled from CPU
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 11, 2013, 01:15:52 AM
I guess it wouldn't. The benchmark for edmac does drop when new peaking is enabled. GD and no GD also makes some difference.

I think its not as much of a problem for the edmac anyway... mainly how many frames can be stored and how fast they can be written. Extra 20MB even goes a ways.... it did on 600D with sraw. I get almost as many frames as on 6D.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: budafilms on May 11, 2013, 07:35:52 AM
A simple question: can the hardware of any CANON DSRL process all that information that we are trying to get?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 11, 2013, 02:50:50 PM
Just added a RAW to DNG converter: https://bitbucket.org/hudson/magic-lantern/commits/5bc3488c7a78

It's just a small command-line tool. Feel free to build a nice GUI on top of it.

For some reason, I couldn't compile it in 64-bit mode, so you may need the 32-bit compatibility libraries. Fixes are welcome ;)

With a little luck, this program may work under Windows too.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 11, 2013, 04:17:15 PM
Quote from: N.Mendes on May 10, 2013, 11:32:06 PM
In raw mode?
I did a quick test in Raw mode. I wanted to see how much higher I could do above 720p. I tried 1928x850 and 1928x902. I think it was 902. 902 the buffer would fill up and video stops after 700 frames or so.  At 850 I am able to get continuous recordings.

The best I am able to obtain in Raw mode right now is continuous 1928x850 at 24fps. I plan to crop the video to 1920x817 to do 2.35:1 wide screen aspect ratio, which is about 1920x817. The video will have black bars on top and bottom to output at 1920x1080.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 11, 2013, 04:23:59 PM
Raw mode accounts for size like yuv mode? Looks like 1 size hardcoded from what I see. Also how to unload modules, right now it just beeps.... I think it was supposed to be something like prop_unregister_slave but I don't see it defined on 5d3.

I get seg fault with the converter for some reason:

raw2dng[15429]: segfault at b7788f08 ip 08048dd0 sp bfda8dfc error 4 in raw2dng[8048000+2000]

segfaults on both PCs
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 11, 2013, 07:04:47 PM
Quote from: a1ex on May 11, 2013, 02:50:50 PM
Just added a RAW to DNG converter: https://bitbucket.org/hudson/magic-lantern/commits/5bc3488c7a78

Cool. I compiled it and works for me. I was able to convert my raw movie file to DNG files.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 11, 2013, 07:14:18 PM
Did you make it for windows? Could you share your compilation with the rest? Thanks
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 11, 2013, 07:30:22 PM
Worked on a raw movie but for 5d3, right?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 11, 2013, 07:55:36 PM
Quote from: 1% on May 11, 2013, 07:30:22 PM
Worked on a raw movie but for 5d3, right?
Yes.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 11, 2013, 07:57:13 PM
Quote from: kgv5 on May 11, 2013, 07:14:18 PM
Did you make it for windows? Could you share your compilation with the rest? Thanks

It is just for Linux for now. This would be a good project for mindogas given he already has the 422 converter for Windows.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 11, 2013, 08:07:33 PM
I keep getting seg faults from the size, no matter what size I set:

Single file from 600D, W/H set to what dng burst reports

http://www.qfpost.com/file/d?g=4ED0ZAaJN

Single file from 6D http://www.qfpost.com/file/d?g=v8T0Ryl5m
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 11, 2013, 08:34:20 PM
Quote from: 1% on May 11, 2013, 08:07:33 PM
I keep getting seg faults from the size, no matter what size I set:

Single file from 600D, W/H set to what dng burst reports

http://www.qfpost.com/file/d?g=4ED0ZAaJN

Single file from 6D http://www.qfpost.com/file/d?g=v8T0Ryl5m

I tried your 2files. Both are ...
$ ./raw2dng I0000000.RAW
Error: This ain't a lv_rec RAW file


So I guess you may upgrade lv_rec.mo first.
Alex added file footer to RAW in current code.
YOur file has no footer I guess.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 11, 2013, 08:38:50 PM
Its a single file, lv_rec is latest one. For some reason footers aren't being written/read for single files.... After getting rid of thumbnail now my problem is size.

Files come out purple with line skips... I think width is wrong somehow.

Here is a big 600D with footer:

http://www.qfpost.com/file/d?g=upRO7f8YH
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 11, 2013, 09:17:13 PM
Quote from: 1% on May 11, 2013, 08:38:50 PM
Its a single file, lv_rec is latest one. For some reason footers aren't being written/read for single files
I also having the same problem just tried individual frames and single file for yuv 422 and both do not have any footer information on the latest lv_rec.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 11, 2013, 09:18:47 PM
This converter is only for movie files, not for single frames.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 12:11:51 AM
How can I find the correct sizes? Shouldn't they match what DNG burst outputs?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 12, 2013, 02:20:01 AM
Raw video test 12 seconds 2.35:1 aspect ratio

vs
standard i-frame
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 12, 2013, 02:31:08 AM
If those 2 videos have the same lens settings... that is a wow in terms of detail!

Is 12 sec the max right now at the current build?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 12, 2013, 02:36:59 AM
Quote from: ted ramasola on May 12, 2013, 02:31:08 AM
If those 2 videos have the same lens settings... that is a wow in terms of detail!

Is 12 sec the max right now at the current build?

At that aspect ratio I can record continuous at 24fps. It helps to have a 5d Mark III with Lexar CF 1000x 32GB card. DNG files take awhile to process on my computer in AE. Yes, the settings were the same. when I was recording i-frame, I used Technicolor CineStyle.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 12, 2013, 02:50:04 AM
Thanks. Wow!

Please clarify when you say continuous, do you mean you can shoot longer than 12 seconds? Have you tested how long you can go?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 12, 2013, 02:52:59 AM
What's the file size of each 1928x850 DNG frame?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 12, 2013, 03:15:16 AM
Quote from: squig on May 12, 2013, 02:52:59 AM
What's the file size of each 1928x850 DNG frame?
It is about 3.08MB. I have since deleted those files. My youtube video has 1928x820 frames, which is closer to the accepted 2.35:1 aspect ratio. The file size is 2.98MB for those.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 12, 2013, 03:25:25 AM
Cool! Have you tried filling up the card and if so does the MK3 start to smoke?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 12, 2013, 03:40:09 AM
Quote from: squig on May 12, 2013, 03:25:25 AM
Cool! Have you tried filling up the card and if so does the MK3 start to smoke?

I did a test to see what would happen if I filled the memory card. It records the first file fine.  It keeps recording until the memory card is full, but I am having strange results after the first file. I just sent a1ex a message about it. 
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 12, 2013, 04:15:42 AM
Big congrats and thx to all the devs, this is huge for digital indie cinema. Beware the Zaibatsu Ninja.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 04:37:35 AM
Figured out how to get pic sizes... 6D its 1920 exactly. Weird, eh? Sped up to 22MB/s too when the size was corrected.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 12, 2013, 04:50:21 AM
So is 1280x720 60p RAW now doable too? Or 48p?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 12, 2013, 06:57:26 AM
Quote from: 1% on May 12, 2013, 04:37:35 AM
Figured out how to get pic sizes... 6D its 1920 exactly. Weird, eh? Sped up to 22MB/s too when the size was corrected.

You mean 1920 instead of 1840 in RAW mode? Sped up to 22 MB/s so what was the speed before correction?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 12, 2013, 07:45:01 AM
Quote from: squig on May 12, 2013, 04:15:42 AM
Big congrats and thx to all the devs, this is huge for digital indie cinema. Beware the Zaibatsu Ninja.

Yup! Seriously well done! Amazing effort the ML wizards!  8)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kabuto1138 on May 12, 2013, 08:56:01 AM
Nice guys!  great job!

This is only working in the 6D, right?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N.Mendes on May 12, 2013, 09:33:46 AM
Ok, Merry Christmas all..


Quote from: lourenco on May 12, 2013, 03:40:09 AM
It keeps recording until the memory card is full, but I am having strange results after the first file.

Did you try to use CF+SD ? In this case, maybe the second file will be as clean as the first?
About the aspect ratio, again, maybe "CF+SD" will allow to shoot 1920x1080 (or more?)

ps: your test is crazy, in term of detail and dynamic range, really really crazy, thanks dude..

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Africashot on May 12, 2013, 09:51:02 AM
Any chance of getting anywhere near 400 frames at 24 fps with the good old 5D2? Sorry for asking, I know you guys are working on it and I don't want to be pushy but man this is so exciting I just couldn't hold myself and if there is only the slightest chance it would be way beyond what I had ever dared to dream of!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 12, 2013, 09:58:32 AM
Run the benchmarks and post screenshots. I don't remember seeing any benchmarks from 1000x cards with 5D2.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Africashot on May 12, 2013, 10:48:57 AM
Quote from: a1ex on May 12, 2013, 09:58:32 AM
Run the benchmarks and post screenshots. I don't remember seeing any benchmarks from 1000x cards with 5D2.
Thanks Alex! Will try when my 1000x card gets here, shipping to Africa takes eternities... :(
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: arrinkiiii on May 12, 2013, 11:03:42 AM
!!! AMAZING !!!

Anybody thinks that this will run on the 7D?

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 12, 2013, 11:25:38 AM
Super impressive stuff. Seriously well done guys!

@1% have you tried this on the 600d yet? I know it probably wont handle what the 6d and Mk3 can but I'm eager to know what, if any chance there is of pushing the shot length up... even if only by a few seconds.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: RenatoPhoto on May 12, 2013, 02:06:46 PM
Congratulations!
You are the Nobel prize winners for arm development period!
Thanks for your amazing work.
Renato
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 12, 2013, 02:20:24 PM
please dont ask about too many models. we are aware that 550d, 50d, 60d, 6d, 600d, 5d2, 7d, etc are also widely used.
before we start analyzing these models, we want prove our theory by making a reference implementation.

as soon this reference implementation is proven stable, we will advance to these models. ;)

current state: trying to make the code as performant, stable and portable as possible.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 12, 2013, 02:56:22 PM
Quote from: a1ex on May 11, 2013, 02:50:50 PM
Just added a RAW to DNG converter: https://bitbucket.org/hudson/magic-lantern/commits/5bc3488c7a78

It's just a small command-line tool. Feel free to build a nice GUI on top of it.

For some reason, I couldn't compile it in 64-bit mode, so you may need the 32-bit compatibility libraries. Fixes are welcome ;)

With a little luck, this program may work under Windows too.


Alex, Are you forget to add this ?(when the file size reached 2Gig , then rotation the file)
I'm tried to concatinate by cat command, but something wrong. (4gig of file not supported??)
And cat need a lot of time and disk spaces. So I think each file must have footer info.

diff -r 5bc3488c7a78 modules/lv_rec/lv_rec.c
--- a/modules/lv_rec/lv_rec.c   Sat May 11 14:35:52 2013 +0300
+++ b/modules/lv_rec/lv_rec.c   Sun May 12 21:51:27 2013 +0900
@@ -649,6 +649,8 @@
                     {
                         yPos++;
                         bmp_printf( FONT(FONT_MED, COLOR_WHITE, COLOR_BLACK), 30, 20 * yPos++, "Creating next file");
+                       save_data.frameCount = 1;
+                       lv_rec_save_footer(save_data.handle, &save_data);
                         FIO_CloseFile(save_data.handle);
                         save_data.handle = NULL;
                         lv_rec_update_suffix(&save_data);

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 12, 2013, 03:16:37 PM
the lv_rec code basically is my work, alex updated it to the latest research results.
so i will answer to your question.

i wanted the split file to be really "split" just as if they were splitted by common split and merge utilities.
this means the payload is split over several files and at the end we have one footer that contains information for all frames.

simply concatenate the files to have a big movie file.

the reason for a footer is simple: in YUV mode you can process the files with the existing tools, you dont have to cut the footer away.
if i had added a header, it wont be that simple - you have to remove the header before you can use old tools.

when the tools are finished, it is as simple as specifying the first file M000000.RAW and it will autodetect that there is a .R00, R01 etc
and will virtually address them as one file.

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 03:20:07 PM
Quote
@1% have you tried this on the 600d yet?

Almost... I got a size of 1652 but the YUV buffer is 1728... when I open the files they are still messed up. Maybe I need to pull a bigger buffer than 2k and run image analysys again... that or sync is fucked?

Have to check on 720P mode, the size might be different. If it is, should pick based on YUV buffer size... like if buffer is 1280 then raw video is XXX, etc.

Quotethe lv_rec code basically is my work,

This in the new memecpy is freaking awesome :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 12, 2013, 03:34:49 PM
Heh, people got excited without even knowing the big news: g3gg0 just discovered how to use the DMA cropping routines, which just made possible RAW video recording at 1920x1080 at 24fps on 1000x cards.

Technical: we now know how to copy a cropped version of some image buffer at very high speeds (over 700MB/s), and with this trick we can save the video data the card at full speed, without being slowed down by image borders, for example.

1920x1080 RAW video now requires 83MB/s at 24fps, so it should work just fine on 1000x cards. I didn't try it.

So, I've lost my patience and rewritten the lv_rec module from scratch, to use these new routines and to experiment with different buffering algorithms. The new module is called raw_rec and outputs the same file format (RAW files).

Main changes:

- The ring buffer only uses 32MB memory blocks (maximum we can get). Reason: card benchmarks showed higher data rates for large buffers.
- Frame copying is done outside the LiveView task (not sure if it has any effect).
- When the buffer gets full, it skips some frames, rather than stopping.
- Fewer hardcoded things: should be easier to port.
- Resolution presets, from 640x320 to 3592x1320.

Just like lv_rec, this is in very early stages, so you have to compile it yourself.

Source code: https://bitbucket.org/hudson/magic-lantern/commits/54537cb85d7d

If you try it, I'd like you to look for any signs of image tearing. The source raw data is single-buffered, but it's possible to make it double-buffered if the vertical sync is less than ideal.

All credits go to g3gg0 - without his reverse engineering work on understanding the image processor (http://magiclantern.wikia.com/wiki/Register_Map#EDMAC), this would have been impossible.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: minimimi on May 12, 2013, 03:41:03 PM
Wrote reply but alex is making a code from scratch,,,,, ignore this.........


Quote from: g3gg0 on May 12, 2013, 03:16:37 PM
i wanted the split file to be really "split" just as if they were splitted by common split and merge utilities.
this means the payload is split over several files and at the end we have one footer that contains information for all frames.

simply concatenate the files to have a big movie file.
I already read and understand your codes.
But , concatenate time is really long.
So I think if the all of files has footer info, we don't need time to concatenate files and tempolary disk spaces.
Also , I'm tried to concat by cat command, I don't know why, but fopen failed (3.6Gig file on 32bit linux).
(?? cat command broken files???)

Quote from: g3gg0 on May 12, 2013, 03:16:37 PM
the reason for a footer is simple: in YUV mode you can process the files with the existing tools, you dont have to cut the footer away.
if i had added a header, it wont be that simple - you have to remove the header before you can use old tools.

when the tools are finished, it is as simple as specifying the first file M000000.RAW and it will autodetect that there is a .R00, R01 etc
and will virtually address them as one file.
If we can cut footer , we have no problem to use old tools when each files has footer info.

It's my thinking.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Stedda on May 12, 2013, 03:49:41 PM
Quote from: a1ex on May 12, 2013, 03:34:49 PM
Heh, people got excited without even knowing the big news: g3gg0 just discovered how to use the DMA cropping routines, which just made possible RAW video recording at 1920x1080 at 24fps on 1000x cards.

True wizards! Nice work guys!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 04:01:28 PM
QuoteSource code: https://bitbucket.org/hudson/magic-lantern/commits/54537cb85d7d

Going to try it... old LV_REC would crash in 720P raw because of fixed size.

Ok, just tested on 600D, thats the repo I have open :)... 32mb is too big to allocate here, tried 16, going to try 24. Wish the skipping was optional, sometimes you want continous frames for less time. Stopping hangs up the ML menu/camera for some reason, LV still moves and I think you can hit canon stuff. Once you switch modes LV won't go away. Have to check this out. Going to see if files are readable with the 1740 preset and try on 6D.


Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 12, 2013, 04:13:00 PM
Don't forget to try "don't click me" - g3gg0's demo is really cool.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 04:17:02 PM
On 600D the demo lagged, on 6D it was perfectly fluid. Image would move back and forth... just there was no way to stop it :)

Oh man,,, the raw file is 640MB... I just recorded a few sec. 296 frames... 15MB/s @16MB malloc.. will try 24MB. Its funny but 600D/6D sd rates are very very close, maybe with the larger writing it will finally crack 30MB/s

1740 its like the files are off by 1 or 2 pixels, slightly skewed to the side. Doing img.py

600D size is 1738.

Ok, more testing: LV_REC will not record the 1738 files, they come out all pink and screwed up, why?
Raw_rec does a good job but ending is screwed up and for some reason the resolution is detected as 1740 and when manually fixed the files say unexpected end of file. Gimp can open them, they look like this:

(http://i.imgur.com/93hZBINl.jpg)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 12, 2013, 04:17:36 PM
On 5D3 it even has rolling shutter :D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 12, 2013, 04:19:32 PM
Quote from: a1ex on May 12, 2013, 03:34:49 PM
1920x1080 RAW video now requires 83MB/s at 24fps, so it should work just fine on 1000x cards. I didn't try it.

It works! This with my Lexar CF 1000X 32GB.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 12, 2013, 04:28:46 PM


I just pulled the trigger on a Toshiba 1066x card, by the time it arrives I imagine the MK3 will be 4k ready :D How's the thermometer looking with continuous recording?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 12, 2013, 04:44:37 PM
Quote from: squig on May 12, 2013, 04:50:21 AM
So is 1280x720 60p RAW now doable too? Or 48p?

Raw mode 1280x672 at 48P is possible. 
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 12, 2013, 04:59:38 PM
Quote from: a1ex on May 12, 2013, 04:17:36 PM
On 5D3 it even has rolling shutter :D

Woah! High end!!  8)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 05:19:16 PM
I've not experienced any overheating on either camera.

Trying 6D now. Shit have to recover my pics before I do it...

600D 1280x698 is perfect, no cuts and no more screwed up images. Records over 50 frames... somewhat like 6D did in the beginning with raw burst.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: -sandro- on May 12, 2013, 05:42:19 PM
You were able to use this on the 600D?

How can I try it? :P

Nevermind: I didn't know there was a build for the 600D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 06:17:37 PM
When it stops properly I'll put up a bin. Right now its annoying, stuff is f'd after recording is finished. When you open the SD slot the light stays lit.

6D will probably get a bin today.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: imperatormher on May 12, 2013, 06:36:09 PM
from where i can download the software or firmware for RAW Video at 24FPS? can you send me link?
my e-mail: [email protected]
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 12, 2013, 07:10:19 PM
thanks alex for making a clean implementation and thanks to 1% and nanomad for porting to other models.
its obvious that our team is very powerful :)

i am sure the fact that ML can provide true 1080 raw video will cause a lot of new users, questions and articles.
we should prepare a news article with some demo video.

so if some users want to provide some cool comparison videos and photos, this would be awesome.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: tferradans on May 12, 2013, 07:15:54 PM
I'm willing to help.

Took a while to learn how to compile this thing, but finally did it. The problem is... where do I find/enable/start RAW recording, in the menus?

(really stupid, I know, but I couldn't find anything anywhere! hahah)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 07:23:07 PM
6D won't do 32MB blocks either. Have to try 16. 600D did 24 or 16, dunno which one is better.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: RenatoPhoto on May 12, 2013, 07:23:29 PM
Would love to try this stuff but I am hung up of compiling with cygwin under windows here: http://www.magiclantern.fm/forum/index.php?topic=5324.25

If it is not possible then should I download the vm machine?  Would that be only other option?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 12, 2013, 07:40:26 PM
Quote from: g3gg0 on May 12, 2013, 07:10:19 PM
thanks alex for making a clean implementation and thanks to 1% and nanomad for porting to other models.
its obvious that our team is very powerful :)

i am sure the fact that ML can provide true 1080 raw video will cause a lot of new users, questions and articles.
we should prepare a news article with some demo video.

so if some users want to provide some cool comparison videos and photos, this would be awesome.

Luke Neumann is on it https://www.facebook.com/NeumannFilms/posts/474872712583502

Yourself, A1ex, Nomad and of course 1% deserve more than just credit! Do you actually realize what you have achieved?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 08:16:22 PM
Well I dunno how much this will do 24fps on SD cameras. Still needs testing for r/w.. I have it alllocating 30MB chunks now. The 600D issues are is from PopEdmac.. I just tried a few on 600D. 6D original was right, others caused 600D issue.

Also... formatting the card for bigger sizes... ie 32mb or 8mb or whatever blocks... I wonder if that is going to help.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: tferradans on May 12, 2013, 09:46:58 PM
Just to be sure I'm following what's going on here: you guys can define a partial crop of the sensor and save it RAW? This could be perfect for anamorphic shooters, just using a square portion in the middle. Possible?

anyway, I'm still trying to compile it, test and post results soon. :P
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 09:54:00 PM
600D issue appears fixed... going into 640P gives me the whole 20MB/s write... 31MB allocation is the best according to card test. Just checking to see if the videos all sized right.

600D is up.. check some of the sizes. the 1080 with is fucked. 1740 = bent, 1738 = half gone.. 1280 is perfect, smaller may work too, check it out. LV_REC not so good in HD for some reason.

600D:

(http://i.imgur.com/5jb2kU9l.png)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 12, 2013, 10:09:15 PM
Quote
you guys can define a partial crop of the sensor and save it RAW? This could be perfect for anamorphic shooters, just using a square portion in the middle. Possible?

Yes, you can select any combination from here:

static int resolution_presets_x[] = {  640,  720,  960,  1280,  1320,  1920,  2048,  2560,  2880,  3592 };
static int resolution_presets_y[] = {  320,  360,  480,  540,  720,  840,  960,  1080,  1152,  1280,  1320 };


or add your own custom resolutions (just put the numbers there).

The high resolutions only work in 5x zoom mode, where you get 1:1 crop from the sensor.

Right now, ML will do a center crop, but I'd like to try to implement a smooth digital panning (sort of a software dolly). Shifting the crop window by 8 pixels at every frame should be easy, and that would result in a 8-second horizontal panning of the 1920x1080 window from sensor center to the edge: (3592-1920) / 8 / 24fps = 8.7 seconds.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 12, 2013, 10:18:10 PM
Quote from: g3gg0 on May 12, 2013, 07:10:19 PM
thanks alex for making a clean implementation and thanks to 1% and nanomad for porting to other models.
its obvious that our team is very powerful :)

i am sure the fact that ML can provide true 1080 raw video will cause a lot of new users, questions and articles.
we should prepare a news article with some demo video.

so if some users want to provide some cool comparison videos and photos, this would be awesome.

Send me some footage dude and I'll get my nerd on in Resolve :) Would love to have a play and if it means helping make something look pretty I'm totally game!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 10:19:38 PM
Wait a little bit and he won't have to send you any footage :)

The 2K width works on 600D in zoom... better than 1080P (which still f'd).
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 12, 2013, 10:30:45 PM
Quote from: 1% on May 12, 2013, 10:19:38 PM
Wait a little bit and he won't have to send you any footage :)

The 2K width works on 600D in zoom... better than 1080P (which still f'd).

Wowzer! Didn't realise you were THAT close dude. Does than mean I might be glued to my 6D for the next few days? Damn... My girlfriend is gonna hate you guys.  8)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: CFP on May 12, 2013, 10:46:09 PM
Damn! You are really awesome! Honestly, you made Magic Lantern by far the best thing that one can download from the internet!

I mean it, THANKS!  (http://forum.chip.de/images/smilies/bw_smilies/gut.gif)

I'm still very excited and even if the EOS 600D won't be able to handle continuous shooting it's still stunning to see what some of these tiny DSLR can do.

(And it's sad that Canon does not implement these cool things like RAW video from the beginning although it would be possible :( )

I also would like to help you a little bit with the comparison videos. I'll see what I can do ...

And I know that's the wrong thread, but do you plan to create a new matrix mode for the silent pictures? So that one can get RAW silent pictures with really high resolutions? That would be fantastic for time lapses.

greetings from Germany :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 12, 2013, 10:52:19 PM
Has anyone managed to compile the raw2dng convertor in windows? I'm VERY rusty. The last time I compiled anything was 20+ years ago
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 12, 2013, 11:00:59 PM
Yes, just uploaded it: https://bitbucket.org/hudson/magic-lantern/downloads/raw2dng.exe

Also got the panning mode working (just a proof of concept, no GUI). Try it in x5 zoom for best results (though you can't see what you are framing).

https://bitbucket.org/hudson/magic-lantern/commits/4eb02c2a4082
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 12, 2013, 11:03:18 PM
Quote from: a1ex on May 12, 2013, 11:00:59 PM
Yes, just uploaded it: https://bitbucket.org/hudson/magic-lantern/downloads/raw2dng.exe

Also got the panning mode working (just a proof of concept, no GUI). Try it in x5 zoom for best results (though you can't see what you are framing).

https://bitbucket.org/hudson/magic-lantern/commits/4eb02c2a4082

A1ex = Legend! Thankyou :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 12, 2013, 11:17:27 PM
5d2 benchmark:

(http://i.imgur.com/FyR12GE.png)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 12, 2013, 11:27:00 PM
So... 1920x840?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: escho on May 12, 2013, 11:41:53 PM
Great work!

I played a litte with this raw-recording on my 600D. I switched to 720 x 540 px for continous shooting and used 5x-crop. The speed is warp2. The results are fine, but under sad light-conditions noisy. Some pics are out of sync. I can open the converted files with ufraw and with showfote (digikam). Trying to open with darktable results in an segmentation-fault in darktable, I don´t know why.

Tomorrow, I will do some further tests. Good night

Edgar
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 12, 2013, 11:50:57 PM
ML benchmark 6D + sandisk extreme 64GB 45mb/s

(http://oi42.tinypic.com/rk0vtw.jpg)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 12, 2013, 11:52:58 PM
For 600D I think 1280 wasn't bad. My sd got corrupted on 6D. develop or boot flag is written somewhere and now it won't format. How can I unset this flag for an unformatted disk? Eoscard doesn't recognize it.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: escho on May 13, 2013, 12:06:27 AM
1280 crashed my camera. Will have a look at it tomorrow.

Edgar
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 12:42:44 AM
@1% - I'm getting pink shots every time. I've tried a lot of different settings. (600d) Also, a few frames have tearing where half the frame is correct colours.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 13, 2013, 12:52:47 AM
There's no audio yet yeah?

So am I correct in assuming that 2048 x 871 should work with a 1000x card on the MK3?

I'm shooting some test scenes for a feature in about 3 weeks so if everything goes smoothly I'll shoot them RAW and post them up
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: jc on May 13, 2013, 01:10:28 AM
These developments are truly amazing.. & assuming it can keep filming for 15 to 30 mins they totally transform the camera .. might even keep people buying the 5d3 over the BMC ! who knows

I am sure that someone has already thought of this, but just in case.. the 5D3 has TWO memory card slots. The ( i presume faster ) CF slot and the SD slot.

Can both be written to near simultaneously?

If we to make the maths easy, say for example the cf slot is 2x the SD slot, we can write 2 images to the CF whilst a 3rd in the buffer is written to the SD card slot. Then a simple merging of the files in the folder on your mac/pc & you have all the frames!

Assuming that 1920x1080 24p is possible ( and maybe even 30p ) just to the CF, am i correct in thinking that 1980x1080 36P to 45P for increased frame rate or maybe 2400x1350 24P might be possible to when written to both cards simultaneously?

Keep up the great work :-)

James C
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Rush on May 13, 2013, 01:14:08 AM
QuoteIf we to make the maths easy, say for example the cf slot is 2x the SD slot, we can write 2 images to the CF whilst a 3rd in the buffer is written to the SD card slot.
SD in 5D3 is about 5x times slower than CF.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: jc on May 13, 2013, 01:28:42 AM
Hi Rush, thanks for your reply... well not the 50% gains that i was hoping for.. but that's still a 20% boost!

If 24P was the total max frame rate at 1080 then that would take you to 29.. so with a tiny bit of tweaking 30P .. i think, if its not too much work, a 20% gain is quite significant :-)

Do you think so?

Also is any form of (lossles) compression possible inside the small cpu on the camera? even if a very basic compression got another say 15% that might also be significant in the long run!

James
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 13, 2013, 01:44:02 AM
Could someone please make another ML benchmark for 6D but with a 95mb/s card? (and post the screenshot)
My sandisk extreme 64GB 45mb/s showed max writing speed 39,3.
1% - you've made a benchmark of 6D before and get max 32 - what card was that?
Maybe we could get somewhere close to that 50 with 95mb/s card?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 13, 2013, 01:50:06 AM
For 5d2, I found all stubs, but the function lv_raw_dump() seems to be inert in the 5d2.. doesn't dump anything!

after I call lv_save_raw() *(*(0x1D78) + 0x3F4) points to the raw buffer's location. Using bmp_hexdump() I can see image data there changing with each frame, but only about 30 bytes of it from where the buffer pointer starts..

I tried dumping 4.2mb from here but all I got was a blank file.

lv_raw_dump() hangs up and never returns, dumps a blank file (0bytes).

damn.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kabuto1138 on May 13, 2013, 02:15:12 AM
I don't think I've ever refreshed a post so many times since the the invention of the internet.


P.S. Alex, can I make "I LOVE ML" shirts?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 02:21:49 AM
Yea, for 600D sometimes there are issues. Test all the rez, use sraw to get the full 100MB buffer. I dunno if overriding the 30P to 24 in 640x480 is helpful or causing problems. Needs more testing for sure but here are some examples:

http://imgur.com/a/CJHN3#0
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 13, 2013, 02:23:22 AM
Wonder how a Lexar Professional 400x SD would work out for us 6D users? Anyone got one?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Rush on May 13, 2013, 02:34:44 AM
Quote from: noisyboy on May 13, 2013, 02:23:22 AM
Wonder how a Lexar Professional 400x SD would work out for us 6D users? Anyone got one?
I wonder too, but according to specs (http://www.bhphotovideo.com/c/product/841483-REG/Lexar_LSD64GCTBNA400_64GB_SDXC_Memory_Card.html), it's max write speed is only 20 mb/s (with 60 mb/s read speed).
Looking forward for 6D + SanDisk 95 mb/s tests which is specced 90 mb/s write speed (http://www.bhphotovideo.com/c/product/824149-REG/SanDisk_SDSDXPA_064G_A75_Extreme_Pro_64GB_SDHC_SDXC.html).
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 13, 2013, 02:37:22 AM
Quote from: Rush on May 13, 2013, 02:34:44 AM
90 mb/s write speed

Cor blimey gavna!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 13, 2013, 02:42:46 AM
Right now 633x is the fastest SD card you buy buy afaik.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 13, 2013, 02:52:43 AM
Quote from: coutts on May 13, 2013, 02:42:46 AM
Right now 633x is the fastest SD card you buy buy afaik.

Cheers for the tip coutts! It's a minefield of BS out there and they all say they have the fastest speeds since sliced bread so good to get a recommendation!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 13, 2013, 02:58:34 AM
That's actually some pretty cool performance but the price! Owie  :o
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 03:50:04 AM
6D is up.. not sure on the best allocation size. 28 is really fast but ending recording freezes. 30 doesn't freeze but its slower. Somewhere in the middle is 31.95 which is what I ended up setting...
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 13, 2013, 03:59:40 AM
Quote from: 1% on May 13, 2013, 03:50:04 AM
6D is up.. not sure on the best allocation size. 28 is really fast but ending recording freezes. 30 doesn't freeze but its slower. Somewhere in the middle is 31.95 which is what I ended up setting...

Dude! Please standby while I freak the shit out (you do mean RAW recording right?) :)

So by freezing I gather you mean when we end recording it freezes the camera? Does pulling the battery fix it if I have a wirl?

I can live with that :D

Seriously can't wait - you are a ledgend! When on earth do you sleep!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 13, 2013, 04:08:49 AM
They're not human
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 04:21:25 AM
I did 1080P video with audio this time. I am going to post it up.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 13, 2013, 04:23:12 AM
I want to check out these new findings so bad on my 5d3, but my lack of knowledge in compiling source code...

Great work team!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 04:33:58 AM
Yea, you can pull battery, turn off camera etc... it shouldn't be freezing with this size. Just hard to pick the "best" one because difference is 20Mb/s or 29MB/s... funny the CF cameras have double speed pretty much.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 13, 2013, 04:36:53 AM
There's a lot more usable dynamic range. When I get raw_rec up and running I'll run Samuel's DR test and we'll see how it compares vs the H.264 profiles.

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: woodybrando on May 13, 2013, 04:45:36 AM
This is amazing! Unfortunately I just sold my 5diii last week cause i pre-ordered the bmcc pocket and switched to m43. Now I may be back in the market for a 5diii before long :)

I was wondering is there any way to run the live view footage through the camera's h.264 encoder? and maybe get more dynamic range and sharper h.264 footage?

Also, I just love that you've unlocked the potential of the 5diii. Canon has become notorious for withholding technology and you guys just broke open the treasure chest they didn't want us to find in their DSLR's!

:)
Jayson
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 04:46:10 AM
Full 1080P test at 24fps.  This time with audio.



vs
I-frame 1080p at 24fps.
Technicolor CineStyle


The camera exposure was the same for both videos.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 13, 2013, 04:50:21 AM
wow... just wow. So raw recording can record now sound? Is it 44khz or 48khz? 
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 13, 2013, 05:04:42 AM
Quote from: lourenco on May 13, 2013, 04:46:10 AM
Full 1080P test at 24fps.  This time with audio.

The camera exposure was the same for both videos.

Holy crap dude! That's amazing! So erm... you DO want to swap me your 5D3 for my 6D right? Okay - cool!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 05:13:39 AM
Probably any CF camera will do well here.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 13, 2013, 05:14:52 AM
I'm bummed the 5d2 doesn't save MM1 files, so raw dumping doesn't work.

it can do 64MB/s or so write speed with a 1000x card, so it can do the 1080p @ 24fps.. just need to make the raws work.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 13, 2013, 05:16:49 AM
Quote from: 1% on May 13, 2013, 05:13:39 AM
Probably any CF camera will do well here.

Sweet! Are there limitations which might prevent my fragile old 7D from getting a new lease of life?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 05:37:08 AM
7D has to be coded... should work tho.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ashtrai on May 13, 2013, 05:44:58 AM
I'm really impressed with all of this progress.  Great job guys.
I don't really expect this to work with my trusty old 550D, although you've all surprised us more than once.
Time to scour the market for a 5DM2.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 06:02:06 AM
Quote from: ted ramasola on May 13, 2013, 04:50:21 AM
wow... just wow. So raw recording can record now sound? Is it 44khz or 48khz?

I added in at post. I used my H4N to record the sound externally.

I am wondering if we could try recording a wav file to the SD card while recording raw video to the CF card. I am wondering what would happen.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 13, 2013, 06:18:52 AM
Anyone care to upload the files(autoexec.bin) for the 5D3 and send me a link?
Would be greatly appreciated
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 06:34:19 AM
Quote from: aaphotog on May 13, 2013, 06:18:52 AM
Anyone care to upload the files(autoexec.bin) for the 5D3 and send me a link?
Would be greatly appreciated


https://docs.google.com/file/d/0B7QlH_BH2m32c0JjTkYtcHQ3Slk/edit?usp=sharing

or

https://docs.google.com/file/d/0B7QlH_BH2m32cWlUNmd6elZsY2c/edit?usp=sharing
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 13, 2013, 06:41:09 AM
Quote from: lourenco on May 13, 2013, 06:34:19 AM

https://docs.google.com/file/d/0B7QlH_BH2m32c0JjTkYtcHQ3Slk/edit?usp=sharing

or

https://docs.google.com/file/d/0B7QlH_BH2m32cWlUNmd6elZsY2c/edit?usp=sharing
Thank you so very much!!!
All I do is replace the autoexec.bin file, with the one currently on my card

PS. This is built today, right? With all of the new findings?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: stevefal on May 13, 2013, 06:42:36 AM
I'm building and running for 5D3, and have raw2dng.exe. Now, how to try the feature? Can someone explain how to set up and shoot, how to stop, and what feedback (if any) to expect?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 13, 2013, 06:44:06 AM
Lourenco, you are wonderful. Now I can't wait to get home!
And to buy 1,000x card

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: budafilms on May 13, 2013, 09:25:51 AM
This will change the idea of Cine. You know what I mean. Congrats everybody!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 13, 2013, 10:25:32 AM
I'm getting continuous 1080p recording with a Lexar 600x CF card. I can actually push it to 1920x1280 and it doesn't appear to be dropping any frames. Is there any kind of alert when it drops frames?

Now all I need is a mac debayer.  :-*
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: arana05 on May 13, 2013, 10:39:37 AM
THIS IS JUSST AMAZING GUYS!
On behalf of all the lurkers out there, thank you congrats.

Is there any chance of somebody sharing a couple of shots with us? I'd like to see how to implement a Cineform RAW workflow (and save 6x space in the HD)

update: Could anybody point me where the donation link is? I've been looking for it, without much success

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 13, 2013, 11:19:57 AM
Quote from: squig on May 13, 2013, 10:25:32 AM
I'm getting continuous 1080p recording with a Lexar 600x CF card. I can actually push it to 1920x1280 and it doesn't appear to be dropping any frames. Is there any kind of alert when it drops frames?

Now all I need is a mac debayer.  :-*

uhm, if you have the latest repository version, you will see the number of dropped frames and the buffer line is highlighted in red.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 13, 2013, 11:21:37 AM
Quote from: arana05 on May 13, 2013, 10:39:37 AM
update: Could anybody point me where the donation link is? I've been looking for it, without much success

at the moment it is disabled. we are thinking about re-enabling it, but this is just a discussion yet.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 13, 2013, 11:27:02 AM
Looks like there's some crazy experiments going on here! These developments aren't trickling to the the nightly builds yet are they? I'll wait (im)patiently to see what my camera lens can show with your software. :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 13, 2013, 11:29:38 AM
Release the Kraken!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 13, 2013, 11:30:05 AM
Quote from: coutts on May 13, 2013, 05:14:52 AM
I'm bummed the 5d2 doesn't save MM1 files, so raw dumping doesn't work.

it can do 64MB/s or so write speed with a 1000x card, so it can do the 1080p @ 24fps.. just need to make the raws work.

we still can set up a EDMAC to read from connection 0 which should be the CRAW source.
if we dont use fragmented memory as buffer, it could work.
in lv_rec i used fragmented memory and at fragment switching, some data seemed to get lost.

this might work on any other model, that doesnt have lv_save_raw.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: mucher on May 13, 2013, 11:31:27 AM
if the raw video is enable, I calculate that it will be producing 500m/s plus bitrate, which is bigger than most 4:4:4 cinema cameras I suppose, and it will be hugely demanding for the CF card and HD and post-processing. For the moment,  this raw output feature is veryvery nice, but I still wish to see than it can be compressed someway, or that 4:2:2 video file output will be better for me.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 13, 2013, 11:41:32 AM
Quote from: lourenco on May 13, 2013, 04:46:10 AM
Full 1080P test at 24fps.  This time with audio.

errrh, why has the driver a helmet on? :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 11:46:10 AM
Ok, this is wild speculation but...

Outputting RAW data via HDMI will likely be impossible due to HDMI limitations and SD is slower compared to CF but is there a ceiling on the level of data throughput of SD (or it's controller) that would prevent higher data rates being sent to the SD card slot - through an adapter to an external recording device (i.e. an SSD) to bypass the card's own speed restrictions? I know the SSD would likely need it's own controller but we're talking pure data dumps and not debayered/cropped/formatted information.

guys?  ::)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 13, 2013, 11:46:25 AM
Quote from: g3gg0 on May 13, 2013, 11:41:32 AM
errrh, why has the driver a helmet on? :)

falling coconuts outside the garage.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 13, 2013, 11:51:40 AM
Quote from: coutts on May 13, 2013, 01:50:06 AM
For 5d2, I found all stubs [...]

Did some small adjustments and seems to work very well on 5D2.

Theoretical figures for the 1000x card, at 24fps, based on Coutts' benchmark (http://i.imgur.com/FyR12GE.png) (64MB/s):

- 1880x840 in full-frame mode (with line skipping)
- 1920x840 or 2152x720 in 1:1 crop mode (zero moire)

With my 266x card, I get 21-22MB/s, so... 720x720 :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Stedda on May 13, 2013, 12:05:07 PM
Quote from: a1ex on May 13, 2013, 11:51:40 AM

With my 266x card, I get 21-22MB/s, so... 720x720 :)

Bring back the donation link so we can get all you guys new UDMA7 1066x Cards! :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 13, 2013, 12:06:02 PM
Quote from: Andy600 on May 13, 2013, 11:46:10 AM
Ok, this is wild speculation but...

Outputting RAW data via HDMI will likely be impossible due to HDMI limitations and SD is slower compared to CF but is there a ceiling on the level of data throughput of SD (or it's controller) that would prevent higher data rates being sent to the SD card slot - through an adapter to an external recording device (i.e. an SSD) to bypass the card's own speed restrictions? I know the SSD would likely need it's own controller but we're talking pure data dumps and not debayered/cropped/formatted information.

guys?  ::)

until now we dont understand the SD/CF hardware good enough.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 12:07:34 PM
@1% I've tried just about every combination/setting and still everything is pink using the new raw module (600d). The green channel looks very weak on a histogram. DNG silentpic bursts are fine.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: rudi on May 13, 2013, 12:26:27 PM
BIG BIG Kudos for you all!!
Just two cents from me for alle developers of ML:

My C-coding  times are about 6 years outdated, but when the sensor only has 12 Stops and and the Raw Data Rate is 14 Bit, maybe you could introduce something like a minimal Quick Compression by shifting away the 2 least significant bits. Maybe the processing power is enough for this litte bit shifting trick during copying the data from buffer to CF. This would lower the datarate with minimal loss.
On the PC/Mac/Linux side you just shift the lost bits back (with random bits) when converting to CinemaDNG or whatever.

Maybe my idea is completely off, then forget it.
Otherwise this could help to bring the datarates down a little without great visual loss...

Greetings

Rudi from slashCAM









Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 13, 2013, 12:37:03 PM
Yeah, it might work at 1 fps or so :D

The secret ingredient here is that CPU usage is almost zero; the hard work is offloaded to the DMA controllers (EDMAC). We can do only fast copying and rectangle cropping.

Look at memory benchmarks: optimized memcpy 73MB/s, 8-bit memcpy 20MB/s, EDMAC 700MB/s.

(http://upload.g3gg0.de/pub_files/32a85708bcf3d15e0a8078805e2b32cf/VRAM2.BMP)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: arrinkiiii on May 13, 2013, 12:40:46 PM
!!! CONGRATULATIONS !!! THIS IS AMAZING !!! AND HUGE GAME TURN !!!

...and we, the humbles mortal geeks, with a 7D (that got 2 processors), no code for we try it ?




;D



Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 13, 2013, 12:42:52 PM
Quote from: g3gg0 on May 13, 2013, 11:41:32 AM
errrh, why has the driver a helmet on? :)

Offers some protection in case his MK3 explodes and adds to the cinematic look.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 13, 2013, 12:44:11 PM
Quote from: arrinkiiii on May 13, 2013, 12:40:46 PM
...and we, the humbles mortal geeks, with a 7D (that got 2 processors), no code for we try it ?

already said: we are working on making it stable and testing what is possible.
it is not time yet to widely announce it. so there is nothing for 7D yet.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: mcnys on May 13, 2013, 12:45:49 PM
so how many frames 600d users can record now?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kuky on May 13, 2013, 12:47:29 PM
Hi,

I'm new to this stuff, never used ML firmware before. But I have an MK3 and want to test/help on this.

I  have the 1.1.3 firmware on the mk3 and i've downloaded the mk3 zip files posted by lourenco and extracted them to an empty lexar cf card.
Then I followed the installation wiki documentation (although I see it was not written for the mk3).
When initiating the firmware update process I see a message superimposed on the menu saying something like "enable bootflag :)" and nothing happens. On restart I have the original 1.1.3 firmware.

I know it's probable trivial but can somebody help?

Thank you,
Cristian
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Steven on May 13, 2013, 12:54:26 PM
Just erase the bootflag fir from the card and make the card bootable with eoscard.

That's how far I got... ML is running, but I can't find the raw features in the menues!?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: arrinkiiii on May 13, 2013, 12:56:02 PM
Quote from: g3gg0 on May 13, 2013, 12:44:11 PM
already said: we are working on making it stable and testing what is possible.
it is not time yet to widely announce it. so there is nothing for 7D yet.

I know (we, 7D users know it) this g3gg0  ;D Is just for saying that we are were, we have trust on the team :D for sure that all the people are truly amazing with this 14 bits raw scene and want to trie out this  :D

Mamamiaaaaaa  ;D

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 01:06:41 PM
Quote from: mcnys on May 13, 2013, 12:45:49 PM
so how many frames 600d users can record now?

Not as many as on a 5d MkIII ;)

I'm still waiting for my fast SD card to arrive. On a class10 SD it only manages continuous recording at the smallest frame settings but even then it drops/corrupts some frames and everything is tinted pink atm on my 600d. At bigger frame sizes you get only slightly more frames than Silentpic DNG bursts. The guys are working hard and it's very early days, especially for the 600d. Give them time :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 13, 2013, 01:09:27 PM
Quote from: Steven on May 13, 2013, 12:54:26 PM
Just erase the bootflag fir from the card and make the card bootable with eoscard.

That's how far I got... ML is running, but I can't find the raw features in the menues!?

in ML menu go to M tab, press "load modules now"
now go to movie tab and go all the way down - there is RAW video option.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Stedda on May 13, 2013, 01:11:31 PM
Quote from: Kuky on May 13, 2013, 12:47:29 PM
Hi,

I'm new to this stuff, never used ML firmware before. But I have an MK3 and want to test/help on this.

I  have the 1.1.3 firmware on the mk3 and i've downloaded the mk3 zip files posted by lourenco and extracted them to an empty lexar cf card.
Then I followed the installation wiki documentation (although I see it was not written for the mk3).
When initiating the firmware update process I see a message superimposed on the menu saying something like "enable bootflag :)" and nothing happens. On restart I have the original 1.1.3 firmware.

I know it's probable trivial but can somebody help?

Thank you,
Cristian

The steps are posted on the 5D III thread half a dozen times...
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Africashot on May 13, 2013, 01:23:26 PM
I have just started a new thread under Hardware and Accessories for anyone to share their experience using different brands and sizes of 1000x CF cards for RAW recording, it would be nice if anyone who has one and tested it could post.
http://www.magiclantern.fm/forum/index.php?topic=5391
Thanks in advance!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: mcnys on May 13, 2013, 01:24:10 PM
Quote from: Andy600 on May 13, 2013, 01:06:41 PM
Not as many as on a 5d MkIII ;)

I'm still waiting for my fast SD card to arrive. On a class10 SD it only manages continuous recording at the smallest frame settings but even then it drops/corrupts some frames and everything is tinted pink atm on my 600d. At bigger frame sizes you get only slightly more frames than Silentpic DNG bursts can do but the guys are working hard and it's very early days, especially for the 600d. Give them time :)

thanks for your answer! will have to wait then :) i just really hope that it will be possible for 600d :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 13, 2013, 01:27:04 PM
Quote from: a1ex on May 13, 2013, 11:51:40 AM
Did some small adjustments and seems to work very well on 5D2.

Theoretical figures for the 1000x card, at 24fps, based on Coutts' benchmark (http://i.imgur.com/FyR12GE.png) (64MB/s):

- 1880x840 in full-frame mode (with line skipping)
- 1920x840 or 2152x720 in 1:1 crop mode (zero moire)

With my 266x card, I get 21-22MB/s, so... 720x720 :)

Is it safe to assume that the 5D2 has some kind of bottleneck before reaching 1000x speeds, considering that other cameras seem to allow much higher rates at that speed? I have a 600x sandisk card and the benchmark topped out somewhere around 50 MB/s for writing. I'll probably edit this post with the benchmark.

edit:(http://i.imgur.com/0wgDlby.png)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 13, 2013, 01:31:22 PM
It doesn't seem safe to assume anything - the benchmarks will tell you the truth. Run them in 24p movie mode, with global draw off, otherwise the numbers may not be relevant.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 13, 2013, 01:32:13 PM
Quote from: lourenco on May 13, 2013, 04:46:10 AM
Full 1080P test at 24fps.

can you post the exact make/name/model of your CF card in the other thread?
perhaps with a photo of the card?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: itsskin on May 13, 2013, 01:33:56 PM
Thx lourenco for posting compiled firmware, I was able to do quick test today in 1920x1080 with Lexar x1000.

In the middle of testing my battery died while NOT recording. This damaged the file system and I lost half of the files on the card. It just started displaying "no space left".

Otherwise it's amazing.

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Steven on May 13, 2013, 01:41:31 PM
I was now able to record 1920x720 to my 32GB Sandisk 60mb/s card until the file size reached 4GB then the recording stopped.
~1min +08sec @25fps
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: mageye on May 13, 2013, 01:59:30 PM
It's very good too see example videos appearing on here. What I am thinking is that it would be nice to see the original compiled videos (as in before YouTube without the YouTube compression ;)).

Maybe someone (or people) could use a file hosting place like mega (https://mega.co.nz/)  to host the files and link them here?

I would especially like to see what 5DMKII as capable of here.

I congratulate the good work and am willing to test on 600x (Duracell) CF card on the 5DMKII if someone could provide me with a build(?)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 13, 2013, 02:07:09 PM
For a 6D with RAW_rec module enabled maximum width res is 1840. I have noticed, than when i press magnification button instead of 5x and 10x it is 1x, 5x, 10x.
1x doesn't change the picture in LV but somehow changes max resolution to 1740. It is good because 1740x720 gives exactly 2,35:1 aspect ratio and to match with 1920x817 it needs 111% scaling. Unfortunatelly all frames are totally corrupted every time. Could you guys take a look at this?

I have also another problem -  i have to reboot every time because raw_rec just don't want to work more than once.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 02:07:13 PM
Quote from: g3gg0 on May 13, 2013, 01:32:13 PM
can you post the exact make/name/model of your CF card in the other thread?
perhaps with a photo of the card?
I think it is the same card as yours? Lexar CF 1000X 32GB.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 02:08:32 PM
This is with more highlight and shadow recovery.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 13, 2013, 02:10:45 PM
So, HDR video is obsolete now? :P
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 13, 2013, 02:11:20 PM
I'm wondering why there is one or two exposure changes when the car pulls out? This was not done full manual? Or there is a setting on auto?

edit- I see the background trees sharpen a bit when exposure changes so I assume IRIS was on auto?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 13, 2013, 02:14:44 PM
Quote from: a1ex on May 13, 2013, 02:10:45 PM
So, HDR video is obsolete now? :P

no! so we can make UHDR videos where the sun spots and the dark side of the moon are perfectly exposed in one image... ;)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 02:17:25 PM
QuoteI'm wondering why there is one or two exposure changes when the car pulls out? This was not done full manual? Or there is a setting on auto?

I am editing the dng files in lightroom.  I am manually changing the exposure in post. The camera exposure setting F stop, shuttor speed, and ISO never changed.

Quoteedit- I see the background trees sharpen a bit when exposure changes so I assume IRIS was on auto?
The focus distance on the lense was changing when the Mustang was reversing. Given one person operating the camera and other person driving the Mustang.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 13, 2013, 02:22:08 PM
Thank you for clarifying that.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: exflyer on May 13, 2013, 02:37:08 PM
Guys, I can not sleep now
I'm your fan  :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ch_d on May 13, 2013, 02:40:44 PM
just waiting here for the 5DMII files to test the new features. awesome work guys.
1000x cards are ordered.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: HugoFilipe on May 13, 2013, 02:45:21 PM
Is there any chance of getting this compiled on an autoexec file uploaded to here? (for 5DIII)
I don't really know how to compile myself, you'd really appreciate if someone could post it here.

Thank you in advance
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kuky on May 13, 2013, 02:48:38 PM
Hi,

Thank to the replies from previous posters I managed to make the lourenco posted firmware to load.
I loaded the raw module.
When I got to the movie raw screen it didn't let me set width/height and there was a message about "write speed needed".
(https://lh4.googleusercontent.com/-bdkkDtFxiaE/UZDfLYKSK3I/AAAAAAAAAfI/CwJVFGBjRJQ/s800/raw_video_screen.jpg)

So I went into the write benchmark but I have the following problem: the benhmark doesn't seem to end, it stays forever on this screen:

(https://lh4.googleusercontent.com/-SXiPMPcXL_Y/UZDfLBLT_JI/AAAAAAAAAfE/KRG8a-6eAbA/s800/benchmark_endless_loop.jpg)

If I press some keys it will revert back to ml menus, but it doesn't seem to retain the becnhamrk speeds.

Thank you,
Cristian
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bart on May 13, 2013, 02:50:28 PM
Wow amazing work guys! Well done!

In my opinion the video quality was the key short coming of the Canon DSLRs and now that limitation is totally gone. Magic Lantern is an amazing product, but in the end it's the video result that counts. I use a GH2 at times for its nice video quality but I miss so many ML features. Raw video puts 5D3 back on the map and a model to be desired. This would indeed make HDR video obsolete now and opens up so much more possibilities. And now the L and Cinema glass will finally show their full potential.

Greetings and good luck to you all
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Rush on May 13, 2013, 02:52:09 PM
QuoteWhen I got to the movie raw screen it didn't let me set width/height and there was a message about "write speed needed".
You should be in LiveView Video mode.

I'd tested 1%'s build for 600D and seems like camera's maximum is 1320х360 or 960х480 in 24p continuous without frame skipping.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Okello on May 13, 2013, 03:03:25 PM
And the inevitable has happened.

One Love to the ML Team!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 03:05:33 PM
Quote from: Rush on May 13, 2013, 02:52:09 PM
You should be in LiveView Video mode.

I'd tested 1%'s build for 600D and seems like camera's maximum is 1320х360 or 960х480 in 24p continuous without frame skipping.

What card did you use on the 600d?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kuky on May 13, 2013, 03:06:04 PM
Mkay, so now I can make it begin recording but didn't figure out how to make it stop. Anyway, everytime I record after a awhile I end up with an Err 70 and camera it's blocked. I have to take out the battery to make it work again. A little bit nerve wracking.

Anyway I observed with lourenco firmware that when loading it says something about "scripts dir missing". Maybe it has to do with benchmark never finishing?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: saad on May 13, 2013, 03:09:43 PM
Here is my comparison test of "image detail quality" on the 5D Mk III with and without ML then against the GH3:



Please notice that this was a fast test and if you find something wrong or unfair do let me know to fix it asap. :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 03:11:31 PM
I will share a newer compiled version soon for the 5D3.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: cschmeer on May 13, 2013, 03:21:01 PM
When I saw that there was a "digital dolly" feature that pans across the image as you're recording, it made me think – the 5D Mark III has a built-in accelerometer, which is used to display whether the camera is "level". However, if one were to somehow use that accelerometer and programme the movement of the recording window to compensate camera movement, could this lead to an in-camera stabilising feature? I know this seems crazy - just a thought :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 03:31:14 PM
Yep tried the pan.. its alright. Would be cool for high res time lapse and soft dolly. I was thinking about it the night before and then the next morning it was in the repo. Pretty neat.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 13, 2013, 03:41:07 PM
The Raw2DNG converter is only available for windows(I'm currently running a mac)?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 13, 2013, 03:42:00 PM
It also works in wine.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: silvertonesx24 on May 13, 2013, 03:53:33 PM
Simply amazing work everyone.

Mark II here, super excited to test this out. Ordered 1000x cards now.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 13, 2013, 03:56:42 PM
Quote from: a1ex on May 13, 2013, 02:10:45 PM
So, HDR video is obsolete now? :P

Haven't you heard?  it's a new feature in the next Canon firmware update coming in April 2014.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: HD Cam Team on May 13, 2013, 03:57:36 PM
Quote from: cschmeer on May 13, 2013, 03:21:01 PM
When I saw that there was a "digital dolly" feature that pans across the image as you're recording, it made me think – the 5D Mark III has a built-in accelerometer, which is used to display whether the camera is "level". However, if one were to somehow use that accelerometer and programme the movement of the recording window to compensate camera movement, could this lead to an in-camera stabilising feature? I know this seems crazy - just a thought :)

If ML team can access to the accelerometer functions, that would be very cool indeed!

Step by step, the 1st thing is to get this amazing development working fine and stable. If so, there are so many great new possible features...

Big congrats to ML Team!!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 13, 2013, 04:00:24 PM
Quote from: lourenco on May 13, 2013, 02:17:25 PM
The focus distance on the lense was changing when the Mustang was reversing. Given one person operating the camera and other person driving the Mustang.

Ahh c'mon man, a true professional does this kinda thing solo.  :D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Lcrusher on May 13, 2013, 04:14:12 PM
Quote from: saad on May 13, 2013, 03:09:43 PM
Here is my comparison test of "image detail quality" on the 5D Mk III with and without ML then against the GH3:



Please notice that this was a fast test and if you find something wrong or unfair do let me know to fix it asap. :)

Was it dropping frames in RAW mode?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 13, 2013, 04:16:22 PM
Quote from: lourenco on May 13, 2013, 03:11:31 PM
I will share a newer compiled version soon for the 5D3.

Just give us 24 hours to test it first so you don't brick your camera.  ;D

I scared the crap out of myself earlier, I tried to boot up my MK3 after it had been switched off for a while and it was totally dead. I think maybe it didn't power down when I switched it off and drained the battery. It might be prudent to pull the battery every time you shut down until the devs iron out the bugs.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 04:20:17 PM
Means the memory wasn't freed when camera turned off.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 13, 2013, 04:29:00 PM
I just downloaded the compiled version (thank you, lourenco), but I don't see this raw shooting menu. Do I need to turn something on to be able to get to this option?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 04:29:48 PM
Quote from: 1% on May 13, 2013, 04:20:17 PM
Means the memory wasn't freed when camera turned off.

This should be the latest one from last night that reports frame drops. It should work better too. let me know.
  https://docs.google.com/file/d/0B7QlH_BH2m32dWpOOEpkWktXMXM/edit?usp=sharing (https://docs.google.com/file/d/0B7QlH_BH2m32dWpOOEpkWktXMXM/edit?usp=sharing)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 04:31:19 PM
Quote from: bumkicho on May 13, 2013, 04:29:00 PM
I just downloaded the compiled version (thank you, lourenco), but I don't see this raw shooting menu. Do I need to turn something on to be able to get to this option?

Please download the new version I just posted. You need to load the module under menu option M for Module. You will see the new raw movie shooting option on the ML movie menu.

I will have to compile a newer version later tonight when I have time. I notice some new changes have been mades since I last compiled. Maybe someone else could post a link until that time?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 04:36:29 PM
Some fixes 4 hours ago too. Now it allocates 30MB block only... more stable... but not as fast.

Either way I can do 960x720 continuous... with raw that will probably scale to 720P or even 1080P.. I bet it will look as good as H264.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 13, 2013, 04:46:58 PM
Quote from: lourenco on May 13, 2013, 04:31:19 PM
Please download the new version I just posted. You need to load the module under menu option M for Module. You will see the new raw movie shooting option on the ML movie menu.

I will have to compile a newer version later tonight when I have time. I notice some new changes have been mades since I last compiled. Maybe someone else could post a link until that time?

Thank you for your help! To shoot in raw, can I start with start/stop button or should I click start from ML menu?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Jason Montalvo on May 13, 2013, 04:49:01 PM
Great Stuff guys !! Patiently waiting for a more stable release of raw video.

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kuky on May 13, 2013, 04:50:56 PM
Quote from: lourenco on May 13, 2013, 04:29:48 PM
This should be the latest one from last night that reports frame skipping. It should work better too. let me know.
  https://docs.google.com/file/d/0B7QlH_BH2m32dWpOOEpkWktXMXM/edit?usp=sharing (https://docs.google.com/file/d/0B7QlH_BH2m32dWpOOEpkWktXMXM/edit?usp=sharing)

Just to be sure:

raw2dng, 5D3-113-bootflag.fir, UserGuide.pdf don't need to be on the CF card. Is this correct?
5D3-113-bootflag.fir has to be used only once to set the bootflag then it can be deleted?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: saad on May 13, 2013, 04:53:11 PM
Quote from: Lcrusher on May 13, 2013, 04:14:12 PM
Was it dropping frames in RAW mode?

There was frame skipping indeed. A fix is already out. :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 04:59:25 PM
Is anyone else trying this on the 600d? Are you getting pink frames (that can't be fixed in post)?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 05:01:23 PM
Sometimes I get pink frames or pink crap in the middle.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 05:06:00 PM
I'm getting it every frame with whatever settings I change  ??? BTW @1% does the way the card is formatted affect anything? I just chose the defaults (exfat) then ran it through EOScard
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kuky on May 13, 2013, 05:08:06 PM
Tested with the lourenco firmware posted on reply 385 on a Lexar 600x CF Professional 16gb & 5D Mk3

It drops frames on both 1920x1080 and 1920x720. Significantly more on the higher resolution but don't know exactly how much because there is no log saved. But there  were many frames dropped.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 05:08:43 PM
Try new build. Card too slow maybe? It only happens sometimes... I posted examples. Also try different modes like 640 (shrinks yuv buffer on this camera)

What card? Size of the blocks is 128-512k, but other than that format should be normal. Default should work.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 05:09:42 PM
Quote from: 1% on May 13, 2013, 05:08:43 PM
Try new build. Card too slow maybe? It only happens sometimes... I posted examples. Also try different modes like 640 (shrinks yuv buffer on this camera)

Just got it. Will do :) Thanks
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 13, 2013, 05:11:31 PM
First test with 5D2:

1880x840 24fps, stretched to 1920x840 with Lexar 1000x 32gb UDMA 7. Processed with Adobe Lightroom 5. It starts dropping frames after about 400 or so (you can see at the end it gets buggy). Alex thinks implementing double buffering should fix the slight tearing so 5D2 looks promising! 720p will for sure work (hardly any buffer usage).

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 05:12:50 PM
Quote from: bumkicho on May 13, 2013, 04:46:58 PM
Thank you for your help! To shoot in raw, can I start with start/stop button or should I click start from ML menu?

Click start on the ML menu.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 05:13:01 PM
Should raw2dng be making a folder when it dumps a video file... they go all over the place.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kuky on May 13, 2013, 05:13:28 PM
Quote from: 1% on May 13, 2013, 05:08:43 PM
Try new build. Card too slow maybe? It only happens sometimes... I posted examples. Also try different modes like 640 (shrinks yuv buffer on this camera)

Hi,

I tried only on lourenco posted firmware on post 385. I don't know if it's the very latest but I don't know how to compile the very last build.

Last time I compiled something it was like 20 years ago :)

It could be cool to save a log with how many frames are dropped.
Also I didn't find a way to stop the recording. For now it seems to just finish the 4gb file. If I press any buttons while recording I get Err 70 and I get very nervous  :o
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 05:17:39 PM
Quote from: Kuky on May 13, 2013, 04:50:56 PM
Just to be sure:

raw2dng, 5D3-113-bootflag.fir, UserGuide.pdf don't need to be on the CF card. Is this correct?
5D3-113-bootflag.fir has to be used only once to set the bootflag then it can be deleted?
correct.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 05:20:49 PM
@1% Pink frames issue appears to be fixed :) Gonna try different settings and see how it scales. I have a new Sandisk Extreme SD card on it's way to me but for now I'm using a Patriot UHS-1. I think it's under performing TBH
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: HugoFilipe on May 13, 2013, 05:22:44 PM
Is there a chance i get some help converting this .raw to .dng?
I downloaded the raw2dng, but when opening it opens and imediatly closes. I supose that there is some specific command to be introduced in the cmd.exe in order to make it process the image.

I'm running windows 8 and tried some compability modes, no luck what so ever.

Thank you
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Audionut on May 13, 2013, 05:28:57 PM
Quote from: HugoFilipe on May 13, 2013, 05:22:44 PM
Is there a chance i get some help converting this .raw to .dng?

You have to open a command prompt, navigate to the location of raw2dng.exe and run it there.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 13, 2013, 05:29:51 PM
Quote from: HugoFilipe on May 13, 2013, 05:22:44 PM
Is there a chance i get some help converting this .raw to .dng?
I downloaded the raw2dng, but when opening it opens and imediatly closes. I supose that there is some specific command to be introduced in the cmd.exe in order to make it process the image.

I'm running windows 8 and tried some compability modes, no luck what so ever.

Thank you

I put raw2dng and raw file in a folder together. from cmd, go to that folder and type raw2dng mxxxxxxx.raw
you'll see dng files created in that folder.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 13, 2013, 05:30:20 PM
Or you can just drag and drop *.raw file onto raw2dng.exe file, raw will extract dng frames to the folder where raw2dng is
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 13, 2013, 05:36:20 PM
lourenco, how did you record sound with this? Did you start raw shooting and then start movie shooting together? Then later sync the sound?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Chucho on May 13, 2013, 05:40:43 PM
This is very beautiful and exciting. I only wish I had a camera to try this out :( Congratulations to everybody involved :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 05:41:07 PM
Yea, I dunno what happened with patriot, only doing 32Mb/s and the boot sector on one of my cards failed.

We could record wav with the video but there is not enough write speed already....

Uhm... don't you have 600D?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: HugoFilipe on May 13, 2013, 05:45:11 PM
Quote from: kgv5 on May 13, 2013, 05:30:20 PM
Or you can just drag and drop *.raw file onto raw2dng.exe file, raw will extract dng frames to the folder where raw2dng is

Incredible how simple it is!
Thank you so much! Now next step is to process all this dng files. Any good workflow advice? I'm importing them to ae but i'm required to edit every single frame, any way of doing this as a batch?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 05:47:19 PM
Lol, didn't even know you could drag and drop onto it.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: sicetime on May 13, 2013, 05:50:07 PM
THIS IS TRULY REMARKABLE, I KNOW YOU HEAR THIS ALL THE TIME, BUT INCREDIBLE WORK!

Does anyone have any experience with komputerbay cards? They have a few 1000x cards that are a little cheaper than the other cards.

http://www.amazon.com/KOMPUTERBAY-Professional-COMPACT-FLASH-Extreme/dp/B009JCL55Y/ref=sr_1_6?ie=UTF8&qid=1368458979&sr=8-6&keywords=1000x+cf+card

I remember reading somewhere that 64gb files are better for speed as apposed the 128gb cards? Am I making this up? What cards do you recommend getting?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kslau8 on May 13, 2013, 05:59:41 PM
There's a thread currently that states those Komputerbay 1000X cards will work.

Quote from: sicetime on May 13, 2013, 05:50:07 PM
THIS IS TRULY REMARKABLE, I KNOW YOU HEAR THIS ALL THE TIME, BUT INCREDIBLE WORK!

Does anyone have any experience with komputerbay cards? They have a few 1000x cards that are a little cheaper than the other cards.

http://www.amazon.com/KOMPUTERBAY-Professional-COMPACT-FLASH-Extreme/dp/B009JCL55Y/ref=sr_1_6?ie=UTF8&qid=1368458979&sr=8-6&keywords=1000x+cf+card

I remember reading somewhere that 64gb files are better for speed as apposed the 128gb cards? Am I making this up? What cards do you recommend getting?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 13, 2013, 06:00:22 PM
Quote from: HugoFilipe on May 13, 2013, 05:45:11 PM
Incredible how simple it is!
Thank you so much! Now next step is to process all this dng files. Any good workflow advice? I'm importing them to ae but i'm required to edit every single frame, any way of doing this as a batch?

Of course there is a batch, in AE you have to press file>import>file, than select first DNG, in the bottom you have to select import as footage and sequence. You make just the first file and the rest are going with the same settings
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: escho on May 13, 2013, 06:03:03 PM
Quote from: 1% on May 13, 2013, 05:13:01 PM
Should raw2dng be making a folder when it dumps a video file... they go all over the place.

This would be great

Edgar
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 13, 2013, 06:04:29 PM
Quote from: Kuky on May 13, 2013, 05:13:28 PM

Also I didn't find a way to stop the recording. For now it seems to just finish the 4gb file. If I press any buttons while recording I get Err 70 and I get very nervous  :o

Hitting the menu button seems to stop recording without crashing.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Danne on May 13, 2013, 06:05:05 PM
Hi! the compiled firmware from Lourenco. Does it run straight from the card involving bootflag procedures like stable builds or do I have to make the card bootable in eoscard before installing it to the camera?
This is so exciting :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: sicetime on May 13, 2013, 06:07:20 PM
Quote from: kslau8 on May 13, 2013, 05:59:41 PM
There's a thread currently that states those Komputerbay 1000X cards will work.

Awesome, can you link to thread? Thanks!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ernestoeduardo on May 13, 2013, 06:15:02 PM
Hi everyone:

First of all THANKS A LOT TO THE ML TEAM for making our lifes easier and our work better. You guys are doing for free what Canon cannot do for a lot of $$$.

I need a compiled build for the MK2 so I can test as much as possible without bricking my camera (here in Cuba 2000$ is a fortune)

BTW: I havfe donated before when 2.3 came out, I will donate again as soon as I get some $ on my account.


Q: Is there any chance the JPG engine can apply some compression to the RAW output ??

SALUDOS !
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 13, 2013, 06:22:09 PM
Quote from: Danne on May 13, 2013, 06:05:05 PM
Hi! the compiled firmware from Lourenco. Does it run straight from the card involving bootflag procedures like stable builds or do I have to make the card bootable in eoscard before installing it to the camera?
This is so exciting :)

run eoscard.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Danne on May 13, 2013, 06:25:07 PM
Thanks a lot Lourenco! Anybody had any luck running the raw2dng through wine bottler in mac?
Cheers :)
//D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 06:33:17 PM
FAO devs - please slow down for f**ks sake. Your amazing ingenuity is making the forum struggle with all the newbies asking questions LOL  ;)

BTW 600d users - I just tested on my 600d and I'm getting continuous shooting at 720x480 on a very under-performing SD card (i.e. 10x slower than what the CF users can get). When scaled to 720p it resolves more detail than native 720p H.264 (at any CBR) and with less moire and aliasing plus you get a little bit of vertical framing to play with. Can't wait till my faster SD gets here. I'm 100% sure the same will be true for scaling slightly larger frames to 1080p. this is bonkers!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: cschmeer on May 13, 2013, 06:37:28 PM
Quote from: Danne on May 13, 2013, 06:25:07 PM
Thanks a lot Lourenco! Anybody had any luck running the raw2dng through wine bottler in mac?

I've been trying. The file Lourenco provided didn't have a suffix in the filename. Adding .exe to it didn't make it work either. Is it compiled or not? How can we get raw2dng running in Wine/WineBottler?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 13, 2013, 06:41:08 PM
anyone got 'raw2dng' working while using a mac?
I am using a mac. downloaded the raw2dng.exe file posted in this thread.
Tried running raw2dng.exe in both 'wine' and 'crossover' and I can't seem to get this figured out.
Anyone successfully converting their files using a mac system?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Danne on May 13, 2013, 06:41:32 PM
To Andy600. Where do you suggest us noobs to ask our questions? Simply don,t answer if you don,t want to right? Or better, let someone else try to answer that feel they do have the time.
Thanks again Lourenco and all the other magical lantern engineers out there.
Proud to be noob ;)
//D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Danne on May 13, 2013, 06:44:21 PM
MAGICAL LANTERN   :D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 13, 2013, 06:47:34 PM
Quote from: aaphotog on May 13, 2013, 06:41:08 PM
anyone got 'raw2dng' working while using a mac?
I am using a mac. downloaded the raw2dng.exe file posted in this thread.
Tried running raw2dng.exe in both 'wine' and 'crossover' and I can't seem to get this figured out.
Anyone successfully converting their files using a mac system?
use the linux version in terminal, dont' use the windows version ;)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Okello on May 13, 2013, 06:48:11 PM
Ahaha Forum's going nuts! Every body needs to calm the F down. Dammm!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: g3gg0 on May 13, 2013, 06:53:22 PM
Quote from: cschmeer on May 13, 2013, 03:21:01 PM
When I saw that there was a "digital dolly" feature that pans across the image as you're recording, it made me think – the 5D Mark III has a built-in accelerometer, which is used to display whether the camera is "level". However, if one were to somehow use that accelerometer and programme the movement of the recording window to compensate camera movement, could this lead to an in-camera stabilising feature? I know this seems crazy - just a thought :)

alex and i already talked about this and pattern detection, but we think that a) the cpu has not enough processing power and b) is the gyro not fast enough.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 06:53:36 PM
Quote from: Danne on May 13, 2013, 06:41:32 PM
To Andy600. Where do you suggest us noobs to ask our questions? Simply don,t answer if you don,t want to right? Or better, let someone else try to answer that feel they do have the time.
Thanks again Lourenco and all the other magical lantern engineers out there.
Proud to be noob ;)
//D

Chill out. It was a compliment to the devs and not in anyway supposed to discourage newcomers from asking questions. I'm still pretty much a newbie myself. That's the great thing about these forums, everybody is welcome and people do help each other. Sorry if my post offended you in any way.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 13, 2013, 06:57:40 PM
Quote from: coutts on May 13, 2013, 06:47:34 PM
use the linux version in terminal, dont' use the windows version ;)

Where can I find the linux version?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: DjJuvan on May 13, 2013, 06:57:52 PM
Hi, it would be nice to upgrade the release to 1.2.1 firmwire for 5d3. Many people go for newer firmwire because of clean HDMI... and ML doesn't work on this version :(
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 13, 2013, 06:58:55 PM
Quote from: 1% on May 13, 2013, 05:13:01 PM
Should raw2dng be making a folder when it dumps a video file... they go all over the place.

raw2dng foo.raw folder/

or: raw2dng.exe foo.raw folder\

;)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 13, 2013, 06:59:28 PM
Quote from: coutts on May 13, 2013, 06:47:34 PM
use the linux version in terminal, dont' use the windows version ;)
Easier said than done hahaha
sooooooo lost
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Rush on May 13, 2013, 06:59:42 PM
Quote from: Andy600 on May 13, 2013, 03:05:33 PM
What card did you use on the 600d?
I used SanDisk 95 MB/S + it helps when Global Draw is off = 960x480 or 1320x360 is capable
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 13, 2013, 07:00:12 PM
Has anyone here went to sleep yet?
I haven't lol
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 07:03:17 PM
Quote from: Rush on May 13, 2013, 06:59:42 PM
I used SanDisk 95 MB/S + it helps when Global Draw is off = 960x480 or 1320x360 is capable

Thanks Rush. It's the same card I've just bought I think.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Danne on May 13, 2013, 07:04:54 PM
Quote from: Andy600 on May 13, 2013, 06:53:36 PM
Chill out. It was a compliment to the devs and not in anyway supposed to discourage newcomers from asking questions. I'm still pretty much a newbie myself. That's the great thing about these forums, everybody is welcome and people do help each other. Sorry if my post offended you in any way.

No offense taken, at all. I guess this place is gonna get filled with all kind of questions :).  I,ll try to dig in a little more by my own first. Maybe an answer pops up meanwhile ;)
//D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: saad on May 13, 2013, 07:08:16 PM
And here is my night (dark) test of the new 14-bit raw ML firmware vs. factory default:



Let me know what you think. :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Stedda on May 13, 2013, 07:11:15 PM
Quote from: saad on May 13, 2013, 07:08:16 PM
Let me know what you think. :)

Wow the difference there is amazing!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: escho on May 13, 2013, 07:14:25 PM
Quote from: a1ex on May 13, 2013, 06:58:55 PM
raw2dng foo.raw folder/

or: raw2dng.exe foo.raw folder\

;)

If raw2dng would create folders by itself, I could type a nice: raw2dng *.RAW, and all my RAW-Videos get decoded and sorted in one step  ;)
I think, I will write a little bash-programm for me, to make work easier  :)

Edgar
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: t2scorp on May 13, 2013, 07:23:00 PM
Quote from: ernestoeduardo on May 13, 2013, 06:15:02 PM
Hi everyone:
I need a compiled build for the MK2 so I can test as much as possible without bricking my camera (here in Cuba 2000$ is a fortune)
Dito,

orderd a KOMPUTERBAY 1000x and would like to test it on the 5D2.

Alex
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 13, 2013, 07:23:00 PM
QuoteIf raw2dng would create folders by itself [...]
Patches are welcome ;)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: CFP on May 13, 2013, 07:25:32 PM
Wow! This night video was really nice. Awesome how much detail one can get thanks of raw. I'm curious to see how the Canon DSLR with ML will perform compared to the Blackmagic Pocket Cinema Camera ...

Quote from: Andy600 on May 13, 2013, 06:33:17 PMThanks Rush. It's the same card I've just bought I think.
And the same I'm going to buy soon. SanDisk SDHC Card Extreme Pro 32GB, right?

It has an average write speed of 60 MByte/s and a maximal write speed of 79 Mbyte/s. Well, I guess the 600D won't let us use this speed ...  :(

Quote from: Andy600 on May 13, 2013, 06:33:17 PM
I just tested on my 600d and I'm getting continuous shooting at 720x480 on a very under-performing SD card
Good to hear that! Thanks for sharing your results.

But now I'm a little bit confused. How did you record in 720 X 480? I've just downloaded the newest Nightly Build and the new "[600D] Auto allocation.zip (https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads/%5B600D%5D%20Auto%20allocation.zip)" folder and put it all on my SD card, like usual.

But when I hit "load modules" inside the Magic Lantern menu, lv_rec and raw_rec both load but I can only access lv_rec. I have no idea how to set up raw_rec and how to start the recording with that ...

So how did you access it? Am I using the wrong version or something?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 07:29:48 PM
CFP - Raw record settings on the 600d are in another menu. No need to touch anything in the modules menu once the module is loaded (I've set to auto load on boot so I don't need to go to the modules menu at all) ;)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 13, 2013, 07:38:46 PM
Has anyone had an issue with either top third or bottom third tearing on a frame?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 13, 2013, 07:44:08 PM
Quote from: bumkicho on May 13, 2013, 07:38:46 PM
Has anyone had an issue with either top third or bottom third tearing on a frame?
confirmed happening on 5d2 for me during my testing.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 13, 2013, 07:47:54 PM
Quote from: coutts on May 13, 2013, 07:44:08 PM
confirmed happening on 5d2 for me during my testing.

I could be wrong, but it seems to be happening when camera is moving. Is that the case for you as well?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: escho on May 13, 2013, 07:50:18 PM
Quote from: a1ex on May 13, 2013, 07:23:00 PM
Patches are welcome ;)

Sorry, but I can only do little /bin/bash-stuff.

This is doing the decoding.work for me now:

#!/bin/bash

for raw in $HOME/test/*.RAW
do
  $HOME/test/raw2dng $raw
  mkdir $HOME/test/`basename $raw .RAW`
  mv $HOME/test/*.dng $HOME/test/`basename $raw .RAW`
done

exit


Edgar
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Alia5 on May 13, 2013, 08:00:07 PM
on 600D recording raw video in 720 X 480 works perfectly fine... didn't test more will try tomorrow!

using a Transcend 16gb class 10 card


This is so fricking awesome!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 13, 2013, 08:02:48 PM
5d2 H264 vs raw comparisons (large images, view at 100% to see details):

https://dl.dropboxusercontent.com/u/33161628/ML/Pictures/H264vsRAW.png
https://dl.dropboxusercontent.com/u/33161628/ML/Pictures/H264vsRAW2.png
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kuky on May 13, 2013, 08:11:42 PM
Ok, little recap, today I used:

- 5D Mark III
- Lexar Professional 600x 16Gb UDMA card (Manufacture Part Numer 508-60729 Rev A)
- firmware posted by lourenco on post nr. 385

I benchmarked the card using the built-in menu command  "Card R/W benchmark (5min)" under "Debug" menu. I have no ideea what is the correct procedure (if have to set something specific before running the bench). The bench just remains on the end screen when finishing. I have no ideea if this is correct, but anyway I found out there is a BMP file saved in the root of the CF.

(https://lh3.googleusercontent.com/-ivEvfhDkTno/UZEYkkUrQnI/AAAAAAAAAgQ/sUf4RI13QpQ/s800/VRAM1.jpg)

Now the results:

A. 1920 x 1080 will drop frames . Probably because of the lower write speed of Lexar 600x (?)
B. 1920 x 1720 will drop frames . Probably because of the lower write speed of Lexar 600x (?)
C. 1280 x 720 will record with no dropped frames. I made some pans to look for tearing and unfortunately there is some from time to time. See below

(https://lh3.googleusercontent.com/-_UaxPEQ3jrY/UZEa_8zWI4I/AAAAAAAAAgs/dJIKI8dLFLA/s800/tearing_example.jpg)



Anyway to share some of the of the content:

Example of file with tearing (but there are more in the sequence):
https://docs.google.com/file/d/0B3ncQl4hSqlncER2VzdZVWk1dWc/edit?usp=sharing (https://docs.google.com/file/d/0B3ncQl4hSqlncER2VzdZVWk1dWc/edit?usp=sharing)

H.264 movie from the first 1300 DNG files of the recording. DNG files were extracted with raw2dng.exe. I did some highlight recovery in Lightroom and there is some flickering on the wall behind. Probably because LR treats recovery on a per file basis. Have to check if this occurs in Davinci Resolve/Speedgrade or similar. 72 MB download, don't bother with the youtube preview, it's nowhere near the mp4 file.
https://docs.google.com/file/d/0B3ncQl4hSqlnUmhWQ2NhaXRNN0E/edit?usp=sharing (https://docs.google.com/file/d/0B3ncQl4hSqlnUmhWQ2NhaXRNN0E/edit?usp=sharing)

First 1300 files of the recording. 1,66 Gb download. If somebody wants to play.
https://docs.google.com/file/d/0B3ncQl4hSqlnRE1BNmdPalp6dms/edit?usp=sharing (https://docs.google.com/file/d/0B3ncQl4hSqlnRE1BNmdPalp6dms/edit?usp=sharing)

Regards,
Cristian
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 13, 2013, 08:17:08 PM
Quote from: bumkicho on May 13, 2013, 07:38:46 PM
Has anyone had an issue with either top third or bottom third tearing on a frame?

Yeah - getting this happening but I think a1ex is on the case:

https://vimeo.com/66077474
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kabuto1138 on May 13, 2013, 08:17:27 PM
Maybe I just read wrong, or I am way too excited.  But is ready to be tested in the 5d Mark II?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: a1ex on May 13, 2013, 08:41:27 PM
Tearing fix: https://bitbucket.org/hudson/magic-lantern/commits/84f657f209eb

(didn't try it)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 13, 2013, 08:45:07 PM
Quote from: a1ex on May 13, 2013, 08:41:27 PM
Tearing fix: https://bitbucket.org/hudson/magic-lantern/commits/84f657f209eb

(didn't try it)

Cool! Does this help with the "skipping frames"? Apologies for not understanding if this is one and the same problem :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Lcrusher on May 13, 2013, 08:45:31 PM
1280*720 works fine with Transcend 400x on 5d3
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: cschmeer on May 13, 2013, 08:51:40 PM
I still haven't found a compiled version of the raw2dng.exe. The file I got from lourenco's post doesn't have an .exe suffix and adding it won't make it run either (I am trying to run it in Wine on a Mac). Could someone please point me in the right direction?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: tferradans on May 13, 2013, 08:57:07 PM
the image is so fuckin' sharper and more detailed that I'm feeling like "where's motion blur?!"

thanks for tear fix, alex!
I'm getting A LOT of tearing when recording 1920x1080 on a Sandisk Extreme Pro 90mbps. Maybe I should make the resolution smaller, right? :P
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: CFP on May 13, 2013, 08:57:40 PM
Quote from: Andy600 on May 13, 2013, 07:29:48 PM
CFP - Raw record settings on the 600d are in another menu.
Ah! I found it, thank you very much!  :)

I was to excited to check the other menus so I didn't even recognized the new "RAW video" option ... Sorry for beeing so blind. But really, thanks a lot!

It seems to work pretty good on my EOS 600D. I was able to record with 720 X 720 at 24 fps. Occasionally a single frame was skipped but I would still call it continously. Awesome! Hopefully something like 1280 X 550 (21:9 aspect ratio) will work one day too.

Could somebody with a fast SD card could give it a try? According to the benchmark my card has a max. write speed of 21 MB/s ... 1280 X 550 would require > 27 MB/s.

Quote from: cschmeer on May 13, 2013, 08:51:40 PM
I still haven't found a compiled version of the raw2dng.exe.
Have you already tried this one: raw2dng.exe (https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads/raw2dng.exe) | From Tragic Lantern 2.0 (https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads)?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 08:59:59 PM
The tearing fix module seems to work smoother.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 13, 2013, 09:04:38 PM
Quote from: tferradans on May 13, 2013, 08:57:07 PM
the image is so fuckin' sharper and more detailed that I'm feeling like "where's motion blur?!"

thanks for tear fix, alex!
I'm getting A LOT of tearing when recording 1920x1080 on a Sandisk Extreme Pro 90mbps. Maybe I should make the resolution smaller, right? :P

tferradans, what camera do you use? 6D?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: tferradans on May 13, 2013, 09:12:45 PM
Quote from: kgv5 on May 13, 2013, 09:04:38 PM
tferradans, what camera do you use? 6D?

I'm using a 5D3 !
--

can anyone get me a working copy for the 600D? please? :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Rush on May 13, 2013, 09:14:04 PM
Quote from: CFP on May 13, 2013, 08:57:40 PM
Could somebody with a fast SD card could give it a try? According to the benchmark my card has a max. write speed of 21 MB/s ... 1280 X 550 would require > 27 MB/s.
check thread - I'd already posted - max possible for 600D/550D/60D is 960x480 or 1320x360.
It is limited not by speed of SD card, but by speed of internal camera SD controller which is limited to 21 MB/s.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 13, 2013, 09:19:54 PM
Quote from: 1% on May 13, 2013, 08:59:59 PM
The tearing fix module seems to work smoother.

Can you share your compiled files with this fix?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: tool on May 13, 2013, 09:24:01 PM
Quote from: Rush on May 13, 2013, 09:14:04 PM
check thread - I'd already posted - max possible for 600D/550D/60D is 960x480 or 1320x360.
It is limited not by speed of SD card, but by speed of internal camera SD controller which is limited to 21 MB/s.
With my 60D (without UHS-1), I must forget anything RAW related.
Transcend 32gb Class 10 (You have to low-format each time (fragmentation stops recording even with CBR 1x)
(http://i.imgur.com/Hb1XOpL.png)
Sandisk 32gb Extreme 45mb/s
(http://i.imgur.com/gHIz9Jn.png)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ernestoeduardo on May 13, 2013, 09:26:31 PM
Can anyone please, point us (OVER ENTHUSIAST NEWBIES) in the direction of a complied safe build for the MK2, so we can test/try the new features and CF performances ?

I think that our intention of being part of the development/test progress is fair, maybe we wont be writing code, but I think that inteligent testing is within our range.

Forgive us those who think that we are slowing the process down with our willing to learn, but I rather have A1ex, Gr3ggo, 1% and others ignoring me (completly understandable and fine), than staying silent waiting on the margin of this technological break trough.

Thanks to everyone again !
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: CFP on May 13, 2013, 09:28:34 PM
Quote from: tferradans on May 13, 2013, 09:12:45 PM
can anyone get me a working copy for the 600D? please? :)
Here it is: [600D] Auto allocation.zip (https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads/%5B600D%5D%20Auto%20allocation.zip) | From Tragic Lantern 2.0 (https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads).

Just replace your current "autoexec.bin" with that new one and put the "MODULES" folder into your "ML" folder.

Quote from: Rush on May 13, 2013, 09:14:04 PM
check thread - I'd already posted - max possible for 600D/550D/60D is 960x480 or 1320x360.
It is limited not by speed of SD card, but by speed of internal camera SD controller which is limited to 21 MB/s.
Yes, I've read your post. I've read ervey post :)

But like I said, 720 X 720 is also possible. 1280 X 550 is just a tiny bit more ... So I thought maybe there would be a chance to get it. But if there's no way to speed up the SD controler I better stop dreaming  ;D

Anyway, thanks for your reply.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 13, 2013, 09:42:07 PM
Is everyone here running windows OS?
I still cant seem to figure out how to use this raw2dng on my mac. I've tried the 'wine' as well as 'crossover' and I just can't seem to get things together.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Andy600 on May 13, 2013, 09:44:24 PM
CFP - Even without being able to shoot 1080p, It's not the end of the world for us 'prosumer camera' users when it comes to RAW. The images still look way better than what we get with H.264 even if we have to shoot smaller and scale up. It's also great that we get the chance to record our own RAW footage to play with and develop/learn RAW post production methods with manageable files sizes. Think of it as a trial period before we upgrade to 5d MkIII's etc. (that won't be anytime soon for me unfortunately)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 13, 2013, 09:54:09 PM
Anyone got a solution when you get the "cannot pars file" error? Is this due to corrupt files? They seem to extract okay from the RAW file.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: tferradans on May 13, 2013, 10:00:46 PM
Did anyone notice that every single frame has 240DPI resolution? A regular H264/RGB frame has 72DPI, which means this can be upscaled about 3-4 times without much damage to image quality, right?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 10:01:16 PM
I think that means you picked a size that doesn't work. Lemme know all the ones that don't work and I'll take them out. We can try some different anamorphic ones.

Coutts made it for 5d2.. not sure where he put it.


Also these clips are useful for short stuff. How big is an average cutaway really? The 1280 will probably uprez to 1080p and the 960 will probably pull off 720P compared to the H264. I don't think you will all need 5d3s... just CF cameras unless something can be done to compress the frames to 1.5MB.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 13, 2013, 10:04:02 PM
Quote from: 1% on May 13, 2013, 10:01:16 PM
I think that means you picked a size that doesn't work. Lemme know all the ones that don't work and I'll take them out. We can try some different anamorphic ones.

Coutts made it for 5d2.. not sure where he put it.


Also these clips are useful for short stuff. How big is an average cutaway really? The 1280 will probably uprez to 1080p and the 960 will probably pull off 720P compared to the H264. I don't think you will all need 5d3s... just CF cameras unless something can be done to compress the frames to 1.5MB.
start writing a raw compressor? :P
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: CFP on May 13, 2013, 10:04:20 PM
Quote from: Andy600 on May 13, 2013, 09:44:24 PM
CFP - Even without being able to shoot 1080p, It's not the end of the world for us 'prosumer camera' users when it comes to RAW. The images still look way better than what we get with H.264 even if we have to shoot smaller and scale up. It's also great that we get the chance to record our own RAW footage to play with and develop/learn RAW post production methods with manageable files sizes. Think of it as a trial period before we upgrade to 5d MkIII's etc. (that won't be anytime soon for me unfortunately)
I know! And like I said before, I'm still very excited about all this! It is truely amazing that the cameras actually can handle RAW video! And even if the resolution isn't as high as I'm used to, it still is enough to work with.

I didn't want to sound ungrateful. Instead, I really appreciate all this! Magic Lantern was already amazing before this feature was added. Now it's just more amazing than ever before  ;D

But I think I'll stuck with my 600D for a little bit longer. I just bought it in July 2012 and I still like it ;)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: tferradans on May 13, 2013, 10:04:46 PM
Quote from: 1% on May 13, 2013, 10:01:16 PM
I think that means you picked a size that doesn't work. Lemme know all the ones that don't work and I'll take them out. We can try some different anamorphic ones.

What do you mean by different anamorphic ones? :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Vegandelight on May 13, 2013, 10:11:47 PM

Edit: N/M

Spoke to soon as always :)

SD: Sandisk Extreme 95Mb/s 16gb.
CF: Komputerbay 1000x 32gb

Now i need to figure out how to record  :o
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 10:25:14 PM
Different anamorphic resolutions that don't come out screwed up.


Raw compressor would be nice... but the CPU to do it, lol. Would need something like this:

http://code.google.com/p/lz4/
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Alia5 on May 13, 2013, 10:33:57 PM
On 600D i get a lot of frames like this...
(http://orly.pwnz.org/jing/trash/2013-05-13_2231.png)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 13, 2013, 10:34:27 PM
Quote from: 1% on May 13, 2013, 10:01:16 PM
I think that means you picked a size that doesn't work. Lemme know all the ones that don't work and I'll take them out. We can try some different anamorphic ones.

1504x640 res works just fine on 6D and it's optimal for continous shooting, we don't loose any line after upscaling. We could try to tweak something about 700 pix high,width 1600 or 1650 maybe
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Colemar on May 13, 2013, 10:35:56 PM
I would be most grateful if a current compilation would be posted for the 5DmkIII.  Many thanks!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 10:40:46 PM
Lol, I should have bought sandisk 45MB card... that's close to continuous for me but not always.

I bet HDR would work with this if it switched while not recording.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kgv5 on May 13, 2013, 10:44:46 PM
Quote from: Vegandelight on May 13, 2013, 10:11:47 PM

SD: Sandisk Extreme 95Mb/s 16gb.
CF: Komputerbay 1000x 32gb


5d3 doesn't supports UHS-I, does it? That's why only 21mb/s from SD.
Do we know SD controller limit of the 6D? Hope it's not 40, in theory probably should be about 50mb/s
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 10:46:22 PM
100mhz so 50 is interface maximum.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: CFP on May 13, 2013, 10:54:58 PM
Quote from: 1% on May 13, 2013, 10:40:46 PM
Lol, I should have bought sandisk 45MB card...
Well, I have that one and it doesn't help at all since the 600D limits the write speed to 21 MB/s ... So I guess you'll be fine with whatever you have at the moment :D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: sicetime on May 13, 2013, 10:56:41 PM
are any of you running raw2dng on a mac? Does it work on wine or vmware fusion?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 13, 2013, 11:02:21 PM
Quote from: 1% on May 13, 2013, 10:01:16 PM
I think that means you picked a size that doesn't work

Sounds about right. I can't remember which size it was but I think the same one gave me the same result.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: mnteddy on May 13, 2013, 11:08:47 PM
Quote from: sicetime on May 13, 2013, 10:56:41 PM
are any of you running raw2dng on a mac? Does it work on wine or vmware fusion?
Works great for me in CrossOver. "Programs - Run Command" drop in the raw2dng.exe, then the .RAW file. Spits out a stream of DNGs.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: kidb on May 13, 2013, 11:12:09 PM
Quote from: sicetime on May 13, 2013, 10:56:41 PM
are any of you running raw2dng on a mac? Does it work on wine or vmware fusion?
Compiled on mountain lion, use in terminal.
http://kidb.tv/.ml/
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 13, 2013, 11:16:11 PM
Double buffering seems to have fixed 5d2:



Frame dropping starts about 18 seconds in and continues for the rest of the video (as expected)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 13, 2013, 11:19:32 PM
Quotefine with whatever you have at the moment :D

I have 600D and 6D so fine on 600D (21Mb/s max) but not on 6D (50MB theoretical)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kuky on May 13, 2013, 11:20:05 PM
I uploaded my test to Vimeo. Play in HD



My setup:

- 5D Mark III
- Lexar Professional 600x 16Gb UDMA card (Manufacture Part Numer 508-60729 Rev A)
- firmware posted by lourenco on post nr. 385
- resolution set to 1280 x 720, only one that does not drop frames on this card

There are two problems I find:

a. tearing on some frames and I think it's always on the same line the tearing happens (may be wrong)
b. stuttering on pans, more evident for example from sec 42 to 52.

Questions:

A. Does the fix posted by Alex solve both? I don't know even if the're connected.
Maybe the raw video feed which the developers are trying to "redirect" to the CF it's not capable of genlocked 24 fps, hence the stuttering.
To me it doesn't seem that this raw feed is used to make the video recordings on the genuine firmware. The quality difference it's too much. Just an opinion  8)

B. It's the clip shot in RAW mode 24,000 fps or 23,976fps ??




PS. I solved the flickering on the wall, going through the AfterEffects - ACR route.

Regards,
Cristian
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ThePhil on May 13, 2013, 11:20:40 PM
I know this is going to ruffle some feathers but I promise I'm not posting out of ignorance.  I've been using ML for over a year on my 5D2 and swear by it.  I've been keeping up with reading the forums and decided to try the nightly build to use RAW Video on my 5D2 but don't see the option after replacing the autoexec.bin and ML folder on the root of the card with the nightly build...  I'm using a Sandisk Extreme Pro 16GB CF 90MB/s... not the 1000x.  Is the card speed the reason or am I missing a step?


Thank you.

Phil
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 13, 2013, 11:21:12 PM
Quote from: kidb on May 13, 2013, 11:12:09 PM
Compiled on mountain lion, use in terminal.
http://kidb.tv/.ml/
How do I use it in terminal?
When I download and double click the file, terminal pops up... what then?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Colemar on May 13, 2013, 11:26:53 PM
Shooting on a 5DmkIII, why are the DNG's converting as 128x96 resolution?  They are almost 4MB each and the .raw file hasn't dropped a frame..I have access to 1000x cards from Delkin, Lexar, and SanDisk, and would like to test them all for members' information, but cannot get useable video from the process..will anyone help me?

-Colemar Nichols
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 13, 2013, 11:30:01 PM
Quote from: mnteddy on May 13, 2013, 11:08:47 PM
Works great for me in CrossOver. "Programs - Run Command" drop in the raw2dng.exe, then the .RAW file. Spits out a stream of DNGs.
first, I open crossover then I click "install windows software"
I choose "other application"
I then select "raw2dng.exe"
I let it install into the bottleneck.
Then I click 'programs-run command'
under "use bottle" it says 'raw2dng'
I drag my raw file into the space thats open that says "command"
then I click "run"

when I do that it says "there is no windows program configured to open this type of file"
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: mageye on May 13, 2013, 11:31:15 PM
Quote from: coutts on May 13, 2013, 11:16:11 PM
Double buffering seems to have fixed 5d2:



Frame dropping starts about 18 seconds in and continues for the rest of the video (as expected)

I am really interested to see 5DMKII stuff and this is some of the first I have seen. I would like to try the build you are using (maybe you could link me to your build?).

Anyway its encouraging footage and I was wondering what CF card you have and write speed you are achieving?

Please people quote numbers!

Keep it up good people. YAY!!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: KahL on May 13, 2013, 11:31:46 PM
Quote from: CFP on May 13, 2013, 08:57:40 PM
Ah! I found it, thank you very much!  :)

I was to excited to check the other menus so I didn't even recognized the new "RAW video" option ... Sorry for beeing so blind. But really, thanks a lot!

It seems to work pretty good on my EOS 600D. I was able to record with 720 X 720 at 24 fps. Occasionally a single frame was skipped but I would still call it continously. Awesome! Hopefully something like 1280 X 550 (21:9 aspect ratio) will work one day too.

Could somebody with a fast SD card could give it a try? According to the benchmark my card has a max. write speed of 21 MB/s ... 1280 X 550 would require > 27 MB/s.
Have you already tried this one: raw2dng.exe (https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads/raw2dng.exe) | From Tragic Lantern 2.0 (https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads)?

Being used on the 600D, does this mean I can test it out on the 550D/T2i as well? If so, how can I go about fetching them files? :)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kuky on May 13, 2013, 11:36:22 PM
Quote from: Colemar on May 13, 2013, 11:26:53 PM
Shooting on a 5DmkIII, why are the DNG's converting as 128x96 resolution?  They are almost 4MB each and the .raw file hasn't dropped a frame..I have access to 1000x cards from Delkin, Lexar, and SanDisk, and would like to test them all for members' information, but cannot get useable video from the process..will anyone help me?

-Colemar Nichols

I think they are recognised only by LR and ACR. Davinci Resolve also sees them as 128x96 (most likely a preview incorporated in the headers).
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: slang on May 13, 2013, 11:40:15 PM
Hmmm... Still getting "EnableBootDisk ;)" error. What am I doing wrong? Have the 1.1.3 firmware, 2 cards (lexar and sandisk SD for ML boot). Copied all from here: https://docs.google.com/file/d/0B7QlH_BH2m32dWpOOEpkWktXMXM/edit to SD card (without raw2dng file of course). Something i missed?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: t2scorp on May 13, 2013, 11:41:34 PM
Quote from: mageye on May 13, 2013, 11:31:15 PM
I am really interested to see 5DMKII stuff and this is some of the first I have seen. I would like to try the build you are using (maybe you could link me to your build?).

Anyway its encouraging footage and I was wondering what CF card you have and write speed you are achieving?

Please people quote numbers!

Keep it up good people. YAY!!
He is using the Lexar 1000x

I'd also like to get your build to make some test shots. Do you use line skipping and how much is the moire a problem? Are you able to get 1:1 crop from sensor without moire?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Colemar on May 13, 2013, 11:42:15 PM
QuoteI think they are recognised only by LR and ACR. Davinci Resolve also sees them as 128x96 (most likely a preview incorporated in the headers)

Thanks Kuky, you must be right.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: t2scorp on May 13, 2013, 11:43:34 PM
Quote from: slang on May 13, 2013, 11:40:15 PM
Hmmm... Still getting "EnableBootDisk ;)" error. What am I doing wrong? Have the 1.1.3 firmware, 2 cards (lexar and sandisk SD for ML boot). Copied all from here: https://docs.google.com/file/d/0B7QlH_BH2m32dWpOOEpkWktXMXM/edit to SD card (without raw2dng file of course). Something i missed?

Install the latest stable build and replace the autoexec
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kuky on May 13, 2013, 11:45:14 PM
Quote from: coutts on May 13, 2013, 11:16:11 PM
Double buffering seems to have fixed 5d2:



Frame dropping starts about 18 seconds in and continues for the rest of the video (as expected)

Very hard to judge from that video. More likely not. Maybe put the camera on a tripod and do a smooth pan. Use the smallest resolution which can eliminate dropped frames (maybe 1280 x 720 ?). I'm curious also what happens.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: eatstoomuchjam on May 14, 2013, 12:05:14 AM
Quote from: ThePhil on May 13, 2013, 11:20:40 PM
I know this is going to ruffle some feathers but I promise I'm not posting out of ignorance.  I've been using ML for over a year on my 5D2 and swear by it.  I've been keeping up with reading the forums and decided to try the nightly build to use RAW Video on my 5D2 but don't see the option after replacing the autoexec.bin and ML folder on the root of the card with the nightly build...  I'm using a Sandisk Extreme Pro 16GB CF 90MB/s... not the 1000x.  Is the card speed the reason or am I missing a step?


Thank you.

Phil

It took me a minute to figure it out.  Assuming that the Mark II works like my Mark III, you'd go into the ML menus and go to the one on the far right labeled "M."  Click "load modules now" and at the bottom, it should say "raw_rec OK."  Now when you go to the movie menu, you should have the raw video menu item as expected.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Colemar on May 14, 2013, 12:07:22 AM
coutts,
   will you please post the compiled modules with the double-buffering commands?  I would like to begin framerate testing on different card makes and speeds asap.

many thanks,
-Colemar Nichols
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 14, 2013, 12:14:56 AM
Quote from: ThePhil on May 13, 2013, 11:20:40 PM
I know this is going to ruffle some feathers but I promise I'm not posting out of ignorance.  I've been using ML for over a year on my 5D2 and swear by it.  I've been keeping up with reading the forums and decided to try the nightly build to use RAW Video on my 5D2 but don't see the option after replacing the autoexec.bin and ML folder on the root of the card with the nightly build...  I'm using a Sandisk Extreme Pro 16GB CF 90MB/s... not the 1000x.  Is the card speed the reason or am I missing a step?


Thank you.

Phil

Go to the last tab on the right (modules) and select Load Modules Now to load new features :) You will have a new tab for raw_rec on the far right and Lv_rec will live  at the bottom of all the stuff in the movie tab

Enjoy ;)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: KahL on May 14, 2013, 12:16:27 AM
Quote from: noisyboy on May 14, 2013, 12:14:56 AM
Go to the last tab on the right (modules) and select Load Modules Now to load new features :) You will have a new tab for raw_rec on the far right and Lv_rec will live  at the bottom of all the stuff in the movie tab

Enjoy ;)

I'm seeing posts for this for the 600D. Assuming it's the same build, would this be possible on the 550D as well? And if so, how do we access it?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 14, 2013, 12:18:53 AM
Quote from: KahL on May 14, 2013, 12:16:27 AM
I'm seeing posts for this for the 600D. Assuming it's the same build, would this be possible on the 550D as well? And if so, how do we access it?

In all honesty dude I can only talk from experience of running on my 6D so I don't really know. I'm sure someone much more helpful than I am will give you a better answer soon :)

Good luck!  8)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ThePhil on May 14, 2013, 12:30:30 AM
Noisyboy and EatsTooMuchJam,

Thank you for the reply!  The far right tab in my menu shows an "i" at the top... not an M :/

The menu choices are the same as are shown here: http://www.eoshd.com/content/10324/big-news-hands-on-with-continuous-raw-recording-on-canon-5d-mark-iii  but without the option for "RAW Video" of course.


Thanks again!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: tferradans on May 14, 2013, 12:38:14 AM
about the fix for tearing, it worked for me, but I got a lot of stuttering this time...
http://www.youtube.com/watch?v=sNqVMvTIqVE

5D3 600x Sandisk Card. This one was shot lo-res (720x480), so, it's way below the card's max speed. any fixes or explanations?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Habitat on May 14, 2013, 12:40:36 AM


Thought I'd mark my first post with something humorous. I bet Hitler is happy now.

Thanks for this by the way I can't wait until I get my CF cards to try out this release with my Mark III
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: TonesProtege on May 14, 2013, 12:41:36 AM
Having the exact same problem with my 5Dmrk II as ThePhil. Have replaced the Autoexec.bin and ML folders with the dailys but cant seem to find the Load Modules tab, the Info tab is the last for me.

Jake
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 14, 2013, 12:49:27 AM
This should work for 5d2, double buffering enabled:
http://bit.ly/10TBM7a

extract the MODULES directory into /ML/ on your card
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ThePhil on May 14, 2013, 12:55:39 AM
Quote from: coutts on May 14, 2013, 12:49:27 AM
This should work for 5d2, double buffering enabled:
http://bit.ly/10TBM7a

extract the MODULES directory into /ML/ on your card


Thank you Coutts! Confirmation that it worked!  I'm going to list a few keywords below to help any other ML users locate this post via search.

5D Mark II Raw Video
5D Mark II RawVideo
Enabling Raw Video
Enabling RawVideo
Load Modules Tab
Loading Modules.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 01:02:07 AM
Quote from: kidb on May 13, 2013, 11:12:09 PM
Compiled on mountain lion, use in terminal.
http://kidb.tv/.ml/

I opened up terminal, dragged and dropped the raw2dng file in and clicked enter.
it said "permission denied"

am I supposed to do something different?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Kuky on May 14, 2013, 01:17:43 AM
Quote from: tferradans on May 14, 2013, 12:38:14 AM
about the fix for tearing, it worked for me, but I got a lot of stuttering this time...
http://www.youtube.com/watch?v=sNqVMvTIqVE

5D3 600x Sandisk Card. This one was shot lo-res (720x480), so, it's way below the card's max speed. any fixes or explanations?

Uff, it's looking bad... Same as on my test (Reply #493) minus the tearing.

Check this test also and look carefully (on the HD version on vimeo website) at the cars on the 14-bit raw crop



To me it appears that the stuttering it's not tied to the movement of the camera, but somehow the frames are not aligned in time very precise. There are more or less 24 fps but not "genlocked" (don't know a better term).
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: eatstoomuchjam on May 14, 2013, 01:19:35 AM
Quote from: aaphotog on May 14, 2013, 01:02:07 AM
I opened up terminal, dragged and dropped the raw2dng file in and clicked enter.
it said "permission denied"

am I supposed to do something different?

You wouldn't drag it into the terminal.  Assuming that it's in your Downloads folder, you'd go into Terminal and run this:


cd ~/Downloads
chmod +x ./raw2dng
./raw2dng


To use it on a file in another folder, go to terminal and do something like this:

cd /Path/To/Folder
~/Downloads/raw2dng M00000000.RAW

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 01:22:03 AM
Ok, I figured it out...

But now, my SD card isn't working.
I tried shooting at 720 and it started but it was missing a bunch of frames so I wanted to cancel it. I did so by flipping the video switch to camera on my 5d3. Then I took my sd and put it in my laptop and it didn't read it. Placed it back into the camera and the camera read it so I did a format on my camera. Put it back into my laptop and it still isn't reading it. Restarted the computer and did a format of the card(in camera) still nothing. Any ideas?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 01:23:18 AM
Quote from: eatstoomuchjam on May 14, 2013, 01:19:35 AM
You wouldn't drag it into the terminal.  Assuming that it's in your Downloads folder, you'd go into Terminal and run this:


cd ~/Downloads
chmod +x ./raw2dng
./raw2dng


To use it on a file in another folder, go to terminal and do something like this:

cd /Path/To/Folder
~/Downloads/raw2dng M00000000.RAW


What eventually worked for me was, right clicking M00000000.RAW and selecting "open with" then opening with the raw2dng and it went to work for me.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: tferradans on May 14, 2013, 01:26:12 AM
Quote from: Kuky on May 14, 2013, 01:17:43 AM
To me it appears that the stuttering it's not tied to the movement of the camera, but somehow the frames are not aligned in time very precise. There are more or less 24 fps but not "genlocked" (don't know a better term).

yeah! pretty much like this. In a shaky shot, I got a frame misplaced by other 10 frames.

by the way, devs: any chance of assigning raw_rec to a particular key, instead of going through the menus?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: eatstoomuchjam on May 14, 2013, 01:27:32 AM
Quote from: aaphotog on May 14, 2013, 01:22:03 AM
Ok, I figured it out...

But now, my SD card isn't working.
I tried shooting at 720 and it started but it was missing a bunch of frames so I wanted to cancel it. I did so by flipping the video switch to camera on my 5d3. Then I took my sd and put it in my laptop and it didn't read it. Placed it back into the camera and the camera read it so I did a format on my camera. Put it back into my laptop and it still isn't reading it. Restarted the computer and did a format of the card(in camera) still nothing. Any ideas?

I've been stopping recording by pulling up the ML menu and hitting "stop" in the same place where I hit "start."  You could try that.  I'd also strongly suggest using something other than an SD card on a 5D3.  A 400x-600x CF card is going to write a lot faster.  I'm not sure that the SD card slot in the 5D3 can keep up with 720x1280.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 01:32:14 AM
Quote from: eatstoomuchjam on May 14, 2013, 01:27:32 AM
I've been stopping recording by pulling up the ML menu and hitting "stop" in the same place where I hit "start."  You could try that.  I'd also strongly suggest using something other than an SD card on a 5D3.  A 400x-600x CF card is going to write a lot faster.  I'm not sure that the SD card slot in the 5D3 can keep up with 720x1280.

Yea, I know it can't. But the first video I tried. It lost ALOT less frames.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Colemar on May 14, 2013, 01:45:36 AM
Quote from: coutts on May 14, 2013, 12:49:27 AM
This should work for 5d2, double buffering enabled:
http://bit.ly/10TBM7a

extract the MODULES directory into /ML/ on your card

coutts,
  thank you for the post.  unfortunately that was a small step backwards for me as now an error message is generated when attempting to load the module.  I do not know how to attach things or I would include a .422 screenshot...

QuoteASSERT: IsSuiteSignature( hSuite )
at ./PackMemory/PackMem.c:599, task RscMgr
lv:1 mode:3


Magic Lantern version : v2.3.NEXT.2013May13.5D3113
Mercurial changeset   :
Built on 2013-05-12 21:25:54 by magiclantern@magiclantern-VirtualBox.
Free Memory  : 145K + 3731K

and

QuoteASSERT: 0
at ./Memory/Memory.c:568, task RscMgr
lv:1 mode:3


Magic Lantern version : v2.3.NEXT.2013May13.5D3113
Mercurial changeset   :
Built on 2013-05-12 21:25:54 by magiclantern@magiclantern-VirtualBox.
Free Memory  : 145K + 3874K


-Colemar
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 14, 2013, 01:53:28 AM
Is lourenco back with newly compiled 5D3 files? Or can anyone else post them please?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 14, 2013, 02:02:07 AM
Quote from: Colemar on May 14, 2013, 01:45:36 AM
coutts,
  thank you for the post.  unfortunately that was a small step backwards for me as now an error message is generated when attempting to load the module.  I do not know how to attach things or I would include a .422 screenshot...

and


-Colemar
use the autoexec.bin in that zip.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: TonesProtege on May 14, 2013, 02:03:49 AM
Thanks coutts, your link worked. However I'm now having trouble getting raw2dng to work. I DL'ed crossover, but i cant seem to get it to open my m0000.raw files. Any chance any of y'all could help me understand how to get that to work? I'm on a Mac.

Jake
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 14, 2013, 02:08:11 AM
Quote from: Rush on May 13, 2013, 06:59:42 PM
helps when Global Draw is off

Can confirm I too had increased performance with global draw off.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Colemar on May 14, 2013, 02:10:13 AM
Quote from: coutts on May 14, 2013, 02:02:07 AM
use the autoexec.bin in that zip.

on the 5DmkIII with that autoexec.bin the camera will not start up.  When reverting to lorenco's autoexec.bin and using the modules you provided, the error messages appear.  When reverting everything to lourenco's build, the camera functions properly but there is some image tearing.  Just to make sure, the autoexec.bin and modules you kindly provided are for the mkIII ?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: coutts on May 14, 2013, 02:22:28 AM
Quote from: Colemar on May 14, 2013, 02:10:13 AM
on the 5DmkIII with that autoexec.bin the camera will not start up.  When reverting to lorenco's autoexec.bin and using the modules you provided, the error messages appear.  When reverting everything to lourenco's build, the camera functions properly but there is some image tearing.  Just to make sure, the autoexec.bin and modules you kindly provided are for the mkIII ?
the filename says 5d2, of course it won't work on the 5d3.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 14, 2013, 02:22:52 AM
Quote from: bumkicho on May 14, 2013, 01:53:28 AM
Is lourenco back with newly compiled 5D3 files? Or can anyone else post them please?
working on it. Just came home from work.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Colemar on May 14, 2013, 02:26:15 AM
Quote from: coutts on May 14, 2013, 02:22:28 AM
the filename says 5d2, of course it won't work on the 5d3.

very good point.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 02:29:49 AM
Quote from: eatstoomuchjam on May 14, 2013, 01:19:35 AM
You wouldn't drag it into the terminal.  Assuming that it's in your Downloads folder, you'd go into Terminal and run this:


cd ~/Downloads
chmod +x ./raw2dng
./raw2dng


To use it on a file in another folder, go to terminal and do something like this:

cd /Path/To/Folder
~/Downloads/raw2dng M00000000.RAW


It says "illegal instruction" when I try that. I put the raw2dng file in my downloads just to try it this way.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 14, 2013, 02:39:31 AM
Is anyone using Lexar 1000x card on 5D3? I am wondering if full 1080P is stable with this card. Or even higher?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 14, 2013, 02:40:29 AM
Here is the current compiled version for the 5D Mark III
https://docs.google.com/file/d/0B7QlH_BH2m32T3RSTXRUM1VXTWM/edit?usp=sharing
Title: Suggestion to change the title of this thread
Post by: ted ramasola on May 14, 2013, 02:40:57 AM
Can a moderator change the title of this fast growing thread to REMOVE the YUV422 and just leave 14bit Raw video recording?

This thread is being referenced to in blogs, news, webpages, social media, all around the world and its adding to confusion when some refer to the raw video as yuv422.
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 14, 2013, 02:42:39 AM
Quote from: bumkicho on May 14, 2013, 02:39:31 AM
Is anyone using Lexar 1000x card on 5D3? I am wondering if full 1080P is stable with this card. Or even higher?
It works fine for me at 1080p. I have not tried anything higher yet.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Colemar on May 14, 2013, 02:46:47 AM
In my brief testing the Lexar 1000x  is not stable at higher than 1920x1080 @23.976 

thank you lourenco, I will be testing the Lexar 1000x, Sandisk 100MB/s, and Delkin 1000x on the 5DmkIII tonight and will post back results asap
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 14, 2013, 02:48:55 AM
Quote from: lourenco on May 14, 2013, 02:40:29 AM
Here is the current compiled version for the 5D Mark III
https://docs.google.com/file/d/0B7QlH_BH2m32NlR1cDR1eS11aVU/edit?usp=sharing

Fonts are missing?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 14, 2013, 02:51:46 AM
Quote from: squig on May 14, 2013, 02:48:55 AM
Fonts are missing?
I will re-upload. You would just want to replace your existing files with this one. Do not delete your existing ML dir. I will make it complete for you. one moment.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 14, 2013, 02:52:56 AM
Quote from: lourenco on May 14, 2013, 02:51:46 AM
I will re-upload. You would just want to replace your existing files with this one. Do not delete your existing ML dir. I will make it complete for you. one moment.

Ahh yeah I just noticed you only put the modules folder in. No problem. Thanks.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Rush on May 14, 2013, 02:54:51 AM
Canon 600D RAW video test footage 1280x360 25p:

Not so compressed version with more details: http://bit.ly/YFM92c

Thanks to 1% for 600D version. I tested "[600D] Stick to 1280".
It sometimes shows many corrupted frames + looks like when accessing ML menu it adds some garbage to frame. In video above I just took clean part of full video.
Can I request you to add 400px height? It seems like max possible and optimal for 600D is 1280x400 - tight 3.2:1 aspect. (960 width is good for achieving 16:9 - 960x540 but have more moire)
Can I request you to add 1280x720 lineskipped to 1280x360? So it will be nice 16:9 when stretched back to 1280x720 (it will look like 720i with one field discarded and second interpolated in post).
Is line skipping possible with RAW?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: N/A on May 14, 2013, 02:58:26 AM
Looks like raw can run continuously on the 600D at 960x540 (at work now so I can't double check) with a Sandisk 95. So how about 1280x480 or 360? Not sure if frames were corrupting yet, I'll run more tests in the morning.

Oh and from what I've seen, raw video has a nice film look to it color-wise, especially on landscape shots. The stock codec looks extremely "digital" next to it.  This makes me happy.

:D

Looks like my question was just answered lol
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 14, 2013, 03:03:47 AM
Quote from: lourenco on May 14, 2013, 02:40:29 AM
Here is the current compiled version for the 5D Mark III
https://docs.google.com/file/d/0B7QlH_BH2m32T3RSTXRUM1VXTWM/edit?usp=sharing

Thank you!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 14, 2013, 03:07:05 AM
Quote from: coutts on May 14, 2013, 12:49:27 AM
This should work for 5d2, double buffering enabled:
http://bit.ly/10TBM7a

extract the MODULES directory into /ML/ on your card

Aw yeah, thanks coutts! I'm testing the waters atm but if all goes well, I'm going to find something with a lot of detail to share with everyone. Today isn't breezy so no swaying tree leaves. I'll probably record sprinkler droplets or something.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 14, 2013, 03:07:45 AM
Quote from: ted ramasola on May 13, 2013, 04:50:21 AM
wow... just wow. So raw recording can record now sound? Is it 44khz or 48khz?
I can record sound now to the SD card. I did a quick test. I did not want to change any code. I went to the canon menu set up1 and selected record to sd card. I went to magic lantern menu for sound recorder. I started sound recorder. I went back to same canon menu before put back to cf card. I was able to record video at 1080p. This means it is possible. The issue is when I start video recording the audio switch from 48hz to 96hz for some reason. I had to change the sample rate to 96hz to play back the audio correctly.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 14, 2013, 03:09:24 AM
Hmm.. with the current version, I get drop frames at 1280x720. I used to get stable 1280x720 with previous version. Is this expected?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 14, 2013, 03:14:08 AM
Quote from: bumkicho on May 14, 2013, 03:09:24 AM
Hmm.. with the current version, I get drop frames at 1280x720. I used to get stable 1280x720 with previous version. Is this expected?

Is global draw off? It's working fine for me with a 600x Lexar
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: hjfilmspeed on May 14, 2013, 03:16:32 AM
OHHHHHHHHHHHH MYYYYYYYYYYYYYYYY WWWWWOOOOOOOOOOOOORRRRRDDDDDDDDDDDDDDDD! IIIIII LOOOOOOOOOOVEEE YOOUUUUUUUUU MMMMMMMMMMMLLLLLLLLLLLLLL
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: hjfilmspeed on May 14, 2013, 03:19:03 AM
Do you think there will be a 5d3 alpha 4 soon for dumb dumbs like me? haha Im so excited!!!!! I want to try this asap but i have know clue how ha ha.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 03:22:40 AM
I've been sitting at my computer for the last 6+ hours straight trying to figure out how to turn the raw file into DNG's and nothing is working out for me.

Working with a mac.
I got it one time but for some reason, none of the steps that I've taken can get me there again.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 14, 2013, 03:28:10 AM
Quote from: squig on May 14, 2013, 03:14:08 AM
Is global draw off? It's working fine for me with a 600x Lexar
With only focus peaking enabled, I can get stable 1280x720. With peaking and histogram on, I get stable 1280x540.
I will order a lexar 1000x card soon. If I can get stable 1920x720, that would be great.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 14, 2013, 03:30:39 AM
Quote from: lourenco on May 14, 2013, 03:07:45 AM
I can record sound now to the SD card. I did a quick test. I did not want to change any code. I went to the canon menu set up1 and selected record to sd card. I went to magic lantern menu for sound recorder. I started sound recorder. I went back to same canon menu before put back to cf card. I was able to record video at 1080p. This means it is possible. The issue is when I start video recording the audio switch from 48hz to 96hz for some reason. I had to change the sample rate to 96hz to play back the audio correctly.

Interesting! I will try this too. Thanks!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ernestoeduardo on May 14, 2013, 03:43:18 AM
Quote from: coutts on May 14, 2013, 12:49:27 AM
This should work for 5d2, double buffering enabled:
http://bit.ly/10TBM7a

extract the MODULES directory into /ML/ on your card

Things I have noticed when testing Newest Dbufer build by Coutts for MK2:

1- I extracted /Modules on /ML; overwrited autoexec.bin and the build loaded OK
2- Loaded the RAW_rec module successfully
3- If you hit "load modules" 2 or 3 times quickly the camera freezes, restart is needed red led remains lit


Benchmark with my CFs:

Average(read/write) all with "Global Draw on"


Duracell 600x                 @ 12mbs/11mbs
Sandisk Extreme 60mbs @ 45mbs/41mbs
Trascend 60mbs             @ 27mbs/72mbs


I think i might need faster CFs don you think ???  ;D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 14, 2013, 03:44:53 AM
Quote from: aaphotog on May 14, 2013, 03:22:40 AM
I've been sitting at my computer for the last 6+ hours straight trying to figure out how to turn the raw file into DNG's and nothing is working out for me.

Working with a mac.

Join the mac lepper club.

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Colemar on May 14, 2013, 03:49:45 AM
5DmkIII preliminary basic Compact Flash card speed tests yield:

Sandisk Extreme Pro 128GB 100MB/s UDMA 7 CF burst ~82MB/s with ~73MB/s sustained write speed @ 1920x960.  It would randomly drop frames at 1920x1080 but did complete the 4GB size limit before ceasing.  It was the only card of the trio that could record 1920 vertical lines whilst simultaneously recording the 1920x1080 H264 "proxy" to the same card.  In fact, it's sustained speed did not waiver whether or not the H264 files were being written.

Lexar Professional 32GB 1000x UDMA 7 CF burst ~87MB/s with ~82MB/s sustained write speed @ 1920x1080.  It was the only card that could repeatedly sustain full 1080p for the duration of the 4GB size limit.

Delkin Devices 32GB 1000x UDMA 7 CF burst ~69MB/s with ~63MB/s sustained write speeds @ 1920x840.  It was the poorest performer with random dropped frames above 840 horizontal lines.  It would complete the duration of the 4GB size limit at 1920x840, but no higher.

None of these cards' write rates seemed affected by panning or tilting or complicated visual patterns in frame.

Global Draw was always on, but only for information overlays, all of the graphical overlays within frame were turned off.

The camera got warm, but not hot, and when the cards were pulled, they were likewise only warm.  Nowhere near as scalding as an SSD mag removed from a Blackmagic camera.  Never a temp or buffer warning from Canon firmware or Magic Lantern software.

In the next few days I will post a blog with test footage for those that are interested: http://shop.texasmediasystems.com/blog.asp

Thanks so much to the committed ML dev team and the forum members who contribute!

best,
-Colemar Nichols
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: arian_dd on May 14, 2013, 03:50:13 AM
Hi everyone. I am having difficulty installing the new rawrec update. Here is what I do:

1. Downloaded this zip file and unzipped and copied it to my SD card.
    https://docs.google.com/file/d/0B7QlH_BH2m32T3RSTXRUM1VXTWM/edit
2. Then I put the SD card back and in the menu I update the firmware like we do for other ML firmwares.
3. When I do this the screen goes black and the LED light keeps flashing and I can not turn the camera off unless I remove the battery.

I have a MkIII with the latest canon firmware 1.2.1.
Please let me know what I am doing wrong.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ernestoeduardo on May 14, 2013, 03:52:16 AM
Another note:

Sometimes RAW_rec module gets unloaded after the camera enters standby mode, even if autoload modules is enabled

Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 14, 2013, 03:53:51 AM
Quote from: arian_dd on May 14, 2013, 03:50:13 AM
Hi everyone. I am having difficulty installing the new rawrec update. Here is what I do:

1. Downloaded this zip file and unzipped and copied it to my SD card.
    https://docs.google.com/file/d/0B7QlH_BH2m32T3RSTXRUM1VXTWM/edit
2. Then I put the SD card back and in the menu I update the firmware like we do for other ML firmwares.
3. When I do this the screen goes black and the LED light keeps flashing and I can not turn the camera off unless I remove the battery.

I have a MkIII with the latest canon firmware 1.2.1.
Please let me know what I am doing wrong.

It does not work with firmware 1.2.1. Only for 1.1.3 for now.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 14, 2013, 03:54:33 AM
Quote from: arian_dd on May 14, 2013, 03:50:13 AM

I have a MkIII with the latest canon firmware 1.2.1.
Please let me know what I am doing wrong.

ML isn't compatible with the new Canon firmware.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 14, 2013, 03:59:32 AM
I'm doing a writeup for planet5D, can one of the devs give me a heads up on any plans to get around the 4Gb file size limit?

And if anyone has been able to open the raw files on a mac could they please throw us a bone, step by step for dummies please.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: arian_dd on May 14, 2013, 04:12:52 AM
Quote from: lourenco on May 14, 2013, 03:53:51 AM
It does not work with firmware 1.2.1. Only for 1.1.3 for now.

Thank you lourenco. I downgraded to 1.1.3 version and was able to install the new firmware. However when I try to load it up I get this message and it does not start.

EnableBootDisk
:)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: lourenco on May 14, 2013, 04:14:15 AM
Quote from: arian_dd on May 14, 2013, 04:12:52 AM
Thank you lourenco. I downgraded to 1.1.3 version and was able to install the new firmware. However when I try to load it up I get this message and it does not start.

EnableBootDisk
:)

Did you run eoscard? 
http://magiclantern.wikia.com/wiki/File:EOSCard_Tutorial
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: arian_dd on May 14, 2013, 04:22:43 AM
Quote from: lourenco on May 14, 2013, 04:14:15 AM
Did you run eoscard? 
http://magiclantern.wikia.com/wiki/File:EOSCard_Tutorial

No I was not using the EOSCard and for some reason I was able to get away with it while working with the previous versions.
Thank you so much. It is working now.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: RenatoPhoto on May 14, 2013, 04:42:25 AM
@ lourenco, Thanks for posting the ML files.  I finally got it working..

My test was simple: 5D3, 1000x Lexar 32 GB, 1920x1080 at (aprox) 24fps + 5X ZOOM

The result was continuous recording of RAW DIGITAL CROPPED VIDEO.  I have been wanting this feature on my cam since I found out it was available for the t3i. 

To get it press the zoom button when in live view and then start the raw recorder.  When in LiveView I had to use the joystick to move the center rectangle (thingy) to the left so that the cropped video would be centered.  It is beautiful!

Is there a way to set the fps besides FPS Override?
Can the shutter speed be set independently or will it be controlled by FPS Override?

It is like putting a 3X multiplier on the lens and not loosing any light!!

Good night to all, specially to the great ML team!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 14, 2013, 05:09:33 AM
There is a 4gb size limit? Or 4gb was set as a limit to not mess up fat 32?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 14, 2013, 05:13:03 AM
Quote from: 1% on May 14, 2013, 05:09:33 AM
There is a 4gb size limit? Or 4gb was set as a limit to not mess up fat 32?

Yeah it stops recording just over 4Gb which is a no go for narrative work.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 14, 2013, 05:33:53 AM
Its just set to stop recording at 4gb in the code.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 14, 2013, 05:37:42 AM
Quote from: 1% on May 14, 2013, 05:33:53 AM
Its just set to stop recording at 4gb in the code.

What does that mean? There is no actual limit then? We can keep recording?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: arian_dd on May 14, 2013, 05:42:08 AM
Has anyone tried the raw2dng program on a 64bit machine? How do you use this program? I tried to add an .exe extension to it and run it but it doesn't work.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 14, 2013, 05:42:55 AM
Quote from: bumkicho on May 14, 2013, 05:37:42 AM
What does that mean? There is no actual limit then? We can keep recording?

With a code change you can keep recording till your card fills up or your camera explodes, whichever comes first.
It's a good thing I wasn't really stoned when I saw that "your camera will explode" warning. A code freak with a sense of humour, who would have thought?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: 1% on May 14, 2013, 05:50:42 AM
I'm guessing this is so the files don't get too large since its testing stages and all that. H264 can be over 4gb on 600D.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 14, 2013, 05:56:45 AM
Loading may 14 build. The M tab where I can load the module is not there. Is the module shared by Coutts not yet included in this build?

Shall I still use the autoexec.bin with the raw_rec  for the 5d2 linked to by coutts earlier together with the module folder instead of this current build?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: noisyboy on May 14, 2013, 06:01:20 AM
Quote from: 1% on May 14, 2013, 05:50:42 AM
H264 can be over 4gb on 600D.

Oo! Does that mean you can keep recording h.264 and not have file size limitations?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 14, 2013, 06:02:11 AM
Quote from: 1% on May 14, 2013, 05:50:42 AM
I'm guessing this is so the files don't get too large since its testing stages and all that. H264 can be over 4gb on 600D.

Do you know if the menu button was programmed to stop recording or was it just a happy accident?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ThePhil on May 14, 2013, 06:11:55 AM
Squig,

I successfully captured a >4GB using a SanDisk Extreme Pro 90MB/s CF card in my ML Canon 5D Mark II.  Largest RAW Video File size as of now (that I've captured) is 4.21GB.

I'm converting it to DNG for edit/upload.


Phil
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 14, 2013, 06:13:44 AM
Quote from: ThePhil on May 14, 2013, 06:11:55 AM
Squig,

I successfully captured a >4GB using a SanDisk Extreme Pro 90MB/s CF card in my ML Canon 5D Mark II.  Largest RAW Video File size as of now (that I've captured) is 4.21GB.

I'm converting it to DNG for edit/upload.


Phil

I'm interested to know at what resolution and framerate? full hd 1920x1080 at 24fps?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 14, 2013, 06:14:32 AM
After playing around with raw recording on the 5D2 for a bit, it's great! The crop selections predict the resolutions the camera/CF card can handle pretty much perfectly. I can record 1880x720@24p continuously with my 600x sandisk card, which gave me 54-55 MB/s from the benchmark. Raising the resolution a tad would eventually cause the buffer to fill up and frameskipping to occur.

I'm still a little hesitant on getting a bigger and faster card since I don't know how well it will scale with this camera. We need something like 80+ MB/s to do 1880x1080 which is quite a ways away from 55. Anyway, just when I thought I couldn't be any more amazed by what ML enabled for our cameras, this beast of a feature goes out to the wild! Thank you everyone for making this happen!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ThePhil on May 14, 2013, 06:18:17 AM
Ted Ramasola,

Resolution:  1880x1080 at 23.976fps at 81.2 MB/s resulting in a 4.21GB file.


Phil
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ThePhil on May 14, 2013, 06:21:42 AM
Ted,

In regard to your other question, try the May 14 build. If it doesn't work with the provided Autoexec.bin swap for the autoexec.bin in the link Coutts provided earlier.


Phil
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 14, 2013, 06:26:13 AM
Thank you. for your reply ThePhil,

at 4.21 gig at that resolution how long is that in recording time? Did you still have to use FPS override to get that framerate?

I have tried the may 14 build. The M tab is gone so I replaced the autoexec.bin back to the one Coutts provided earlier. I'm just concerned if the updates and new commits made by a1ex as indicated in the changelog is not implemented.

I'm interested in the fix for pink frames indicated in the log. I always get it in the last frame of a take.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ThePhil on May 14, 2013, 06:40:15 AM
Ted,

I'm not advanced enough to say yes or no to the swap. I'd wait until one of the more knowledgable users posts in that regard.  However, in reference to the file size... I won't know for sure how well it performed until I get RAW2dng to work on my mac but I didn't have to use FPS override on it.  I monitored the fps during capture without a drop in frame rate.  I've found that turning off the camera and turning it back on right before recording helps clear the memory and allows for longer record times.  Disclaimer: I'm not an expert on the topic but theoretically the restart should clear the memory.


Phil
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 14, 2013, 06:46:22 AM
Thank you Phil,

Keep us posted on your test results. :)

Ted
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 14, 2013, 06:57:27 AM
4 gig should be roughly around a minute, give or take.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 07:07:43 AM
Quote from: squig on May 14, 2013, 03:44:53 AM
Join the mac lepper club.

;D ;D ;D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: NtiT on May 14, 2013, 07:40:26 AM
Anyone succeeded with raw2dng on 64bit Windows? Windows 8, if it makes a difference.
I get an "Unsupported 16-bit application error" message.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: ted ramasola on May 14, 2013, 07:42:21 AM
Quote from: NtiT on May 14, 2013, 07:40:26 AM
Anyone succeeded with raw2dng on 64bit Windows? Windows 8, if it makes a difference.
I get an "Unsupported 16-bit application error" message.

successful runing raw2dng in 64bit win 7. 
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 14, 2013, 07:44:06 AM
Quote from: NtiT on May 14, 2013, 07:40:26 AM
Anyone succeeded with raw2dng on 64bit Windows? Windows 8, if it makes a difference.
I get an "Unsupported 16-bit application error" message.

Works fine here. I also have win8 64-bit. It works by a simple drag-and-drop as people mentioned.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: NtiT on May 14, 2013, 07:49:34 AM
Quote from: Yoshiyuki Blade on May 14, 2013, 07:44:06 AM
Works fine here. I also have win8 64-bit. It works by a simple drag-and-drop as people mentioned.
I feel like an idiot, but can you run me through the entire process like I'm 5?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 14, 2013, 07:54:18 AM
Quote from: NtiT on May 14, 2013, 07:49:34 AM
I feel like an idiot, but can you run me through the entire process like I'm 5?

Click and hold the mouse button on the raw file (ex. M0000000.RAW), move it on the top of the raw2dng.exe icon and release the mouse button. It should start making .dng files immediately. Make sure you have both the program and raw video file in a folder and not your desktop since it'll produce hundreds of files.
Title: Re: uncompressed 14-bit RAW video recording
Post by: NtiT on May 14, 2013, 08:01:12 AM
To be clear, I'm using the raw2dng file from https://docs.google.com/file/d/0B7QlH_BH2m32T3RSTXRUM1VXTWM/edit (https://docs.google.com/file/d/0B7QlH_BH2m32T3RSTXRUM1VXTWM/edit), renamed with a .exe extension.
When I try to load a RAW file with CMD: http://i.imgur.com/aegEtle.png
Drag and Drop: http://i.imgur.com/I30EMqr.png
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 14, 2013, 08:02:29 AM
Quote from: NtiT on May 14, 2013, 08:01:12 AM
To be clear, I'm using the raw2dng file from https://docs.google.com/file/d/0B7QlH_BH2m32T3RSTXRUM1VXTWM/edit (https://docs.google.com/file/d/0B7QlH_BH2m32T3RSTXRUM1VXTWM/edit), renamed with a .exe extension.
When I try to load a RAW file with CMD: http://i.imgur.com/aegEtle.png
Drag and Drop: http://i.imgur.com/I30EMqr.png

Try the one alex posted here: http://www.magiclantern.fm/forum/index.php?topic=5247.msg34132#msg34132
Title: Re: uncompressed 14-bit RAW video recording
Post by: NtiT on May 14, 2013, 08:08:20 AM
Quote from: Yoshiyuki Blade on May 14, 2013, 08:02:29 AM
Try the one alex posted here: http://www.magiclantern.fm/forum/index.php?topic=5247.msg34132#msg34132
Thanks! Worked like a charm.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 08:34:41 AM
Quote from: Yoshiyuki Blade on May 14, 2013, 07:54:18 AM
Click and hold the mouse button on the raw file (ex. M0000000.RAW), move it on the top of the raw2dng.exe icon and release the mouse button. It should start making .dng files immediately. Make sure you have both the program and raw video file in a folder and not your desktop since it'll produce hundreds of files.
I wish it were this simple for mac users :sigh:
Title: Re: uncompressed 14-bit RAW video recording
Post by: zachnfine on May 14, 2013, 08:35:57 AM
Two questions:

1. I see a decent number of samples on the net of the new feature recorded using Technicolor Cinestyle. Is there a better S-log profile to use with 14-bit raw than Cinestyle, or is a style even necessary with raw? Isn't Cinestyle designed specifically to compress both the shadows and the highlights within the range of exposure that would otherwise be truncated by h264 compression -- and now that we're skipping h264 altogether we can take advantage of that extra space in which to record shadows and highlights? Cinestyle may be compressing the range more than necessary, it may be a waste to use it with raw.

2. There are a bunch of indirect warnings from Canon engineers popping up in this twitter feed (note: Mr. Sutton is not a troll, and is undoubtedly trying to pass along what he deems useful info from the source, so please give him the benefit of the doubt):
https://twitter.com/mns1974/status/334132774132908032
I'm skeptical, as I would expect that there'd be safety shutdown procedures in the firmware on the sensor that don't depend on the proper behavior of applications running on the camera's OS, but who knows. I guess we'll all know within a couple of weeks. I've ordered my 1000X 32Gb CF card, and will definitely be shooting some raw in the near future unless peoples' cameras start exploding.

Thanks so much A1ex and all Magic Lantern contributors. As if magic zoom wasn't enough!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Danne on May 14, 2013, 08:47:48 AM
Quote from: aaphotog on May 14, 2013, 08:34:41 AM
I wish it were this simple for mac users :sigh:

Here comes another mac leper;).
I,m on a mac OS X 10.6.8 and I can,t get wine bottler workwith the exe. I also can,t get the script to work in terminal. Don,t really know what I,m doing here. Is there any smart solution how to make the conversion an a mac and that doesn,t involve complicated programming instructions?
Thanks everyone
//D
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 08:48:58 AM
Quote from: Danne on May 14, 2013, 08:47:48 AM
Here comes another mac lepper;).
I,m on a mac OS X 10.6.8 and I can,t get wine bottler workwith the exe. I also can,t get the script to work in terminal. Don,t really know what I,m doing here. Is there any smart solution how to make the conversion an a mac and that doesn,t involve complicated programming instructions?
Thanks everyone
//D

lol what is a lepper?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Danne on May 14, 2013, 08:54:38 AM
Quote from: aaphotog on May 14, 2013, 08:48:58 AM
lol what is a lepper?

I spelled it wrong, It,s leper :)
Well, It,s not a good thing ;)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 09:02:06 AM
Quote from: Danne on May 14, 2013, 08:54:38 AM
I spelled it wrong, It,s leper :)
Well, It,s not a good thing ;)
well, if it means "fool that doesn't understand coding"... that's me!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 09:14:59 AM
You said you got the conversion to work by right clicking on your mac and choosing "open with" and opened the script. Did that work for you or not? When I tried everything was greyed out except for programs on my mac.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 09:15:38 AM
Quote from: zachnfine on May 14, 2013, 08:35:57 AM
2. There are a bunch of indirect warnings from Canon engineers popping up in this twitter feed (note: Mr. Sutton is not a troll, and is undoubtedly trying to pass along what he deems useful info from the source, so please give him the benefit of the doubt):
https://twitter.com/mns1974/status/334132774132908032

Sounds like bullshit to me. What does the raw_rec have to do with the sensor? My knowledge is pretty limited in this area but my understanding is that this patch is just recording a data feed from the sensor that was always there, someone correct me if I'm wrong.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mcnys on May 14, 2013, 09:16:26 AM
so rebel line and 60d is limited to 21MBps, is it software limitation(canon firmware or something) or hardware limitation?
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 09:18:39 AM
Quote from: Danne on May 14, 2013, 09:14:59 AM
You said you got the conversion to work by right clicking on your mac and choosing "open with" and opened the script. Did that work for you or not? When I tried everything was greyed out except for programs on my mac.

I tried it on lion, it wasn't greyed out but didn't work. I installed 10.8 and it was greyed out  :'(
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 09:25:42 AM
Quote from: squig on May 14, 2013, 09:18:39 AM
I tried it on lion, it wasn't greyed out but didn't work. I installed 10.8 and it was greyed out  :'(
Hi Squig. You actually installed another operative only to find out it was greyed out? Bummer :). Did anyone get the mac-conversion to work really? I have another question. I ran my 64gb kingston 600x card through eoscard and loaded lourencos compilation. Everything worked fine. Do I just format the card to erase magic lantern when I want to get back to original camera firmware?
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 09:28:00 AM
Someone tried to compile on mac the raw2dng tool?
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 14, 2013, 09:29:46 AM
Quote from: zachnfine on May 14, 2013, 08:35:57 AM
Two questions:

1. I see a decent number of samples on the net of the new feature recorded using Technicolor Cinestyle. Is there a better S-log profile to use with 14-bit raw than Cinestyle, or is a style even necessary with raw? Isn't Cinestyle designed specifically to compress both the shadows and the highlights within the range of exposure that would otherwise be truncated by h264 compression -- and now that we're skipping h264 altogether we can take advantage of that extra space in which to record shadows and highlights? Cinestyle may be compressing the range more than necessary, it may be a waste to use it with raw.
there is *no* need for any style to apply in camera.
its the same like picture style for JPEGs - no need if you shoot raw.

Quote from: zachnfine on May 14, 2013, 08:35:57 AM
2. There are a bunch of indirect warnings from Canon engineers popping up in this twitter feed (note: Mr. Sutton is not a troll, and is undoubtedly trying to pass along what he deems useful info from the source, so please give him the benefit of the doubt):
https://twitter.com/mns1974/status/334132774132908032
I'm skeptical, as I would expect that there'd be safety shutdown procedures in the firmware on the sensor that don't depend on the proper behavior of applications running on the camera's OS, but who knows. I guess we'll all know within a couple of weeks. I've ordered my 1000X 32Gb CF card, and will definitely be shooting some raw in the near future unless peoples' cameras start exploding.

Thanks so much A1ex and all Magic Lantern contributors. As if magic zoom wasn't enough!

this warning is fake. there is *no* impact on sensor power consumption and thus no impact on its temperature.
the only thing that could get warmer is DIGiC and the CF circuitry, but i am sure that the power dissipation that reaches the sensor
through all that plastic housing will not have any noticeable temperature raise.

detailed: when doing that much DMA transfers and CF writing, we may cause a bit more current drain (which causes squared power dissipation)
but we do not encode any H.264 while recording, so we also use less power in total.
its *possible* that the CF writing will consume less energy than the encoding with H.264, which will result in *less* power consumption.
raw is being produced by the DIGiC for every single frame anyway. we "just" save it away.

still this is a *theory*, but i expect the consumption and the temperatures not to raise at all.
bitrate hack on 7D in contrast did both: massive CF access and H264 encoding, but noone who calls himself one that is "talking with canon engineers on a regular basis" cried about sensor temperature there :)

people who have no clue what is happening in the circuitry are telling us what can go wrong?
so yes, i call it bullshitting.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 09:29:57 AM
Quote from: Danne on May 14, 2013, 09:25:42 AM
Hi Squig. You actually installed another operative only to find out it was greyed out? Bummer :). Did anyone get the mac-conversion to work really? I have another question. I ran my 64gb kingston 600x card through eoscard and loaded lourencos compilation. Everything worked fine. Do I just format the card to erase magic lantern when I want to get back to original camera firmware?

Yes, yes, and yes.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 09:34:39 AM
Quote from: squig on May 14, 2013, 09:29:57 AM
Yes, yes, and yes.
Thanks!
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 09:35:47 AM
Quote from: scrax on May 14, 2013, 09:28:00 AM
Someone tried to compile on mac the raw2dng tool?

They did but they didn't explain to us mere mortal non command line literate types how the fuc# to use it.  ::)

If anything the MKIII feels a bit cooler.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: itux on May 14, 2013, 09:43:59 AM
Quote from: aaphotog on May 14, 2013, 08:34:41 AM
I wish it were this simple for mac users :sigh:

1. Download raw2dng for Mac from here:

Quote from: kidb on May 13, 2013, 11:12:09 PM
http://kidb.tv/.ml/

2. Put raw2dng into the same folder where your raw file is. (for example a folder called "raw" on your desktop)
3. Open Terminal.
4. Go to the folder (cd /Users/username/Desktop/raw)
5. Launch raw2dng (./raw2dng RAWFILE.RAW)

This will extract the file into the separate DNGs.

I hope this helps!
Title: Re: uncompressed 14-bit RAW video recording
Post by: X-RAY on May 14, 2013, 09:50:24 AM
Very nice guys.
Now, I can be proud again on having a Canon-DSLR ... but those words surely aren't ment for the canon engineers but for you fellows. :-)
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Danne on May 14, 2013, 09:52:43 AM
Quote from: itux on May 14, 2013, 09:43:59 AM
1. Download raw2dng for Mac from here:

2. Put raw2dng into the same folder where your raw file is. (for example a folder called "raw" on your desktop)
3. Open Terminal.
4. Go to the folder (cd /Users/username/Desktop/raw)
5. Launch raw2dng (./raw2dng RAWFILE.RAW)

This will extract the file into the separate DNGs.

I hope this helps!



Hi! WHen I follow your steps this is what happens in the terminal. I have the script and raw in the same folder. Something about Illegal instruction??

"Last login: Tue May 14 10:17:19 on ttys000
itgusers-MacBook-241:~ itguser$ /Users/itguser/Desktop/raw/raw2dng ; exit;
Illegal instruction
logout

[Processen slutförd]"
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 09:56:21 AM
I've no raw files to test it, here a maybe more easy to use app for mac: https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dng.app.zip

drag your raw file on the MLicon and it should start conversion

EDIT: removed link until not fixedfixed now dng saved in same folder of raw file
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: scrax on May 14, 2013, 09:57:17 AM
Quote from: Danne on May 14, 2013, 09:52:43 AM
Hi! WHen I follow your steps this is what happens in the terminal. I have the script and raw in the same folder. Something about Illegal instruction??

"Last login: Tue May 14 10:17:19 on ttys000
itgusers-MacBook-241:~ itguser$ /Users/itguser/Desktop/raw/raw2dng ; exit;
Illegal instruction
logout

[Processen slutförd]"

you are missing the point
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 10:01:06 AM
Quote from: scrax on May 14, 2013, 09:56:21 AM
I've no raw files to test it, here a maybe more easy to use app for mac: https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dng.app.zip

drag your raw file on the MLicon and it should start conversion

CLose but noe cegar. I did the drag and dropbut this pops up "Error: /Users/itguser/Desktop/Ny". Maybe because I,m on Mac OS X 10.6.8 and not MLion?
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 10:03:18 AM
Quote from: Danne on May 14, 2013, 09:14:59 AM
You said you got the conversion to work by right clicking on your mac and choosing "open with" and opened the script. Did that work for you or not? When I tried everything was greyed out except for programs on my mac.
It did work. But I lucked up on it. I had installed a bunch of programs and downloaded all types of different raw2dng exe's and some without exe extenstion. When my procedure worked for me, I deleted the software I downloaded thinking that it was as simple as I made it seem... now, it doesn't work when doing that :-(
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 10:03:25 AM
can you post a screen like this:
(http://snapplr.com/snap/x2ak)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 10:05:42 AM
Wait, It worked!!!
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 10:07:48 AM
Quote from: itux on May 14, 2013, 09:43:59 AM
1. Download raw2dng for Mac from here:

2. Put raw2dng into the same folder where your raw file is. (for example a folder called "raw" on your desktop)
3. Open Terminal.
4. Go to the folder (cd /Users/username/Desktop/raw)
5. Launch raw2dng (./raw2dng RAWFILE.RAW)

This will extract the file into the separate DNGs.

I hope this helps!
I just KNEW that was going to work. Everything was fine until step 5. When I tried to launch raw2dng, it said
"-bash: ./raw2dng: Permission denied"
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 10:10:46 AM
Quote from: scrax on May 14, 2013, 09:56:21 AM
I've no raw files to test it, here a maybe more easy to use app for mac: https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dng.app.zip

drag your raw file on the MLicon and it should start conversion

OOOOHHHHHHHHHH SHIT!
It WORKED!!!!!!
THANKS!
Thanks to the devs, thanks to you scrax, and thanks to the others patient enough to continuously trying to help me. It's truly appreciated
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 10:11:36 AM
Probably the .app is saving the dng inside the .app package itself. Someone can confirm me that? To check that right click; select Show package contents and go to the Resources folder
(http://snapplr.com/snap/0crj)

I will fix it but someone can tell me where the dng are saved now? in the same folder of the .rawfile or it the Resources folder in the package (.app)
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 10:15:40 AM
Quote from: aaphotog on May 14, 2013, 10:10:46 AM
OOOOHHHHHHHHHH SHIT!
It WORKED!!!!!!
THANKS!
Thanks to the devs, thanks to you scrax, and thanks to the others patient enough to continuously trying to help me. It's truly appreciated

Damn! it didn't work. It looked like it was working, but when it was all said and done, there were no DNG's created in the folder.
I think theres something wrong with MY specific computer. or am I really messing this up royally
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 10:16:40 AM
Quote from: aaphotog on May 14, 2013, 10:15:40 AM
...there were no DNG's created in the folder.
which folder?

EDIT: removed download link until not fixed the .app
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: squig on May 14, 2013, 10:17:20 AM
Quote from: itux on May 14, 2013, 09:43:59 AM
1. Download raw2dng for Mac from here:

2. Put raw2dng into the same folder where your raw file is. (for example a folder called "raw" on your desktop)
3. Open Terminal.
4. Go to the folder (cd /Users/username/Desktop/raw)
5. Launch raw2dng (./raw2dng RAWFILE.RAW)

This will extract the file into the separate DNGs.

I hope this helps!

Danke   ;D
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 10:17:46 AM
Quote from: scrax on May 14, 2013, 10:11:36 AM
Probably the .app is saving the dng inside the .app package itself. Someone can confirm me that? To check that right click; select Show package contents and go to the Resources folder
(http://snapplr.com/snap/0crj)

I will fix it but someone can tell me where the dng are saved now? in the same folder of the .rawfile or it the Resources folder in the package (.app)

They are saved in the Resources folder.SWEET!!!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 10:19:56 AM
Thanks Scrax! I you could find a way to get the files saved outside the app that would be awesome :)

Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 10:20:01 AM
Ok will fix the script to move them out in the same .raw file folder, someone has a link to a downloadable .raw file to test the conversion?

Quote from: Danne on May 14, 2013, 10:19:56 AM
Thanks Scrax! I you could find a way to get the files saved outside the app that would be awesome :)

Will post a fix ASAP, but maybe better if I can test it locally before  :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 10:20:38 AM
(http://i43.tinypic.com/35d2b7c.png)
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 10:21:32 AM
Quote from: scrax on May 14, 2013, 10:16:40 AM
which folder?

EDIT: removed download link until not fixed the .app
I placed the app that you created, and my raw file in the same folder. I had assumed that the DNG's would just pop up in that folder
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 10:23:41 AM
Quote from: Danne on May 14, 2013, 10:17:46 AM
They are saved in the Resources folder.SWEET!!!
hahaha they are in there lol.
thx! Guess it did work
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 10:24:54 AM
Quote from: aaphotog on May 14, 2013, 10:21:32 AM
I placed the app that you created, and my raw file in the same folder. I had assumed that the DNG's would just pop up in that folder
You assumed right ;) but I didn't thinked ablut this when doing the package, so I have to fix it, for now look inside the .app package with right click and you should fond the .dng like Danne
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 10:26:04 AM
Awesome! I,ll post a short raw-file for you scrax, in a moment!
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 10:26:49 AM
Quote from: aaphotog on May 14, 2013, 10:23:41 AM
hahaha they are in there lol.
thx! Guess it did work
Quote from: Danne on May 14, 2013, 10:26:04 AM
Awesome! I,ll post a short raw-file for you scrax, in a moment!
Perfect, just a sec should be enough, I'll post the link again when fixed and tested.
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 10:28:09 AM
I really wish I knew coding now. you all are REALLY DOING SOME AMAAAAAAAZING shit in such a short amount of time.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 10:30:52 AM
It's like being back in kindergarten.

I honestly don't know how you guys do this, I thought working with actors was torture.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Lcrusher on May 14, 2013, 10:33:32 AM
Has anyone tried chroma-keying?
Title: Re: uncompressed 14-bit RAW video recording
Post by: actioncamstudio on May 14, 2013, 10:37:46 AM
Hallo and sorry for my really super dummy question and for my bad english!!!
It will be possible to use this raw super cool function on the MKII someday? I have a bit of problems with english but i've read only of the MKIII,6D and 600D.
Again,sorry for the stupid question,but if there is a code to download i will be happy to test it...if some good guy will explain to me how to do it!
Bests from Italy and keep it up,this community is FANTASTIC!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 10:43:11 AM
Quote from: actioncamstudio on May 14, 2013, 10:37:46 AM
Hallo and sorry for my really super dummy question and for my bad english!!!
It will be possible to use this raw super cool function on the MKII someday? I have a bit of problems with english but i've read only of the MKIII,6D and 600D.
Again,sorry for the stupid question,but if there is a code to download i will be happy to test it...if some good guy will explain to me how to do it!
Bests from Italy and keep it up,this community is FANTASTIC!

Go back a page or two,they are on it. Looks as if it will work on mark 2 aswell.
Tried it on my mark 3 right now. IT. IS. CRAZY GOOD!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 10:45:43 AM
Quote from: scrax on May 14, 2013, 10:26:49 AM
Perfect, just a sec should be enough, I'll post the link again when fixed and tested.

Here,s a link Scrax to a short file :). Again, biggest thanks!

https://docs.google.com/file/d/0B4tCJMlOYfirdWdIdjlxSDd3cms/edit?usp=sharing

//Daniel
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 14, 2013, 10:53:35 AM
I assume someone had already thought about this or at least asked a question like this, but just in case, with the new 5D3 firmware, whenever it becomes compatible with ML, would it be possible to feed this raw directly to Ninja2? I wonder if atomos will look into it also unless there is legal issues with canon.
Title: Re: uncompressed 14-bit RAW video recording
Post by: actioncamstudio on May 14, 2013, 11:00:07 AM
Thank you so much Danne!
I will check the previous pages. Damn,there are too many entries in this thread!  ;D ;D ;D
Really fantastic!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 14, 2013, 11:01:21 AM
I just pooped my pants 5 times already. It,s that good  ;D
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 11:03:39 AM
Quote from: Danne on May 14, 2013, 10:45:43 AM
Here,s a link Scrax to a short file :). Again, biggest thanks!

https://docs.google.com/file/d/0B4tCJMlOYfirdWdIdjlxSDd3cms/edit?usp=sharing

//Daniel

Perfect, downloaded and tested with raw2dng.app if the video was all black it worked good.
It converts the file inside the app and then moves all the dng to the same folder where the .raw file is.
Best would be to convert file directly to the same folder of the raw file, will try this later.

Link up again: https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dng.app.zip
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 11:03:49 AM
Quote from: bumkicho on May 14, 2013, 10:53:35 AM
I assume someone had already thought about this or at least asked a question like this, but just in case, with the new 5D3 firmware, whenever it becomes compatible with ML, would it be possible to feed this raw directly to Ninja2? I wonder if atomos will look into it also unless there is legal issues with canon.

nope.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: scrax on May 14, 2013, 11:06:40 AM
Quote from: aaphotog on May 14, 2013, 10:07:48 AM
I just KNEW that was going to work. Everything was fine until step 5. When I tried to launch raw2dng, it said
"-bash: ./raw2dng: Permission denied"
chmod +x raw2dng will make it executable
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: Redrocks on May 14, 2013, 11:08:50 AM
Quote from: coutts on May 14, 2013, 12:49:27 AM
This should work for 5d2, double buffering enabled:
http://bit.ly/10TBM7a

extract the MODULES directory into /ML/ on your card

Downloaded this and installed as per instructed, keep getting a message on power up "Scripts dir missing" and WAS getting another about an improper power off, but that seems to have rectified itself...

http://www.youtube.com/watch?v=7SeH9LVVpJg

Just grabbed a few frames at 1880 x 720 (the highest I can go with my 8GB 30MB/s Sandisk) and a h.264 for comparison, neither has had any kind of work done. For some reason the RAW files come into AE at 30 FPS even though I'd had FPS set to 24 in camera and I have no idea if I dropped any frames.

I'm still finding my way with ML in general and have zero to offer in terms of coding, but I'm happy to be a lab rat and I'll continue testing in between slamming F5 and pinching myself. A month ago I was hoping you guys would crack the HDMI out and was actually looking forward to shelling out for a 10 bit recorder and a V mount power solution and cables and and and... now I only have to wait for my 1000x cards to drop through the door.

Thank you to all who have contributed to this amazing community. Expect a contribution from me when you reopen the donations.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kabuto1138 on May 14, 2013, 11:09:32 AM
It works Scrax!!  thanks!
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 11:12:37 AM
Quote from: Kabuto1138 on May 14, 2013, 11:09:32 AM
It works Scrax!!  thanks!

Good, thinking about adding it to MLTools too.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 11:13:47 AM
perfect! thx scrax
Title: Re: uncompressed 14-bit RAW video recording
Post by: ch_d on May 14, 2013, 11:17:43 AM
Quote from: scrax on May 14, 2013, 11:03:39 AM

Link up again: https://bitbucket.org/600dplus/magic-lantern-things/downloads/raw2dng.app.zip

Thanks - works perfectly!
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 11:18:13 AM
Just one workflow suggestion though, when the DNGs go into a folder with the original RAW file you can't just "select all" when you import them into AE.
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 11:19:44 AM
Quote from: squig on May 14, 2013, 11:18:13 AM
Just one workflow suggestion though, when the DNGs go into a folder with the original RAW file you can't just "select all" when you import them into AE.
so better make a dng subfolder in the raw folder?
we can use the time of conversion to have a unique name for the subfolder
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 11:22:55 AM
Yeah a subfolder would be great, thx
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 11:35:47 AM
Quote from: squig on May 14, 2013, 11:22:55 AM
Yeah a subfolder would be great, thx
Updated the zip in the link, redownload it
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 14, 2013, 11:42:52 AM
moved here: http://www.magiclantern.fm/forum/index.php?topic=5404.msg34761#msg34761
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 14, 2013, 11:46:09 AM
STOP

All questions regarding RAW VIDEO POST PROCESSING on any OS
HERE: http://www.magiclantern.fm/forum/index.php?topic=5404.0

All questions regarding RAW_REC MODULE COMPILATION/INSTALLATION/USAGE
HERE: http://www.magiclantern.fm/forum/index.php?topic=5405.0

And this thread here should from now on just be used for video results, and open discussion.
NOT for any installation or processing help.

thanks.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 11:51:06 AM
Quote from: scrax on May 14, 2013, 11:35:47 AM
Updated the zip in the link, redownload it

Cool thx, all good.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 14, 2013, 12:00:49 PM
Quote from: squig on May 14, 2013, 11:03:49 AM
nope.

It's no because?
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: bumkicho on May 14, 2013, 12:03:51 PM
Quote from: 1% on May 14, 2013, 05:33:53 AM
Its just set to stop recording at 4gb in the code.

Is this precautionary limit in the code? Will this be removed later?
Title: Re: uncompressed 14-bit RAW video recording
Post by: zachnfine on May 14, 2013, 12:05:37 PM
Quote from: g3gg0 on May 14, 2013, 09:29:46 AM
still this is a *theory*, but i expect the consumption and the temperatures not to raise at all.
bitrate hack on 7D in contrast did both: massive CF access and H264 encoding, but noone who calls himself one that is "talking with canon engineers on a regular basis" cried about sensor temperature there :)

people who have no clue what is happening in the circuitry are telling us what can go wrong?
so yes, i call it bullshitting.

That'd be my guess, but speaking as someone who has no knowledge of how the sensor actually operates, I wonder if setting it to record to the live-view buffer at high resolutions may possibly heat the sensor more than is normal during recording (given that normal h264 recording is probably only using every third line of the sensor at some absurdly low res, or some such thing).

I figured I'd just pass long these reservations just to see how easily they can be knocked down by people who understand how the sensor and camera actually operate. I don't myself expect that heat will be an insurmountable problem for the sensor, and as I've said, I've cast my lot by buying a 1000x CF card.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 12:07:15 PM
Quote from: bumkicho on May 14, 2013, 12:00:49 PM
It's no because?

For starters the HDMI spec is 8bit 422, this is 14bit RAW 4444. The ninja only does 442 too.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bryanfieldhouse on May 14, 2013, 12:46:41 PM
Quote from: squig on May 14, 2013, 12:07:15 PM
For starters the HDMI spec is 8bit 422, this is 14bit RAW 4444. The ninja only does 442 too.

http://en.wikipedia.org/wiki/HDMI

"To ensure baseline compatibility between different HDMI sources and displays (as well as backward compatibility with the electrically compatible DVI standard) all HDMI devices must support the sRGB color space at 8 bits per component.[51] Support for the YCbCr color space and higher color depths ("deep color") is optional. HDMI permits sRGB 4:4:4 chroma subsampling (8–16 bits per component), xvYCC 4:4:4 chroma subsampling (8–16 bits per component), YCbCr 4:4:4 chroma subsampling (8–16 bits per component), or YCbCr 4:2:2 chroma subsampling (8–12 bits per component).[52][53] The color spaces that can be used by HDMI are ITU-R BT.601, ITU-R BT.709-5 and IEC 61966-2-4.[52]"

Atomos should send Ninja2's to each of the developers... :)

I'm crossing my fingers that it's eventually possible, as it would reduce the pain/expense of working with multiple cards..

Either way, this is all very exciting :-)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Lcrusher on May 14, 2013, 12:51:27 PM
No. HDMI only supports the transfer of debayered information. RAW is the data from the sensor that is not yet debayered. The process of debayering is CPU intensive.
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 14, 2013, 12:54:20 PM
the main problem is the fact that the HDMI hardware module does some colorspace transformation on its own.
we cannot simply send our data as we want it and we dont know much about the configuration of the HDMI controller.

we are aware of possibilities, but we are happy to present what we have right now. :)
but thanks for providing alternative theories :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: bryanfieldhouse on May 14, 2013, 01:02:58 PM
Quote from: g3gg0 on May 14, 2013, 12:54:20 PM
the main problem is the fact that the HDMI hardware module does some colorspace transformation on its own.
we cannot simply send our data as we want it and we dont know much about the configuration of the HDMI controller.

we are aware of possibilities, but we are happy to present what we have right now. :)
but thanks for providing alternative theories :)

Understood, however even 8bit 2.2.2 (with the added vertical resolution from the ML hack) would be better than the mushy video via HDMI at present..

I appreciate it's not just as simple as flicking a switch :) I just feel, given the input from someone knowledgeable in the HDMI controller possibilities could become realities :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Lcrusher on May 14, 2013, 01:07:12 PM
Take a look at this.

http://www.globalsources.com/gsol/I/Disk-on/p/sm/1056225379.htm

Title: Re: uncompressed 14-bit RAW video recording
Post by: bryanfieldhouse on May 14, 2013, 01:16:33 PM
Quote from: Lcrusher on May 14, 2013, 01:07:12 PM
Take a look at this.

http://www.globalsources.com/gsol/I/Disk-on/p/sm/1056225379.htm

:)

That looks interesting (dimensionally too wide), plus the MKIII does not power up with the card flap open..

interesting times...

It would have to be some kind of port extension device that plugs into the compact flash socket on the MKIII and a cable (You'd have to drill a hole in the flap) that then plugs into the drive..

I'm sure it would be doable..


EDIT: Something like this could be modded to work...

http://www.amazon.co.uk/compact-flash-SATA-adapter-COMPLIANT/dp/B0039O6WK6

Title: Re: uncompressed 14-bit RAW video recording
Post by: Redrocks on May 14, 2013, 01:27:37 PM
Quote from: Lcrusher on May 14, 2013, 01:07:12 PM
Take a look at this.

http://www.globalsources.com/gsol/I/Disk-on/p/sm/1056225379.htm

I was looking into this the other day.

http://www.waeplus.co.uk/item/186209/Socket-EA2917-722-Fast-Ethernet-Card---C

This particular model isn't fast enough and the card door will need tricking, plus you need another device to store the data... but it's another possible solution.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Lcrusher on May 14, 2013, 01:29:55 PM
That model doesn't meet the needs. But the concept still stands :)
Title: Re: uncompressed 14-bit RAW video recording - Comparison and Working Proof
Post by: zmelms on May 14, 2013, 01:47:54 PM
The announcement about RAW recording on the 5dMK3 intrigued me so much that I finally picked one up.  I've been on the fence about the camera as I'm NOT a fan of the DSLR video look and frankly it's a tool I usually avoid.  However the idea of such improved image quality made it interesting once again.  After today's experience the RAW functionality will be useful for specific shots while the remainder of a project could be captured using the standard video function.

Here's the video with the side by side comparison's that are imho more than convincing, even with limitations this makes DSLR film making and footage gathering more interesting than ever.



IMAGES PROCESSED on a PC using raw2dng.exe
then IMPORTED on A MAC to CS6 After Effects
Edited in Premiere on a MAC - it's time consuming but the results are beautiful

CAMERA SETTINGS: Technicolor CineStyle was enabled for all shooting, everything else was default.
POST: the h.264's are untouched, the RAW files were processed minimally to ensure the highlights/shadows weren't clipping for maximum dynamic range.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Stedda on May 14, 2013, 01:53:40 PM
Awesome work guys! It works great!

lourenco, thanks for sharing your compiled files. I tried for 3 days to compile my own but just cant get it to work.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ch_d on May 14, 2013, 01:58:32 PM
Quote from: g3gg0 on May 14, 2013, 11:46:09 AM
STOP

And this thread here should from now on just be used for video results, and open discussion.
NOT for any installation or processing help.

thanks.


(watch HD fullscreen - or download it)

I am here on a 5DMII with the latest 5d2.doublebuffer on a SanDisk 60Mb/s (terrible, but waiting for a faster card ... there are framedrops - maybe because the slow card).
Just to show you folks how awesome the AA Filter worx.
Best show: Download the file!
The video is not sharpened (RAW settings to 0) just quick adjustments on the highlights and shadows.
I´ve transcoded the file with x264 10.000kBit/s (very slow).

Nice work guys! Keep´em coming!  8)
Title: Re: uncompressed 14-bit RAW video recording
Post by: mndrn on May 14, 2013, 01:59:34 PM
Title: Re: uncompressed 14-bit RAW video recording
Post by: t2scorp on May 14, 2013, 02:31:46 PM

Raw Video on the Canon 5D Mark III (Magic Lantern) - "Smith Rock"
From: Neumannsfilm
Title: Re: uncompressed 14-bit RAW video recording
Post by: olik on May 14, 2013, 02:45:18 PM
I did my first tests with the http://bit.ly/10TBM7a 5dmk2 double buffer from coutts version. I have 90mbit/sec sandisc cards (about 600x). Only with 1880x720/25fps and global draw off can I record without dropped frames. Is the limitation the CF card or the 5dmk2 hardware (buffer?).

Great work folks!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Lcrusher on May 14, 2013, 02:45:43 PM
It there a way to write EXIF data to the RAW files? Would be helpful for lens distortion and vignetting correction in LR.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 14, 2013, 03:05:43 PM
Quote from: olik on May 14, 2013, 02:45:18 PM
I did my first tests with the http://bit.ly/10TBM7a 5dmk2 double buffer from coutts version. I have 90mbit/sec sandisc cards (about 600x). Only with 1880x720/25fps and global draw off can I record without dropped frames. Is the limitation the CF card or the 5dmk2 hardware (buffer?).

Great work folks!

My card's the same brand/speed and my limits are the same, which tops out at approx 55 MB/s. Another user posted a benchmark with a 1000x CF card but it topped out only around 65 MB/s. Overall, I'm not sure what the 5D2 is still capable of and would like to see if anyone else managed to get higher rates somehow.
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 14, 2013, 03:08:45 PM
Is there a way to set the fps besides FPS Override?
Can the shutter speed be set independently or will it be controlled by FPS Override?
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 14, 2013, 03:30:38 PM
Ran some tests on the 600D raw capabilities, definitely not going to get anywhere near the mkiii capabilities unless a miracle happens but 1280x360 is definitely usable and runs continuously, and 1280x480 records a solid 5-6 secs of great footage until frames skip or corrupt.

Going to try and post a video tomorrow. Hmm, may be time to retire the 600D and splurge on a mkiii?

The entire ML team is way ahead of their time, huge thank you to all of 'em. I think you guys just revolutionized the entire cinematography world.

8)

Title: Re: uncompressed 14-bit RAW video recording
Post by: JackDaniel412 on May 14, 2013, 03:43:26 PM
Continuous RAW video on 5D Mark II res 1880x720 without frame skipping (stopped automatically on 3.9 GB), CF SanDisk 16gb Extreme PRO 90 MB/s.
Great job ML TEAM!
Title: Re: uncompressed 14-bit RAW video recording
Post by: cinema5D on May 14, 2013, 04:21:58 PM
We've got our first video up and will post the BMCC comparison later:
(http://www.cinema5d.com/seb/vimeo.jpg) (https://vimeo.com/66153520)

CHEERS
Article to the video: http://c5d.at/1o5
Title: Re: uncompressed 14-bit RAW video recording
Post by: Redrocks on May 14, 2013, 04:22:40 PM
Just took some more shots which I'm transferring at the moment and during the session the 5d2 crashed.  It's no big deal and the camera was fine, but a bunch of text documents appeared in the root (ASSERT00 to 11, log000 and CRASH00) and I was told to inform the devs. I'll upload the log000 file if it's needed, here is the text in the crash document:

ASSERT: 0
at Stub.c:29, task MovieRecorder
lv:0 mode:3


Magic Lantern version : v2.3.NEXT.2013May13.5D2212
Mercurial changeset   : 84f657f209eb+ (unified) tip
Built on 2013-05-13 21:04:58 by [email protected].
Free Memory  : 126K + 3992K


Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 14, 2013, 04:25:53 PM
Video Mode vs photo mode? Any speed difference?
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 14, 2013, 05:20:10 PM
Quote from: 1% on May 14, 2013, 04:25:53 PM
Video Mode vs photo mode? Any speed difference?

I noticed in photo mode the speed/frame rate indicator at the bottom of the menus fluctuates?
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 14, 2013, 05:27:09 PM
By how much?
Title: Re: uncompressed 14-bit RAW video recording
Post by: ajay on May 14, 2013, 05:33:17 PM
Tried some tests this morning with the 5DM3. I am noticing vertical banding in blurred backgrounds. They appear on the dng files. Anyone else seeing this?

V2.3.NEXT.2013May14.5D3113, Build:2013-0514 00:26:08

AJ
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 14, 2013, 05:34:25 PM
Quick and dirty Taxi Driver test.

Title: Re: uncompressed 14-bit RAW video recording
Post by: Redrocks on May 14, 2013, 05:45:00 PM
http://youtu.be/TPUylKvTXn0

1880 x 1152 vs h.264.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 14, 2013, 05:49:37 PM
Quote from: ajay on May 14, 2013, 05:33:17 PM
Tried some tests this morning with the 5DM3. I am noticing vertical banding in blurred backgrounds. They appear on the dng files. Anyone else seeing this?

V2.3.NEXT.2013May14.5D3113, Build:2013-0514 00:26:08

AJ

I've noticed this a few times on the 5D2. I've always assumed that it was an artifact from line skipping. Now that it's no longer smeared to death by the default video modes, it becomes clearer.
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 14, 2013, 05:49:46 PM
Quote from: a1ex on May 14, 2013, 05:27:09 PM
By how much?

Between about 29.959-29.978fps and 42.2-42.3MB/s respectively. I guess this is because of there being no defined frame rate. And of course in photo mode there isn't an FPS override. The reason I wanted this to work was on the 6D you could have *drumroll* WiFi monitoring. When trying it out it kinda half worked. One half sort of working, the other half crashing the shit out of my camera :P
Title: Re: uncompressed 14-bit RAW video recording
Post by: Redrocks on May 14, 2013, 05:52:30 PM
http://youtu.be/3Y4Gkr3ufzw

1280 x 720
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 14, 2013, 05:52:51 PM
Quote
Between about 29.959-29.978fps and 42.2-42.3MB/s respectively.
It's because Canon code switches between those two frame rates.
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 14, 2013, 05:57:17 PM
Quote from: a1ex on May 14, 2013, 05:52:51 PM
It's because Canon code switches between those two frame rates.

Ah I see :) So is it a possibility to override the FPS in photo mode? It would be amazing if I could monitor Liveview from my iPad while shooting!
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 14, 2013, 05:58:07 PM
In fact is it best I continue this over in the 6D thread?
Title: Re: uncompressed 14-bit RAW video recording
Post by: stevefal on May 14, 2013, 06:03:32 PM
Quote from: ajay on May 14, 2013, 05:33:17 PM
Tried some tests this morning with the 5DM3. I am noticing vertical banding in blurred backgrounds. They appear on the dng files. Anyone else seeing this?

V2.3.NEXT.2013May14.5D3113, Build:2013-0514 00:26:08

AJ

Yes, I am seeing that too. I'm shooting normal (not 5x). I found that you can knock it right out with a little bit of Luma noise reduction in Adobe Camera RAW.

By the way, that noise reduction seems to clear up the very fine noise that I see in many of the test footage, with little loss of detail, to my eye.
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 14, 2013, 06:04:51 PM
It probably has to do with Canon's downsampling algorithm?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 14, 2013, 06:08:43 PM
If you want it to stop switching just put on FPS override... exact FPS... but shutter values get limited.

P.S. You can monitor with wifi if you record real tiny frames... the wifi uses shoot malloc and shrinks largest contiguous to like 19MB.
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 14, 2013, 06:18:24 PM
Quote from: 1% on May 14, 2013, 06:08:43 PM
If you want it to stop switching just put on FPS override... exact FPS... but shutter values get limited.

P.S. You can monitor with wifi if you record real tiny frames... the wifi uses shoot malloc and shrinks largest contiguous to like 19MB.

My bad! Didn't realize FPS override worked. Looks like I just wasn't in LV at the time  :-[
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 14, 2013, 06:28:10 PM
1% - Didn't notice any speed change in photo mode no.

Liveview stream to seemed to work for a second to 6D and them camera gets error 70.

Second attempt seems to work okay but must be using same buffer as recording time was significantly reduced. Guess this ones a no starter.
Title: Re: uncompressed 14-bit RAW video recording
Post by: olik on May 14, 2013, 06:33:12 PM
With the risk that this has already been discussed; is there a way to shoot 1280x720 without the crop factor (from the 1920x1080 resolution) on 5dmk2?
I would also be very interested what gives me more sharpness, 1920x1080 h264 or 1280x720 raw.

thanks
O.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 14, 2013, 06:37:49 PM
Ok, did some quick tests:

10X zoom with 720P overriden = fastest speed.
640x480 with 10x zoom and override = close (works for 600D but 6D still makes HD buffer same size)
1080P slower
720P better than 1080P

so this confirms if we shut down some buffers it will help. Also only get 3 buffer chunks... shut anything down and we'll get an extra one so more likely to be continuous at higher res.
Title: 5D2 Dbuffer weird border on the right
Post by: ernestoeduardo on May 14, 2013, 06:42:56 PM
Hi everyone, i am getting this weird frame border on my 1880x720 videos, is 16px wide, and not completly black.


[I can not attach images, all i get when trying to is the bracketed tags, but no dialog to explore my PC ????]
Title: Re: uncompressed 14-bit RAW video recording
Post by: stevefal on May 14, 2013, 06:54:50 PM
Quote from: a1ex on May 14, 2013, 06:04:51 PM
It probably has to do with Canon's downsampling algorithm?

Could be. I shot the same static gradient in normal and 5x and the latter has none of the "cardboard" effect. ACR luma noise reduction seems to take care of it. I've been setting it to 50.

Possibly related moire at :45 (distant building) and :50 (vertical column):

Quote from: cinema5D on May 14, 2013, 04:21:58 PM
(http://www.cinema5d.com/seb/vimeo.jpg) (https://vimeo.com/66153520)
Title: Re: uncompressed 14-bit RAW video recording
Post by: stevefal on May 14, 2013, 06:59:23 PM
Idea: save copy of first frame as JPG with the same filename as the RAW, as a way to identify clips.
Title: Re: uncompressed 14-bit RAW video recording
Post by: kslau8 on May 14, 2013, 08:02:57 PM
Olik,

I did exactly those tests on the 5DMk2, it was quick but I've uploaded it to my site.  I also did a test to see what I could get from the shadows and highlights compared to h264.

Please visit my blog post for more info about settings:
http://www.walkhumblyfilms.com/?p=1257 (http://www.walkhumblyfilms.com/?p=1257)

Scaling Test

Dynamic Range - DNG vs H264 Test


Quote from: olik on May 14, 2013, 06:33:12 PM
With the risk that this has already been discussed; is there a way to shoot 1280x720 without the crop factor (from the 1920x1080 resolution) on 5dmk2?
I would also be very interested what gives me more sharpness, 1920x1080 h264 or 1280x720 raw.

thanks
O.
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 14, 2013, 08:13:37 PM


Interesting :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: JackDaniel412 on May 14, 2013, 08:34:15 PM
Hi everybody, there's a method to set personal height crop like 800px?
Thanks!
Title: Re: uncompressed 14-bit RAW video recording
Post by: olik on May 14, 2013, 08:35:20 PM
Thank you kslau8,

how did you managed to get rid of the crop factor of the 720p footage? Did you just changed the distance to the subject to compensate the difference or is there a way in the menu to change the crop?

thanks
O.
Title: Re: uncompressed 14-bit RAW video recording
Post by: CFP on May 14, 2013, 09:13:40 PM
Quote from: noisyboy on May 14, 2013, 08:13:37 PM

The Canon EOS 5D Mark III has nearly the same image quality as the Blackmagic Cinema Camera? Shit! This is way more than I have expected!

Canon, why don't you just give us what we want in the first place? And why do you have to give us such shitty SD controllers?  ;D
I would almost say I dislike Canon for creating this artificial limitations. But since Magic Lantern is only for Canon DSLRs there's no alternative anymore.

Well, again: Congratulations to all of you developers. Like I said, Magic Lantern truly is the hidden treasure of the internet.

I guess I'll start to save money ...
Title: Re: uncompressed 14-bit RAW video recording
Post by: Brian@202020 on May 14, 2013, 09:17:21 PM
Quote from: noisyboy on May 14, 2013, 08:13:37 PM


Interesting :)

Wow they are very comparable. They would cut together very nicely. You can see in the windows in the Dojo that the Blackmagic Cinema Camera has a little more latitude with less blow out. It's probably only a stop tho. Very nice. Thanks for the comparison.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ajay on May 14, 2013, 09:17:33 PM
Please excuse my ignorance here, but I keep hearing about the ability to crop/pan the sensor. I have not been able to figure out how to do this.

FYI...Transcend's 32GB, 1000x card works fine up to but not including 1920 x 1080. At 1920 x 1080, it does drop frames intermittently.

Thanks for any help.

AJ
Title: Re: uncompressed 14-bit RAW video recording
Post by: seant89 on May 14, 2013, 09:30:04 PM
So are MKii users restricted to resolutions below 1920 due to camera hardware or do we just need faster memory cards? Mine is a pathetic 266x 8GB PNY so I'm aware it's not ideal but I don't want to invest in a more expensive/faster memory card if it's the camera itself.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Luiz Roberto dos Santos on May 14, 2013, 09:36:36 PM
Quick test with the 600D. I get files 1280x540, but with macroblocks magenta (from time to time), perhaps due to my card "super-slow".
Processing was nil in both sequence (H.264 and RAW).
Title: Re: uncompressed 14-bit RAW video recording
Post by: Director on May 14, 2013, 09:37:58 PM
Mr. Alex

Can you make tutorial video.

How install RAW update in camera

because, I did't see link for download update

also I don't understand for commen txt.

Thanks
Title: Re: uncompressed 14-bit RAW video recording
Post by: Vegandelight on May 14, 2013, 09:40:09 PM
Quote from: Director on May 14, 2013, 09:37:58 PM
Mr. Alex

Can you make tutorial video.

How install RAW update in camera

because, I did't see link for download update

also I don't understand for commen txt.

Thanks

http://www.magiclantern.fm/forum/index.php?topic=5405.0
Title: Re: uncompressed 14-bit RAW video recording
Post by: DjJuvan on May 14, 2013, 09:52:40 PM
Holy crap Black Magic has more moire problems than RAW footage from 5dmk3

EDIT: Which CF cards does support 1080p without glitches/dropframes?

Has anyone tested this card? http://www.ebay.co.uk/itm/Compact-Flash-Card-64GB-1066x-Toshiba-the-world-fastest-Speed-CF-UDMA7-Ultra-/321106087953?pt=Digital_Camera_Memory_Cards&hash=item4ac36a1011 It has 150MB write speed.
Title: Re: uncompressed 14-bit RAW video recording
Post by: silvertonesx24 on May 14, 2013, 10:06:01 PM
Quote from: DjJuvan on May 14, 2013, 09:52:40 PM
Holy crap Black Magic has more moire problems than RAW footage from 5dmk3

EDIT: Which CF cards does support 1080p without glitches/dropframes?

I can get up to 1880x720 no problem on Mark II with KomputerBay 1000x CF. Any higher than that and I start to drop frames.
Title: Re: uncompressed 14-bit RAW video recording
Post by: olik on May 14, 2013, 10:23:20 PM
Quote from: silvertonesx24 on May 14, 2013, 10:06:01 PM
I can get up to 1880x720 no problem on Mark II with KomputerBay 1000x CF. Any higher than that and I start to drop frames.

well I can record 1880x720 on my mk2 with 600x cards without problems, so this indicates that the bottle-neck is not the card. :-/
Title: Re: uncompressed 14-bit RAW video recording
Post by: pavelpp on May 14, 2013, 10:24:48 PM
Wow, this is so great news! Anyone working on a 550d port? There are so many 550d owners who would appreciate raw recording mode.. Maybe there is some experimental build ?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 14, 2013, 10:27:54 PM
Quote from: olik on May 14, 2013, 10:23:20 PM
well I can record 1880x720 on my mk2 with 600x cards without problems, so this indicates that the bottle-neck is not the card. :-/

Yeah, I think it depends on whether or not he had global draw on. Like a1ex mentioned earlier, the benchmarks will tell you the truth. Another user benched about 65 MB/s on his 5D2 with a 1000x card, so it should be able to handle at least a step higher than 720. Though it's not a huge improvement it's still an improvement!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 14, 2013, 10:39:52 PM
Quote from: ajay on May 14, 2013, 09:17:33 PM
Please excuse my ignorance here, but I keep hearing about the ability to crop/pan the sensor. I have not been able to figure out how to do this.

FYI...Transcend's 32GB, 1000x card works fine up to but not including 1920 x 1080. At 1920 x 1080, it does drop frames intermittently.

Thanks for any help.

AJ

Is this on the mark III? Cause I am expecting 2 of these tomorrow and that will be disappointing! Can you upload footage/benchmark for your tests with the Transcend 32gb 1000x?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 14, 2013, 10:45:57 PM
Sorry, this is slightly OT but Is my understanding of this correct?

The RAW files we are now getting are the exact same data that is usually processed in-camera by the H.264 encoder to make .mov files? or is the sensor sampled differently (i.e. whole sensor) somehow before it makes a H.264?

If that's correct, the H.264 encoder itself must be lightning fast and use a lot of processing power to convert in real time? Is it hardware or software?
Title: Re: uncompressed 14-bit RAW video recording
Post by: -sandro- on May 14, 2013, 10:51:25 PM
I wonder what Canon is thinking about this. Their crippling has been converted into a better version of the high end video cameras they want to protect! AHAHA!
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 14, 2013, 10:53:27 PM
RAW (dng, etc) -> yuv -> h264

Every one of those is a resize and then people wonder about aliasing and moire.
Title: Re: uncompressed 14-bit RAW video recording
Post by: pravdomil on May 14, 2013, 10:57:52 PM
Well, RAW recording with sound (via ML audio recorder) works nicely! But you have to synchronize it manually.
Sandisk extreme 16GB 60MB/s
Global redraw off
5D2 720p 25fps (up to 4GB)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 14, 2013, 11:05:45 PM
@1% - Thanks. You're educational mate :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: ajay on May 14, 2013, 11:10:48 PM
Quote from: Habitat on May 14, 2013, 10:39:52 PM
Is this on the mark III? Cause I am expecting 2 of these tomorrow and that will be disappointing! Can you upload footage/benchmark for your tests with the Transcend 32gb 1000x?

I'm about to run out the door, maybe later I can post a benchmark, but yes, that's on the Mark III.

Can someone answer my question regarding cropping? Is is possible to crop and only use the center pixels? Or some other smaller width than the actual sensor for 1920 x 1080? That's what I was hoping for...similar to the Panasonic GH2/GH3.

Thanks,

AJ
Title: Re: uncompressed 14-bit RAW video recording
Post by: silvertonesx24 on May 14, 2013, 11:16:23 PM
Quote from: Yoshiyuki Blade on May 14, 2013, 10:27:54 PM
Yeah, I think it depends on whether or not he had global draw on. Like a1ex mentioned earlier, the benchmarks will tell you the truth. Another user benched about 65 MB/s on his 5D2 with a 1000x card, so it should be able to handle at least a step higher than 720. Though it's not a huge improvement it's still an improvement!

The fastest write speed I can achieve with Global Draw off is 65.1 MB/s at buffer 15625k. Card is a KomputerBay 32gb 1000x

On my KomputerBay 32gb 600x, I can only manage about 23.8 MB/s max write. Read speeds are also very slow, around 11-15 vs. 60-70 on the 100x. So unless I have a dud card, the 1000x is significantly faster.

Title: Re: uncompressed 14-bit RAW video recording
Post by: -sandro- on May 14, 2013, 11:17:05 PM
What is the size of 1 minute of 1920x1080 RAW footage?
Title: Re: uncompressed 14-bit RAW video recording
Post by: vitos1k on May 14, 2013, 11:18:28 PM
Quote from: Luiz Roberto dos Santos on May 14, 2013, 09:36:36 PM
Quick test with the 600D. I get files 1280x540, but with macroblocks magenta (from time to time), perhaps due to my card "super-slow".
Processing was nil in both sequence (H.264 and RAW).


How fast benchmark was for you? I've got 650D and SDHC 32GB Class 10 UHS-I Transcend TS32GSDHC10U1
and benchmark gives me speed up to 37.9mb/s
i was wondering, which resolution i could handle with RAW recording?!
(https://dl.dropboxusercontent.com/u/3904468/VRAM0.BMP)
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 14, 2013, 11:34:42 PM
I just installed the May 14th build on my 5d3. But I dont see the 'M' tab in the Magic Lantern Menu. So how am I supposed to enable the raw recording on this build?
Title: Re: uncompressed 14-bit RAW video recording
Post by: vitos1k on May 14, 2013, 11:38:13 PM
hm, using simple math i found that i could handle 1280x720 with 37.9 MB/s writing speed
Cool! Waiting for 650D RAW footage implementation :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 14, 2013, 11:44:12 PM
Suddenly the 650d looks like a winner based on those card benchmarks (and there are even faster UHS-I cards out there). That will be the BMCC pocket killer... eventually ::)


@Vitos1k - can you run the card benchmarks again with global draw OFF and post the results?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Rush on May 15, 2013, 12:01:26 AM
Andy600, no way because it can't go higher than ~1280x720 because of SD interface limit. BM Pocket is capable of 1920x1080

For 600D/60D/550D:
If you record in canon 720p mode, height is line skipped by 1.66x.
So 1280x400 will give you 1280x664 after restoring correct aspect ratio - it is so close to 16:9! (434px height will result in 16:9, but it hits SD limit and results in frameskipping...)

1280x360 will result in 2.13:1 aspect 1280x600
1280x320 will result in 2.40:1 aspect 1280x533 (cinemascope!)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 15, 2013, 12:12:35 AM
@Rush the theoretical limit for UHS-I is 104MB/s and there are cards capable of 80+ MB/s so I think it will go higher no? http://panasonic.net/avc/sdcard/features/uhs-i.html

update - oops, read the wiki wrong. It's 50MB/s unless it's UHS104. I need to check the specs. It's still better than the 600d  :-\
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 12:14:07 AM
Also if 650D is anything like EOS M, the buffer is really small.

I get only 3x30MB allocated... but I have 152MB on 6D... what gives?

That UHS-I is the 208mhz which no canon camera has at the moment.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Luiz Roberto dos Santos on May 15, 2013, 12:14:37 AM
Quote from: vitos1k on May 14, 2013, 11:18:28 PM
How fast benchmark was for you? I've got 650D and SDHC 32GB Class 10 UHS-I Transcend TS32GSDHC10U1
and benchmark gives me speed up to 37.9mb/s
i was wondering, which resolution i could handle with RAW recording?!
(https://dl.dropboxusercontent.com/u/3904468/VRAM0.BMP)

So ... I have a card very slow, as I said in the video. His capacity is 8.8 Mb/s, ridiculously incapable of it all.
The resolution that was able to record a few seconds of 1280x540.

[sorry for the english]
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 15, 2013, 12:19:54 AM
@1% - yes, that is a bit odd  ??? Canon cutting corners?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 12:29:19 AM
I think something wrong with the allocation.. or the other 2x30MB is the double buffer?
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 15, 2013, 12:39:38 AM
Allocation is not yet fully understood, so it may have glitches. These malloc commands never fail, but get queued and stay there until you reboot the camera.

ML is trying to autodetect the free RAM, but that means it will perform malloc calls that will timeout (and will bite you later). It still needs research; right now it's something that kinda works and is portable... sort of.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shawn_Lights on May 15, 2013, 12:47:09 AM
Not sure about the tech stuff, but If you guys can get a useable raw on the 550D/600D I'm going to assume you guys are aliens.

It seems the limitations will be SD card write speeds. SanDisk 64GB SDXC Memory Card Extreme Pro Class 10 UHS-I claims 95MB/s read speeds, but I'm not sure if that is so. I remember reading somewhere that you only get 30-40MB/s with SD cards. It may be Canon cheating somewhere.
Title: Re: uncompressed 14-bit RAW video recording
Post by: vitos1k on May 15, 2013, 12:57:55 AM
Quote from: Andy600 on May 14, 2013, 11:44:12 PM
Suddenly the 650d looks like a winner based on those card benchmarks (and there are even faster UHS-I cards out there). That will be the BMCC pocket killer... eventually ::)


@Vitos1k - can you run the card benchmarks again with global draw OFF and post the results?

Second test, with GLOBAL DRAW OFF

(https://dl.dropboxusercontent.com/u/3904468/VRAM1.BMP)

not much of a difference. But still 37.9MB/s
Title: Re: uncompressed 14-bit RAW video recording
Post by: Rush on May 15, 2013, 01:05:48 AM
Quote from: Shawn_Lights on May 15, 2013, 12:47:09 AM
Not sure about the tech stuff, but If you guys can get a useable raw on the 550D/600D I'm going to assume you guys are aliens.
The best you can with 600D (550D/60D) is: (limited by hardware SD transfer rates)
- 960х540, upscale it to 1280х720 (24 fps max)
- 1280х400, tight aspect 3.20:1 (24 fps max)
In Canon's 720p - lineskipping 1.66х, so you should upscale to restore correct ratio:
- 1280x400 upscales to 1280x664 - near to 16:9 (24 fps max)
- 1280x360 upscale to 1280x600 - 2.13:1 aspect
- 1280x320 upscale to 1280x533 - 2.40:1 cinemascope aspect

It is only 1280px wide, but pretty nice looking though :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 01:06:39 AM
~40 is the max for UHS-I capable stuff. Mainly Sandisk 45MB/s, the pro is supposedly benching the same?

~20 is the max for non UHS capable stuff.


Yea. The memory thing is weird. On 600D I get 3X30MB buffers and I have 104MB.. on 6D its the same... but technically we should have 5 buffer stars there. If this is fixed I think a whole lot more resolutions will be usable.

On 5d3 you get how many? Since we have a similar amount of memory.
Title: Re: uncompressed 14-bit RAW video recording
Post by: kgv5 on May 15, 2013, 01:08:32 AM
Is is possible to have raw resolution preset cropmarks visible before the start of shooting?

It would help a lot with proper framing, now we can see cropmarks after pressing "start" .
Title: Re: uncompressed 14-bit RAW video recording
Post by: Luiz Roberto dos Santos on May 15, 2013, 01:09:52 AM
Quote from: Rush on May 15, 2013, 01:05:48 AM
The best you can with 600D (550D/60D) is: (limited by hardware SD transfer rates)
- 960х540, upscale it to 1280х720 (24 fps max)
- 1280х400, tight aspect 3.20:1 (24 fps max)
In Canon's 720p - lineskipping 1.66х, so you should upscale to restore correct ratio:
- 1280x400 upscales to 1280x664 - near to 16:9 (24 fps max)
- 1280x360 upscale to 1280x600 - 2.13:1 aspect
- 1280x320 upscale to 1280x533 - 2.40:1 cinemascope aspect

It is only 1280px wide, but pretty nice looking though :)


I did 1280x540, not crop, although data blocks magenta, perhaps by the speed of the card.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Rush on May 15, 2013, 01:13:37 AM
Quote from: Luiz Roberto dos Santos on May 15, 2013, 01:09:52 AM

I did 1280x540, not crop, although data blocks magenta, perhaps by the speed of the card.
I posted max possible continuous RAW-recrording on 600D without frameskipping. Any higher will result in frameskipping...

Magenta blocks appears randomly, but it looks like accessing canon menu or ML menu while recording will result in this blocks too.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shawn_Lights on May 15, 2013, 01:17:52 AM
Quote from: Rush on May 15, 2013, 01:05:48 AM
The best you can with 600D (550D/60D) is: (limited by hardware SD transfer rates)
- 960х540, upscale it to 1280х720 (24 fps max)
- 1280х400, tight aspect 3.20:1 (24 fps max)
In Canon's 720p - lineskipping 1.66х, so you should upscale to restore correct ratio:
- 1280x400 upscales to 1280x664 - near to 16:9 (24 fps max)
- 1280x360 upscale to 1280x600 - 2.13:1 aspect
- 1280x320 upscale to 1280x533 - 2.40:1 cinemascope aspect

It is only 1280px wide, but pretty nice looking though :)

Really can't complain with that. With it being RAW detail should be amazing. Better than the h.264 codec. Not to mention cinemascope is where it's at :D

I haven't been able to test it out. How is framing done? Does it change in liveview mode so you can frame properly or does it require cropbar overlays?
Title: Re: uncompressed 14-bit RAW video recording
Post by: ashtrai on May 15, 2013, 01:20:57 AM
Quote from: Rush on May 15, 2013, 01:05:48 AM
The best you can with 600D (550D/60D) is: (limited by hardware SD transfer rates)
- 960х540, upscale it to 1280х720 (24 fps max)
- 1280х400, tight aspect 3.20:1 (24 fps max)
In Canon's 720p - lineskipping 1.66х, so you should upscale to restore correct ratio:
- 1280x400 upscales to 1280x664 - near to 16:9 (24 fps max)
- 1280x360 upscale to 1280x600 - 2.13:1 aspect
- 1280x320 upscale to 1280x533 - 2.40:1 cinemascope aspect


It is only 1280px wide, but pretty nice looking though :)

Can someone post samples from these resolutions from a 600D? I want to try to compare it upscaled against the H264 equivalents.
Title: Re: uncompressed 14-bit RAW video recording
Post by: tferradans on May 15, 2013, 01:29:31 AM
managed to get about 400 frames in 1920x1080 on a 5D3 today, using 90mbps Sandisk Card.
Turning off Global Draw does a huge difference when it comes to dropping frames.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 01:32:04 AM
Should compare 1080P/720P/320P and zoom. Seems performance is different based on those and the raw image will differ between 1080/zoom/720. Taking a crop of that may give you a higher res but I'm looking at the 720P mode stuff and its not as good as 1080 or zoom.

But better res vs more frames... I dunno. You don't have to turn global draw off completely, just shut down things like zebrase, etc.
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 15, 2013, 01:32:25 AM
QuoteTurning off Global Draw does a huge difference when it comes to dropping frames.
Try setting both task priorities to 0x18 instead of 0x1a in raw_rec.c, maybe it helps with globaldraw.
Title: Re: uncompressed 14-bit RAW video recording
Post by: tferradans on May 15, 2013, 01:32:28 AM
forgot to ask: is it possible to record raw files to the CF card, and regular h264 video to the SD card? Would be great for previewing on set and making raw shooting a lot safer. :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: kgv5 on May 15, 2013, 01:34:02 AM
Hey guys,
with something about 1504x600 crop resolution set we can use tokina 11-16 at 11,5mm with absolutely no vignette on full frame cameras  :D
So now it's like a crop sensor DSLR (with low light abilities of FF). It would give ~15mm equivalent for full frame  (crop factor ~1,3) :D
(15mm is also about maximum usable value for 11-16 on full frame).
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 15, 2013, 01:34:49 AM
Quoteis it possible to record raw files to the CF card, and regular h264 video to the SD card?
Should work if you hardcode the raw file path to A:/ and select B:/ from canon menu.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 15, 2013, 01:38:43 AM
Quote from: ajay on May 14, 2013, 09:17:33 PM
Please excuse my ignorance here, but I keep hearing about the ability to crop/pan the sensor. I have not been able to figure out how to do this.

FYI...Transcend's 32GB, 1000x card works fine up to but not including 1920 x 1080. At 1920 x 1080, it does drop frames intermittently.

Thanks for any help.

AJ

Also was that with Global Draw on or Off?
Title: Re: uncompressed 14-bit RAW video recording
Post by: tferradans on May 15, 2013, 01:39:21 AM
Quote from: a1ex on May 15, 2013, 01:34:49 AM
Should work if you hardcode the raw file path to A:/ and select B:/ from canon menu.

Niiice.  How can I hardcode that? I tried looking into raw.c (no decent programming skills) and found no mention to where it saves the raw file (except for a comment)...
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 01:43:56 AM
Quote15mm is also about maximum usable value for 11-16 on full frame

Ha! good thing for mine when I send it in.. will also fix my 35-135 which still works :)

Recorded a 10gb and 25gb file... checking to see if I can bust up the 10gb... write speed was really slowing down to like 1/2 towards the end.
Title: Re: uncompressed 14-bit RAW video recording
Post by: VectorZ on May 15, 2013, 01:59:13 AM
Does Shutter Speed adjustment affect motion blur the same way with the Raw Recordind module? Do you guys think it will be possible to have simultaneous HDMI out and onboard monitor and record raw when ported to 5DMk3 1.2.1 firmware?
Title: Re: uncompressed 14-bit RAW video recording
Post by: hirethestache on May 15, 2013, 02:13:42 AM
Quote from: VectorZ on May 15, 2013, 01:59:13 AM
Does Shutter Speed adjustment affect motion blur the same way with the Raw Recordind module? Do you guys think it will be possible to have simultaneous HDMI out and onboard monitor and record raw when ported to 5DMk3 1.2.1 firmware?

I was about to ask these exact same questions...beat me to it!


Since the last I inquired about recording to the HDMI-out, the developers implied that you did not even thing continuous recording would be possible...but your magical ingenuity has broken through! Is the idea of running through the HDMI's limit of 10gpbs (realistically 3.2) even a feasible approach, or is it completely unrealistic? This would reduce the strain on the camera, and heat would be much less of an issue...
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 15, 2013, 02:16:58 AM
Quote from: 1% on May 15, 2013, 01:43:56 AM
Ha! good thing for mine when I send it in.. will also fix my 35-135 which still works :)

Recorded a 10gb and 25gb file... checking to see if I can bust up the 10gb... write speed was really slowing down to like 1/2 towards the end.

Did you record a 25gb file? What about the 4gb file limit? How did you get passed that?
Title: Re: uncompressed 14-bit RAW video recording
Post by: KenFTW on May 15, 2013, 02:29:02 AM
ive tried many configurations to be able to get usable footage with my 600d, the only problem is that damn magenta!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shawn_Lights on May 15, 2013, 02:31:05 AM
Quote from: KenFTW on May 15, 2013, 02:29:02 AM
ive tried many configurations to be able to get usable footage with my 600d, the only problem is that damn magenta!

It's a RAW file...turn the magenta/green slider to 0.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 15, 2013, 02:34:49 AM
I think he's talking about the magenta blocks..
Title: Re: uncompressed 14-bit RAW video recording
Post by: KenFTW on May 15, 2013, 02:36:12 AM
@shawn_lights perhaps I phrased it wrong, its actually a file corruption that shows up in a magenta color.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shawn_Lights on May 15, 2013, 02:41:29 AM
Oh I see.
Title: Re: Anyone get the 600D to work without the magenta blocks/tearing?
Post by: mattmvpmedia on May 15, 2013, 02:46:57 AM
Dear ML team: you rock. Thank you.

I've been testing out RAW on my 600D T3i and I always get tearing or magenta blocks in some of the frames. Has anyone figured out a solution to this yet?

Thanks!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 15, 2013, 02:54:13 AM
I'm sure if/when there are fixes and improvements the devs will upload them. Just keep an eye on https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads for the 600d and use the latest files  :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 02:54:32 AM
Yea, the 4gb limit is software.... but the converter hit a snag:

(http://i.imgur.com/ca85bAdl.jpg)

This footer looks good?

I tried adding D_LARGEFILE_SOURCE and changing the fseeks to fseeko64.. still no go.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Brian@202020 on May 15, 2013, 03:44:23 AM
Maybe I'm jumping ahead of myself, but do you think RAW HDR could eventually be possible?
Title: Re: uncompressed 14-bit RAW video recording
Post by: mixmastermike on May 15, 2013, 03:45:55 AM
Just updated to the May 14th build.

Im using a Lexar 32Gb x1000 CF and I didn't have any dropped frames. With earlier builds I got dropped frames, the recording only stopped when I hit 4Gb.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mattmvpmedia on May 15, 2013, 03:48:23 AM
Thanks Andy600
Title: Re: uncompressed 14-bit RAW video recording
Post by: TjS on May 15, 2013, 03:52:32 AM
//
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 03:56:22 AM
Yea, I experienced some of that... especially some resolution leaves it allocating the buffer and not recording.

Quote
RAW HDR could eventually be possible

Yea, probably with existing functions just raw recording needs to trigger it. It moves too slow during preview
Title: Re: uncompressed 14-bit RAW video recording
Post by: tackyattack on May 15, 2013, 04:03:24 AM
I don't think the UHS-I pins on the cards are supported by the Canon DSLRs, only the normal interface. So you're still going to get the normal SD card speeds whether it's UHS or not. I have been designing a device that plugs into the SD card slot of the DSLR and accesses the SPI pins. An ARM chip then emulates an SD card with data storage on a FAT32 partitioned SSD. This would allow very high data speeds. That will erase the problems of the SD card being too slow, but I have no idea how fast the SPI Bus on the canon cameras can go. Does anybody know of the max mhz the bus can run at?

Thanks
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 15, 2013, 04:12:03 AM
Okay so this is pretty damn cool. Just thought I'd have another go at 1540x640 as I only ever get 700 frames MAX before it starts skipping frames @25fps.

Just tried at 24fps instead and the difference is immense! All of a sudden hitting between 1500-2000 frames before any skipping even begins to start! Gonna try other resolutions to see if I can get some better performance! Whaaaaaat a difference a frame makes ;)

On 6D by the way.
Title: Re: uncompressed 14-bit RAW video recording
Post by: silvertonesx24 on May 15, 2013, 04:14:39 AM
Quote from: KenFTW on May 15, 2013, 02:36:12 AM
@shawn_lights perhaps I phrased it wrong, its actually a file corruption that shows up in a magenta color.

I am getting the same magenta block issues on 5Dii. Big magenta blocks randomly in footage. I didn't notice it until I went out to do a low light test and used my external HDMI monitor. I also noticed it on a h264 recording, so this may be a general ML bug and not a raw one. I will try it again later without the HDMI monitor to see if that could be causing the issue.

Regardless, the test results were just amazing. ML RAW->ProRes 444 and ColorGhear is absolutely stunning.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 15, 2013, 05:02:53 AM
Quote from: DjJuvan on May 14, 2013, 09:52:40 PM

EDIT: Which CF cards does support 1080p without glitches/dropframes?

Has anyone tested this card? http://www.ebay.co.uk/itm/Compact-Flash-Card-64GB-1066x-Toshiba-the-world-fastest-Speed-CF-UDMA7-Ultra-/321106087953?pt=Digital_Camera_Memory_Cards&hash=item4ac36a1011 It has 150MB write speed.

I've got one on order, it's on a slow boat from China, should be here in a week or so. They're not available anywhere outside of Asia yet. The Lexar 1000x is the most robust with no dropped frames @ 1080p.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Nspa32 on May 15, 2013, 05:07:01 AM
How's this looking for the 550D, guys?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shawn_Lights on May 15, 2013, 05:10:25 AM
Quote from: Nspa32 on May 15, 2013, 05:07:01 AM
How's this looking for the 550D, guys?

Wondering the same thing. I suppose the 600D development is an indication of what we'll see on the 550D
Title: Re: uncompressed 14-bit RAW video recording
Post by: Nspa32 on May 15, 2013, 05:19:47 AM
Quote from: Shawn_Lights on May 15, 2013, 05:10:25 AM
Wondering the same thing. I suppose the 600D development is an indication of what we'll see on the 550D

Right... If we can get 720p RAW with 24fps I'll be happy. I'm sure 720P RAW blows 1080P h.264 away.
Title: Re: uncompressed 14-bit RAW video recording
Post by: jasonp on May 15, 2013, 05:41:45 AM
@TjS

I'm using a Komputer bay 64GB x1000 and have the same issue as you. Although the benchmarks are looking good (90MB/s), and I can record 1920x960 @ 75MB/s, when I try recording 1920x1080 it only runs @50-65MB/s and drops frames. I'm using the latest (May 14) built by lourenco, I didn't have any issues with the previous built. I guess we'll have to wait :)   
Title: Re: uncompressed 14-bit RAW video recording
Post by: Brian@202020 on May 15, 2013, 06:02:04 AM
Now if Mosaic Engenering would ever make an AA filter for the 5D3 then we'd be in business.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ajay on May 15, 2013, 06:25:56 AM
Here are test results of the Transcend 1000x 32 GB on the 5DM3:

(http://iwishicouldfly.com/iwishicouldfly/images/VRAM0.BMP)

In practice I have found 1920 x 840 seems to work flawlessly, greater than that eventually skips frames.

AJ

Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 15, 2013, 06:29:22 AM
Quote from: Brian@202020 on May 15, 2013, 06:02:04 AM
Now if Mosaic Engenering would ever make an AA filter for the 5D3 then we'd be in business.

What for? There's no aliasing or moire.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mixmastermike on May 15, 2013, 06:50:37 AM
Quote from: ajay on May 15, 2013, 06:25:56 AM
Here are test results of the Transcend 1000x 32 GB on the 5DM3:

(http://iwishicouldfly.com/iwishicouldfly/images/VRAM0.BMP)

In practice I have found 1920 x 840 seems to work flawlessly, greater than that eventually skips frames.

AJ

Try turning global draw off, that should release a little more memory for 1920 x 1080 recording.

I have the same card and it fluctuates between 80-84MB/s recording 1920 x 1080.
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 15, 2013, 07:02:36 AM
Quote from: tackyattack on May 15, 2013, 04:03:24 AM
I don't think the UHS-I pins on the cards are supported by the Canon DSLRs, only the normal interface. So you're still going to get the normal SD card speeds whether it's UHS or not. I have been designing a device that plugs into the SD card slot of the DSLR and accesses the SPI pins. An ARM chip then emulates an SD card with data storage on a FAT32 partitioned SSD. This would allow very high data speeds. That will erase the problems of the SD card being too slow, but I have no idea how fast the SPI Bus on the canon cameras can go. Does anybody know of the max mhz the bus can run at?

Thanks

Kickstarter baby  8)
Title: Re: uncompressed 14-bit RAW video recording
Post by: ajay on May 15, 2013, 07:03:17 AM
Quote from: mixmastermike on May 15, 2013, 06:50:37 AM
Try turning global draw off, that should release a little more memory for 1920 x 1080 recording.

I have the same card and it fluctuates between 80-84MB/s recording 1920 x 1080.

Funny you should mention that, cuz I just did that and it did help a bit. I also turned off audio...not sure if it matters, but I turned off just about everything I could think of. I can now get 1920 x 960 to work reliably although 1920 x 1080 still skips frames intermittently. Sometimes I can go pretty far w/o skips.

Thanks,

AJ
Title: Re: uncompressed 14-bit RAW video recording
Post by: mixmastermike on May 15, 2013, 07:23:25 AM
Quote from: ajay on May 15, 2013, 07:03:17 AM
Funny you should mention that, cuz I just did that and it did help a bit. I also turned off audio...not sure if it matters, but I turned off just about everything I could think of. I can now get 1920 x 960 to work reliably although 1920 x 1080 still skips frames intermittently. Sometimes I can go pretty far w/o skips.

Thanks,

AJ

I wasn't able to record 1920 x 1080, with earlier builds the buffer kept filling up but doing some testing today with the May 14th build it seemed to sort it out. I didn't stress it out with leave's or trees or anything like that but the frame were busy and it handled it well till i hit the 4Gb limit.

Are you running the latest build?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 15, 2013, 07:37:02 AM
Quote from: ajay on May 15, 2013, 06:25:56 AM
In practice I have found 1920 x 840 seems to work flawlessly, greater than that eventually skips frames.

AJ

You gotta be in liveview video mode for a more realistic benchmark. It appears you tested this in playback mode (and as mentioned, turn off global draw to maximize your card's performance in liveview). At the rate resolution you stated, you're getting more than 64-65 MB/s, but less than 80ish.
Title: Re: uncompressed 14-bit RAW video recording
Post by: pavelpp on May 15, 2013, 08:38:16 AM
would 1280x400 raw crop on 550d look better than native 1080p H.264? Somehow I doubt that... You would still need to encode the material for viewing..
Title: Re: uncompressed 14-bit RAW video recording
Post by: Chungdha on May 15, 2013, 08:45:59 AM
I would not mind a 720p that been crop to a cinescopish crop as I dont even upload anything bigger than 720p. But would like to test it if any want to compile it for 550d, also wasnt 550d more stable than 600d with allot of the coding?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Felixlgr on May 15, 2013, 09:37:06 AM
5D3 with transcend 1000x CF card

i have been experiencing with the nightly build and latest RAW features...its amazing !!! Big thanks to all the developers...i had a few crash and i kept al the logs... anybody from dev team interested in getting them ?

Im also wondering if it would be better to have ML installed on the SD card and only use the CF for recording... would it help reduce skipped frames ?

thanks all and keep up the good work so we can have some kind of stable build soon
Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 15, 2013, 10:15:43 AM
Quote from: Felixlgr on May 15, 2013, 09:37:06 AM
5D3 with transcend 1000x CF card

i have been experiencing with the nightly build and latest RAW features...its amazing !!! Big thanks to all the developers...i had a few crash and i kept al the logs... anybody from dev team interested in getting them ?

Im also wondering if it would be better to have ML installed on the SD card and only use the CF for recording... would it help reduce skipped frames ?

thanks all and keep up the good work so we can have some kind of stable build soon

Could you post a benchmark plz? From live view with global draw off?
Title: Re: uncompressed 14-bit RAW video recording
Post by: DjJuvan on May 15, 2013, 10:34:32 AM
I ordered the Toshiba 1066x CF card.... I hope I get it in three weeks and I'll report about the speeds. I hope for non-dropping frames at 1080p.
Title: Re: uncompressed 14-bit RAW video recording
Post by: di2010 on May 15, 2013, 10:42:25 AM
I hope, this hasn't been asked yet or I could have found an answer myself. Does anyone please have a compiled version of the RAW enabled firmware for the 5D Mk II?

Thanks so much to the forum!

Title: Re: uncompressed 14-bit RAW video recording
Post by: vicnaum on May 15, 2013, 10:52:12 AM
Benchmark of SDHC 16GB Sandisk Extreme HD Video (UHS-I, 45MB/s*, 300X, 10 class) on 600D

(http://www.naumik.com/temp/scr3/20130515-v0g-134kb.jpg)

Same slow 21MB/s as my old SanDisk Ultra (Class 10, 30MB/s*).

So, I guess no luck for good RAWs on 600D, until any optimizations (maybe compression, or something...).
Will hope for good.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 15, 2013, 10:57:45 AM
@vicnaum - can you rerun the benchmark with Global Draw OFF? It's already near the maximum for the 600d buffer but interested to see if there's anymore
Title: Re: uncompressed 14-bit RAW video recording
Post by: DjJuvan on May 15, 2013, 11:16:32 AM
Just a little tip for developers... I think you should add an option to delete .RAW files from inside the camera.... if you make a lot of mistakes while shooting and not able to delete last failed shoot, it's gonna be a problem shooting outside. :)

PS: And adding the "start/stop" recording in RAW outside the ML menu.
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 15, 2013, 11:25:04 AM
its time for a file manager module ;)

who does it?
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 15, 2013, 11:30:36 AM
Quote from: DjJuvan on May 15, 2013, 11:16:32 AM
Just a little tip for developers... I think you should add an option to delete .RAW files from inside the camera.... if you make a lot of mistakes while shooting and not able to delete last failed shoot, it's gonna be a problem shooting outside. :)

PS: And adding the "start/stop" recording in RAW outside the ML menu.

+1

Thank you for suggesting this!!!
Title: Re: uncompressed 14-bit RAW video recording
Post by: saad on May 15, 2013, 12:27:08 PM
And another test I did, this time with Cinestyle on:



:)
Title: Re: uncompressed 14-bit RAW video recording
Post by: HugoFilipe on May 15, 2013, 12:31:18 PM
Is there any chance of a benchmark from the Komputerbay 128GB 1000x card?
Will it be able to hold 1920x1080 continuously?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Lcrusher on May 15, 2013, 12:42:51 PM
Both EosHD and Cinema5D confirm that Komputerbay works on fullHD
Title: Re: uncompressed 14-bit RAW video recording
Post by: ExplosiveFilms on May 15, 2013, 12:49:56 PM
Sorry if I missed it if it was clearly stated before:
Is there a difference in RAW video performance on the Mark II compared to the Mark III?
Title: Re: uncompressed 14-bit RAW video recording
Post by: DjJuvan on May 15, 2013, 12:57:11 PM
Quote from: Lcrusher on May 15, 2013, 12:42:51 PM
Both EosHD and Cinema5D confirm that Komputerbay works on fullHD

Not exactly true... they had dropframes.
Title: Re: uncompressed YUV422 and 14-bit RAW video recording
Post by: tuzzio on May 15, 2013, 01:02:59 PM
Quote from: coutts on May 14, 2013, 12:49:27 AM
This should work for 5d2, double buffering enabled:
http://bit.ly/10TBM7a

extract the MODULES directory into /ML/ on your card

this link is down because of too much traffic.. can i download it somewhere else??
Title: Re: uncompressed 14-bit RAW video recording
Post by: Rush on May 15, 2013, 01:05:01 PM
Quote from: ashtrai on May 15, 2013, 01:20:57 AM
Can someone post samples from these resolutions from a 600D? I want to try to compare it upscaled against the H264 equivalents.
1280х360:
http://www.youtube.com/watch?v=w48Eqz7DAlg
Title: Re: uncompressed 14-bit RAW video recording
Post by: Brian@202020 on May 15, 2013, 01:10:09 PM
Quote from: squig on May 15, 2013, 06:29:22 AM
What for? There's no aliasing or moire.

That's sarcasm I hope. Regardless of whether the 5D3 is shooting H264 or RAW it's still line skipping. It's very apparent in the RAW shots posted, except the 1-1 stuff. That said it's good enough for most people in most situations, I just strive for perfection. Magic Lantern is one of the keys, and Mosaic Engineering is the other.
Title: Re: uncompressed 14-bit RAW video recording
Post by: kgv5 on May 15, 2013, 01:10:45 PM
I think that canon bosses are now thanking to God that engeneers didn't give 208mhz UHS-I card controller for 600d, 6d and other cameras...
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 15, 2013, 01:16:23 PM
@1% - just found a bug on the 600d. If you use 5x/10x zoom, the readout that tells you the write speed and FPS (RAW) changes from 24fps to 29.974fps even if FPS override is enabled
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 15, 2013, 01:20:53 PM
LV zoom modes have a different video mode and fixed FPS
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 15, 2013, 01:23:43 PM
Quote from: g3gg0 on May 15, 2013, 01:20:53 PM
LV zoom modes have a different video mode and fixed FPS

Exclusive of NTSC/PAL selection?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kuky on May 15, 2013, 01:23:58 PM
What worflows do you guys use with the DNG files in Windows environment?

I found out that:

A. Davinci Resolve does not support them.
B. Adobe Speedgrade does not support them.
C. Adobe Lightroom it's ok, but only for simple tasks. For example if you do a highlight recovery and sync all dng files then export to a nle will result in flicker. Probably because although all files have exactly the same settings, internally LR will try to optimise the highlight recovery on a per file basis. (?).
D. AfterEffects through the ACR import. This will have best results. Once you set the interpretation on first file, it's smart enough to keep things consistent for the sequence.  Very, very slow.
E. Cineform Studio/Pro (also the dpx2cf tool) will recognise dng files and convert the sequence to a cineform raw file. You have to fiddle with demosaic (advanced detail 3 seems to work best) and all other settings. The fastest way to preview your movies. Converting it's ultrafast and the result is realtime playable. Fastest way to edit, until now.
F. uffraw batch. Tried once, but slow.

Any other ideas? It seems that cinemadng files and dng files are not *exactly* the same thing and didn't find a way to convert from one to another.

Regards,
Cristian


PS. If the raw2dng could convert directly to cinemadng would be awesome.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 15, 2013, 01:34:26 PM
@Kuky - I think Ginger HDR uses a DNG/CinemaDNG wrapper in AE and Premier Pro. Not sure if that would improve things?

http://19lights.com/wp/tutorials/raw-and-cinemadng-wrapper-workflow/
Title: Re: uncompressed 14-bit RAW video recording
Post by: jipo on May 15, 2013, 01:41:16 PM
Hi,
first of all, i would like to thank you all developers, activities around the raw recording are very promising...nice work:-)
Maybe i missed it, but i didn't find any discussion about fps, so me question is simple:
is it possible (theoretically in the future) use this lv/raw recording functionality to enable currently not possible frame rates(for slow motion) in the limited resolution to satisfy discussed write speeds? Or there is no relation with the livv view buffer reading?



Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 15, 2013, 01:44:44 PM
Quote from: mixmastermike on May 15, 2013, 07:23:25 AM
I wasn't able to record 1920 x 1080, with earlier builds the buffer kept filling up but doing some testing today with the May 14th build it seemed to sort it out. I didn't stress it out with leave's or trees or anything like that but the frame were busy and it handled it well till i hit the 4Gb limit.

Are you running the latest build?

I want to draw attention to something here. You talk about 'stressing stuff out'. You need to think of RAW as a whole different beast (from compressed formats such as h264). With RAW a 'frame' will always be exactly the same size(in Mb). This means that a highly detailed image will contain exactly the same amount of data as a low detail image (even a blank frame) There is NO compression going on at all so there is no 'stressing'. Whereas compression based formats (h264) vary in size (and of course quality). This is why there is a variability with the image sizes of files in h264.

Just to clarify this you might want to check the file sizes of your RAW (*.dng) frames. You will find that they are always the same size(of course dependent on the resolution). This is the double edged sword that is RAW. On the one hand it has the absolute best quality at the expensive of being so space hungry (I mean in MB/s). Once you can deal with the data rate you should be fine. In simple terms the data requirements are a constant.
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 15, 2013, 01:56:44 PM
Quote from: Kuky on May 15, 2013, 01:23:58 PM
What worflows do you guys use with the DNG files in Windows environment?

I found out that:

A. Davinci Resolve does not support them.
B. Adobe Speedgrade does not support them.
C. Adobe Lightroom it's ok, but only for simple tasks. For example if you do a highlight recovery and sync all dng files then export to a nle will result in flicker. Probably because although all files have exactly the same settings, internally LR will try to optimise the highlight recovery on a per file basis. (?).
D. AfterEffects through the ACR import. This will have best results. Once you set the interpretation on first file, it's smart enough to keep things consistent for the sequence.  Very, very slow.
E. Cineform Studio/Pro (also the dpx2cf tool) will recognise dng files and convert the sequence to a cineform raw file. You have to fiddle with demosaic (advanced detail 3 seems to work best) and all other settings. The fastest way to preview your movies. Converting it's ultrafast and the result is realtime playable. Fastest way to edit, until now.
F. uffraw batch. Tried once, but slow.

Any other ideas? It seems that cinemadng files and dng files are not *exactly* the same thing and didn't find a way to convert from one to another.

Regards,
Cristian


PS. If the raw2dng could convert directly to cinemadng would be awesome.

I open (all DNGS at once) with Photoshop (ACR 7.1), select all thumbnails on the left first and then correct white balance, tint, and anything else needed like shadows, lumminance, etc.  Then make sure all the thumbnails are selected and and click on "Save Image" on the bottom left, select other options like jpg or tiff.  If you select jpg then set quality to 12 or some artifacts will show.  Saving in tiff works a lot faster but it uses tons of space.  jpg is much slower but uses little space.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 15, 2013, 02:10:30 PM
Did a couple tests. Nothing particularly special here, I just splashed some water on tree leaves and shook them around a bit:


As expected, the quality is still butchered a bit in this low motion video. The first clip is in crop mode and the second one in normal mode. You may notice that the video "judders" a couple times near the end of each clip. Not sure what's causing it.

Testing out Mega for file hosting. Never tried it before, but let's see if it works. This one was arbitrarily encoded to BD specs and looks much, much better.
https://mega.co.nz/#!eJsWWARR!T_0A3AH4IW7Ec71kncXg5x1dV2ZenFu2o1OkXJ-hN-8

I'm not gonna post the lossless RGB source because this video isn't particularly interesting and that it's nearly 5 GB lol.
Title: Re: uncompressed 14-bit RAW video recording
Post by: vicnaum on May 15, 2013, 02:24:38 PM
Quote from: Andy600 on May 15, 2013, 10:57:45 AM
@vicnaum - can you rerun the benchmark with Global Draw OFF? It's already near the maximum for the 600d buffer but interested to see if there's anymore

Here it is:
(http://www.naumik.com/temp/scr3/20130515-llo-122kb.jpg)
SDHC 16GB Sandisk Extreme HD Video (UHS-I, 45MB/s*, 300X, 10 class) on 600D. Global Draw: OFF.

(Did a format in-camera with "Keep ML")

Looks the same. Not more than 22 MB/s

Btw, is there any FAQ here with all the info in one place? E.g. I want to know which resolutions on 600D are crop, and which are line-skip full-sensor. Want to do some tests on different resolutions.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ajay on May 15, 2013, 02:27:15 PM
For those who are interested in the performance of the Transcend 32GB, 1000x. I reran the test with global draw off and in liveview:

(http://iwishicouldfly.com/iwishicouldfly/images/VRAM0%20%282%29.BMP)

AJ
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 15, 2013, 02:34:12 PM
@Vicnaum - Thanks for the new benchmark. Ok, it's not really any different so it means we can probably use global draw without affecting bandwidth.

@Rush posted earlier in this thread regarding best settings for the 600d. Here's what he wrote:

The best you can with 600D (550D/60D) is: (limited by hardware SD transfer rates)
- 960х540, upscale it to 1280х720 (24 fps max)
- 1280х400, tight aspect 3.20:1 (24 fps max)
In Canon's 720p - lineskipping 1.66х, so you should upscale to restore correct ratio:
- 1280x400 upscales to 1280x664 - near to 16:9 (24 fps max)
- 1280x360 upscale to 1280x600 - 2.13:1 aspect
- 1280x320 upscale to 1280x533 - 2.40:1 cinemascope aspect

It is only 1280px wide, but pretty nice looking though :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 15, 2013, 02:34:42 PM
Quote from: ajay on May 15, 2013, 02:27:15 PM
For those who are interested in the performance of the Transcend 32GB, 1000x. I reran the test with global draw off and in liveview:

AJ

Interesting. So it can do 86 MB/s, but still can't quite do full 1080p? What does the raw recorder say when you set it that high? For the 5D2, it says it needs 81.2 MB/s for 1880x1080 and 86.6 MB/s for 1880x1152.
Title: Re: uncompressed 14-bit RAW video recording
Post by: coutts on May 15, 2013, 02:47:57 PM
Quote from: g3gg0 on May 15, 2013, 11:25:04 AM
its time for a file manager module ;)

who does it?
chdk has a file manager gui i think, I remember using it back in my point & shoot days.
Title: Re: uncompressed 14-bit RAW video recording
Post by: tackyattack on May 15, 2013, 02:56:46 PM
So it turns out the 650d (t4i) can support the high speed UHS-I cards!!!!
(http://www.photoplusmag.com/files/2013/01/Canon_memory_card_compatibility.png)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Rush on May 15, 2013, 03:03:11 PM
Quote from: Andy600 on May 15, 2013, 02:34:12 PM
@Vicnaum - Thanks for the new benchmark. Ok, it's not really any different so it means we can probably use global draw without affecting bandwidth.
I just checked what affects RAW recording - Magic Zoom, Histogram, general Global Draw are ok (i'm not sure if MZ makes corrupted frames or not - it looks messed when use while recording). But focus peaking or zebras have great impact on frameskipping :( , so try to avoid it when recording RAW

QuoteSo it turns out the 650d (t4i) can support the high speed UHS-I cards!!!!
Yes, and no. 40 mb/s maximum just like 6D - it is RAW 1280x720 max
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 15, 2013, 03:06:52 PM
Quote from: Yoshiyuki Blade on May 15, 2013, 02:10:30 PM
Did a couple tests. Nothing particularly special here, I just splashed some water on tree leaves and shook them around a bit:

As expected, the quality is still butchered a bit in this low motion video. The first clip is in crop mode and the second one in normal mode. You may notice that the video "judders" a couple times near the end of each clip. Not sure what's causing it.

Testing out Mega for file hosting. Never tried it before, but let's see if it works. This one was arbitrarily encoded to BD specs and looks much, much better.
https://mega.co.nz/#!eJsWWARR!T_0A3AH4IW7Ec71kncXg5x1dV2ZenFu2o1OkXJ-hN-8

I'm not gonna post the lossless RGB source because this video isn't particularly interesting and that it's nearly 5 GB lol.

Yoshiyuki,

What brand and specs of CF card are you using? You shot that on a 5d2 right?

Title: Re: uncompressed 14-bit RAW video recording
Post by: daimon on May 15, 2013, 03:17:44 PM
I just tried the new RAW feature after reading a lot about it, The detail is amazing!, thank you ML team.
The only problem I had are the vertical noise lines (more like pattern).
I uploaded a frame for you guys to check it out and tell me if I am doing something wrong.
https://skydrive.live.com/#cid=85F9A6A11BA54BA1&id=85F9A6A11BA54BA1%21105
Title: Re: uncompressed 14-bit RAW video recording
Post by: vicnaum on May 15, 2013, 03:20:26 PM
Quote from: Andy600 on May 15, 2013, 02:34:12 PM
@Vicnaum - Thanks for the new benchmark. Ok, it's not really any different so it means we can probably use global draw without affecting bandwidth.

@Rush posted earlier in this thread regarding best settings for the 600d. Here's what he wrote:

The best you can with 600D (550D/60D) is: (limited by hardware SD transfer rates)
- 960х540, upscale it to 1280х720 (24 fps max)
- 1280х400, tight aspect 3.20:1 (24 fps max)
In Canon's 720p - lineskipping 1.66х, so you should upscale to restore correct ratio:
- 1280x400 upscales to 1280x664 - near to 16:9 (24 fps max)
- 1280x360 upscale to 1280x600 - 2.13:1 aspect
- 1280x320 upscale to 1280x533 - 2.40:1 cinemascope aspect

It is only 1280px wide, but pretty nice looking though :)

Yes, saw that. Are these all partial sensor crop? Is there a full-sensor mode on 600d?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Rush on May 15, 2013, 03:24:32 PM
Quote from: vicnaum on May 15, 2013, 03:20:26 PM
Yes, saw that. Are these all partial sensor crop? Is there a full-sensor mode on 600d?
Yes, partial. Full sensor mode is only achieved with 1740x1160, but transfer rate is too high for SD controller to record 24p continuously.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 15, 2013, 03:25:20 PM
Quote from: ted ramasola on May 15, 2013, 03:06:52 PM
Yoshiyuki,

What brand and specs of CF card are you using? You shot that on a 5d2 right?

Yeah, its with a 5D2. I used a Sandisk Extreme Pro 16GB 90 MB/s (600x). The benchmark gave me 54-55 MB/s write so it's just enough to do 1880x720 in normal mode, 1920x720 in crop mode.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ajay on May 15, 2013, 03:29:02 PM
Yoshiyuki,

I'm not seeing those options (1880 x 1080 or 1880x1152). Is there something I need to set? I also have no idea how to use crop mode. Can you explain?

As far as 1920 x 1080, it states I need 82.9 MB/s and it works for a limited time and eventually stops or skips frames. 1920 x 960 now works w/o issues.

Thanks,

AJ

Quote from: Yoshiyuki Blade on May 15, 2013, 02:34:42 PM
Interesting. So it can do 86 MB/s, but still can't quite do full 1080p? What does the raw recorder say when you set it that high? For the 5D2, it says it needs 81.2 MB/s for 1880x1080 and 86.6 MB/s for 1880x1152.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 15, 2013, 03:41:38 PM
Has there been any development about removing 4gb limit?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 15, 2013, 03:41:54 PM
Quote from: ajay on May 15, 2013, 03:29:02 PM
I'm not seeing those options (1880 x 1080 or 1880x1152). Is there something I need to set? I also have no idea how to use crop mode. Can you explain?

Those are the limits of the 5D2 (well, it can go up to 1880x1250 in normal mode) so those exact resolutions probably won't show up on the 5D3.

I'm not sure if my vocabulary is correct, but by crop mode I mean the zoom mode. You press the AF point selection button, which is the upper-right-most button on the 5D2 (next to the star). I don't know where it is on the 5D3. These buttons also double as a zoom in/out feature.

At the moment, the zoom mode is stuck at ~30 fps, but can be changed with FPS override. Once you're zoomed in and have the desired FPS set, you can record RAW from there.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 03:46:24 PM
My fps override works in zoom on 6D. Haven't tried 600D.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 15, 2013, 04:00:11 PM
Quote from: daimon on May 15, 2013, 03:17:44 PM
I just tried the new RAW feature after reading a lot about it, The detail is amazing!, thank you ML team.
They only problem I had are the vertical noise lines (more like pattern).
I uploaded a frame for you guy to check it out.
https://skydrive.live.com/#cid=85F9A6A11BA54BA1&id=85F9A6A11BA54BA1%21105

That's the fixed pattern noise from the sensor. Shoot a few seconds with the body cap on and push it 3-4 stops and you'll see exactly what it looks like. The Adobe RAW noise reducer is you friend.
Title: Re: uncompressed 14-bit RAW video recording
Post by: daimon on May 15, 2013, 04:28:16 PM
Quote from: squig on May 15, 2013, 04:00:11 PM
That's the fixed pattern noise from the sensor. Shoot a few seconds with the body cap on and push it 3-4 stops and you'll see exactly what it looks like. The Adobe RAW noise reducer is you friend.

Thanks a lot I will give it a try  :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 15, 2013, 04:35:14 PM
Has there been any development about removing 4gb limit?

Has anyone tried this raw shooting with a hdmi monitor? I assume it won't affect anything, but want to confirm it if anyone tried.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 04:39:45 PM
Might lower performance from using another edmac. Try it.
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 15, 2013, 04:42:30 PM
So, everyone keeps mentioning crop and normal mode.
if I record with any 720 settings that would be a crop?
what are the true HD settings that I can record with?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 04:44:44 PM
Video mode selects how big the raw image is.

Then you crop whatever you want from that.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Audionut on May 15, 2013, 04:48:16 PM
Is 5x or 10x zoom a 1:1 crop of the sensor?
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 15, 2013, 04:52:36 PM
Quote from: Brian@202020 on May 15, 2013, 01:10:09 PM
That's sarcasm I hope. Regardless of whether the 5D3 is shooting H264 or RAW it's still line skipping. It's very apparent in the RAW shots posted, except the 1-1 stuff. That said it's good enough for most people in most situations, I just strive for perfection. Magic Lantern is one of the keys, and Mosaic Engineering is the other.

If you want perfection get an Arriflex, I'm happy with satisfaction.

Pixel peep for aliasing till your eyes bleed.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 05:02:47 PM
QuoteIs 5x or 10x zoom a 1:1 crop of the sensor?

Its pretty close.. you only get 1/2-1/3 of the height. I guess it would add up to the whole thing.
Title: Re: uncompressed 14-bit RAW video recording
Post by: TjS on May 15, 2013, 05:28:40 PM
hi guys - first of all big thanks to the whole ML team. Pretty incredible what you guys pulled off!!

I shot some tests today with 5d mk3 and the Lexar 64GB 1000x and have a couple questions/things that I noticed.

First of all here are two benchmark tests:

ML v2.3.NEXT.2013May13.5d3113
(http://uploadpic.org/storage/2011/CKtW77G2gQdwU5KLEtHF1emHO.jpg)

ML v2.3.NEXT.2013May14.5d3113
(http://uploadpic.org/storage/2011/DiGSxsaT3Mlx77Ngwu270pbX.jpg)

They seem fairly similar.

Now - with the May13 version I can run 1920x1080 at 23.976fps without skipping any frames - it is recording at around 83mbps.
While the May14 version 1920x1080 is only recording at around 51mbps and skips a lot of frames. I need to go down to 1920x840 to actually get a clean recording. 
That seems a bit odd or? Not sure if thats already a common problem - but I  thought it would be good to mention.

Other problems are:
- Sometimes (very randomly) its not possible to stop the recording. In the moment that I want to end the "Recording..." and press SET, it changes to "Stopping..." but then freezes and keeps recording.
- Also randomly when I press record it seems to start recording, the "Buffer usage: <* >" shows up, but nothing is happening. In that case it always creates a very small file thats a - i think, need to confirm though - 192kb .RAW file.

Cheers!
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 15, 2013, 05:37:12 PM
Somebody else did mention that May 14 was dropping frames. I'm still on May 13.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 05:38:18 PM
Ok, I put minimal buffers to 4 and now 6D allocates 4... but the last one is only 18MB.. there is 32MB still free but largest contiguous is like 1mb? what gives... is there  a way to reset the suite? It will allocate up to 6 but then it screws up because the latest buffers end up smaller and smaller.

OK, here is an idea... can we allocate the smaller double buffer AFTER allocating the other ones?
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 15, 2013, 05:43:17 PM
The tearing fix is eating quite a bit of RAM. Exmem is not yet fully understood.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 15, 2013, 05:48:17 PM
Quote from: a1ex on May 15, 2013, 05:43:17 PM
The tearing fix is eating quite a bit of RAM. Exmem is not yet fully understood.

That explains! Thanks.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 06:03:24 PM
But there is the 18MB at the end.. .only needs 10... so if we were to do 30-30-30-30 and then the 18 is left over there is enough for the double buffer. Right now its 30 getting taken for the buffer and 18 left over at the end?

*my write speed takes a dive when it gets into that last buffer but with only 3 it was doing much worse. It would hit that 3rd one and then start skipping.
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 15, 2013, 06:06:46 PM
Then alloc the fullsize at the end. On 5D3 this doesn't work, because it has 4x32M and a bunch of tiny ones.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 15, 2013, 06:12:30 PM
macgregor wrote "since there's no metadata on the raw files, there are no color profiles asigned to the images. Camera raw I suspect is using the standard adobe color profile, which in my opinion sucks. Canon profiles are much better. Adobe over saturates blues and skintones are less nice. So I wonder if the ML guys could apply the canon profiles to the dng so we could fix this. We could even use VSCO film raw picture profiles, for some extra fun."
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 06:24:49 PM
When I try placing
fullsize_buffers[0] = shoot_malloc(10*1024*1024); /* any image size bigger than that? */
   
after allocating all of the buffers I get a memory error.
Title: Re: uncompressed 14-bit RAW video recording
Post by: eatstoomuchjam on May 15, 2013, 06:26:02 PM
Quote from: bumkicho on May 15, 2013, 04:35:14 PM
Has anyone tried this raw shooting with a hdmi monitor? I assume it won't affect anything, but want to confirm it if anyone tried.

I just shot a 4G clip with an external monitor plugged in.  ML reported 8 skipped frames at 1080p on my Lexar 1000x card which is the same as what it reports when I'm on the internal screen.  I haven't looked at the footage, but I'd expect it to be fine.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 15, 2013, 06:32:23 PM
Quote from: eatstoomuchjam on May 15, 2013, 06:26:02 PM
I just shot a 4G clip with an external monitor plugged in.  ML reported 8 skipped frames at 1080p on my Lexar 1000x card which is the same as what it reports when I'm on the internal screen.  I haven't looked at the footage, but I'd expect it to be fine.

I just tried it myself and yes, it seems to work fine except for one. Did you notice oddity on frame overlay when shooting 1080p? It looks way off center. Although footage looks good.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Jonas on May 15, 2013, 06:33:50 PM
Would it be silly to think we could make a CF to CF cable and then use an apple camera connection kit to transfer the files directly to an ipod/iPhone/ipad?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 06:35:30 PM
I think would be better if it used shoot malloc contig for all buffers and just use the last smallest one for the double buffer.

1/2 working.. now get 4x32 buffers then freeze. Error handling is why it wasn't working... doh

Hmm... I think it has to do allocate contiguous for all buffers and then use the last smallest one. I can't get it to use shoot_malloc after running the allocation loop.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Dash on May 15, 2013, 06:37:13 PM
You have RAW files. You can apply (create) any color profile you want on your favourite RAW software after all.
I think there are more important things to do right now.


Quote from: squig on May 15, 2013, 06:12:30 PM
macgregor wrote "since there's no metadata on the raw files, there are no color profiles asigned to the images. Camera raw I suspect is using the standard adobe color profile, which in my opinion sucks. Canon profiles are much better. Adobe over saturates blues and skintones are less nice. So I wonder if the ML guys could apply the canon profiles to the dng so we could fix this. We could even use VSCO film raw picture profiles, for some extra fun."
Title: Re: uncompressed 14-bit RAW video recording
Post by: JohnBarlow on May 15, 2013, 06:54:13 PM
My first post, so my thanks and congratulations go out to all the devs involved in bringing this home.

I saw this post


Quote from: tackyattack on May 15, 2013, 04:03:24 AM
I don't think the UHS-I pins on the cards are supported by the Canon DSLRs, only the normal interface. So you're still going to get the normal SD card speeds whether it's UHS or not. I have been designing a device that plugs into the SD card slot of the DSLR and accesses the SPI pins. An ARM chip then emulates an SD card with data storage on a FAT32 partitioned SSD. This would allow very high data speeds. That will erase the problems of the SD card being too slow, but I have no idea how fast the SPI Bus on the canon cameras can go. Does anybody know of the max mhz the bus can run at?

Thanks

and I also would like to vent an idea with all concerned.

For starters check out this vid--->
http://www.youtube.com/watch?feature=endscreen&NR=1&v=qgwTCn_X8Wg

This geeza managed to replace the SD Slot controller board on a 450D with complete instructions. At time 3:16 you can see the board, which by the way only costs $30 from Canon spares.

This got me thinking - what if the board from a 700D has a faster transfer rate and can be used in the 550D/600D etc ??? OR what if a cable can be hooked up to an exterior device like a CF or SSD????

I dont have the skills but a hardware guru could in principle cook something up.

Share your thoughts......

Title: Re: uncompressed 14-bit RAW video recording
Post by: deleted.account on May 15, 2013, 06:56:10 PM
Quote from: squig on May 15, 2013, 06:12:30 PM
macgregor wrote "since there's no metadata on the raw files, there are no color profiles asigned to the images. Camera raw I suspect is using the standard adobe color profile, which in my opinion sucks. Canon profiles are much better. Adobe over saturates blues and skintones are less nice. So I wonder if the ML guys could apply the canon profiles to the dng so we could fix this. We could even use VSCO film raw picture profiles, for some extra fun."

There's no color profile assigned because its raw, color primaries and therefore gamut are not even defined, when raw is 'developed' in whatever app used you define the color space, dcraw for example lets you develop as XYZ, Prophoto, Adobe RGB or sRGB, lets you choose any gamma to be applied, so use a color managed app, define your working color space, choose different working spaces, import dng's, compare histograms and channel clipping, go from there.

I believe for example Apple products choose ProPhoto by default if no color space is chosen.

Another thing dcraw will do is let you set sensor saturation default which varies with camera model and manufacturer to avoid the magenta tint.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kuky on May 15, 2013, 07:13:41 PM
Quote from: squig on May 15, 2013, 06:12:30 PM
macgregor wrote "since there's no metadata on the raw files, there are no color profiles asigned to the images. Camera raw I suspect is using the standard adobe color profile, which in my opinion sucks. Canon profiles are much better. Adobe over saturates blues and skintones are less nice. So I wonder if the ML guys could apply the canon profiles to the dng so we could fix this. We could even use VSCO film raw picture profiles, for some extra fun."

Colour profiles have nothing to do with raw formats. When you transcode to an RGB format you can assign a colour profile to it.

But what can you do is make camera calibration (dcp file) by shooting a colourchecker passport card (macbeth). But soon you'll find out that LR doesn't let you assign the dcp file to the dng files created by the raw module. That is happening because LR doesn't recognise the DNG as being created with a Canon 5D (or whatever). LR thinks it is clever and doesn't let you apply custom camera calibrations.

But RawTherapee will let you do that (much clever program to decode raw) and you end up with a colour managed shooting with the Canon RAW.

Regards,
Cristian
Title: Re: uncompressed 14-bit RAW video recording
Post by: VectorZ on May 15, 2013, 07:14:48 PM
The community (Or Canon) should buy you guys a C300.

This would actually turn the C300 into a professional Tool.

Title: Re: uncompressed 14-bit RAW video recording
Post by: tackyattack on May 15, 2013, 07:16:36 PM
Quote from: JohnBarlow on May 15, 2013, 06:54:13 PM
My first post, so my thanks and congratulations go out to all the devs involved in bringing this home.

I saw this post


and I also would like to vent an idea with all concerned.

For starters check out this vid--->
http://www.youtube.com/watch?feature=endscreen&NR=1&v=qgwTCn_X8Wg

This geeza managed to replace the SD Slot controller board on a 450D with complete instructions. At time 3:16 you can see the board, which by the way only costs $30 from Canon spares.

This got me thinking - what if the board from a 700D has a faster transfer rate and can be used in the 550D/600D etc ??? OR what if a cable can be hooked up to an exterior device like a CF or SSD????

I dont have the skills but a hardware guru could in principle cook something up.

Share your thoughts......

Very interesting...

I'm planning on running a speed test on the 550d SPI bus to see what I can get. If it can sustain a very high mhz, I might have a few ideas. One being able to record using a computer as a storage device via a ethernet cable. So you'd plug this small SD card connector into the card slot of the camera, then it'd go to a small conversion board, then ethernet to the computer. Much like the SI-2k camera.

Also, I think CF cards communicate with an ATA interface. I think both parallel and serial. So if we could create a connector, it could be very simple to slip it into a CF card camera (5d/7d) and have it connect to an SSD with a PATA/SATA interface. Cheaper, faster, larger drives!
Title: Re: uncompressed 14-bit RAW video recording
Post by: payne on May 15, 2013, 07:21:08 PM
Quote from: Rush on May 15, 2013, 03:24:32 PM
Yes, partial. Full sensor mode is only achieved with 1740x1160, but transfer rate is too high for SD controller to record 24p continuously.

Two ideas:

1) Is there anyway to bypass the SD controller to get closer to HD resolutions out of these APS-C cameras that so many of us have (60D, T2i, 600D)? Could we send out data via HDMI? This would remove the speed bottlenecks for at least 1080p out.

2) Is it conceivable in a future update to apply a 2:1 or 3:1 compression of some kind in camera so that we can lower the data rate if the SD controller is our only way out?

I'm very worried that a mentality will set in to abandon work and hope on the vast majority of these Canon DSLRs that most of us can afford. I'm a 60D guy myself due to budget and when the camera needed to be purchased.
Title: Re: uncompressed 14-bit RAW video recording
Post by: eatstoomuchjam on May 15, 2013, 07:23:49 PM
Quote from: bumkicho on May 15, 2013, 06:32:23 PM
I just tried it myself and yes, it seems to work fine except for one. Did you notice oddity on frame overlay when shooting 1080p? It looks way off center. Although footage looks good.

I didn't notice anything weird at 1080p, but I set it down to 1920x720 and the overlay was way off center, yeah.
Title: Re: uncompressed 14-bit RAW video recording
Post by: coutts on May 15, 2013, 07:28:09 PM
Quote from: JohnBarlow on May 15, 2013, 06:54:13 PM
This got me thinking - what if the board from a 700D has a faster transfer rate and can be used in the 550D/600D etc ??? OR what if a cable can be hooked up to an exterior device like a CF or SSD????

and who will write the new firmware for DryOS to communicate with the changed slot?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 07:53:40 PM
The slot probably just has power stuff... I doubt it has the IO chip on it.. also IO lines are limited to the MHZ that the SD runs at.

Better off messing with the USB... it has UHS theoretical write, facilities are there for the WFT stuff. Just someone has to figure out if its possible thru camera port or grip.
Title: Re: uncompressed 14-bit RAW video recording
Post by: JohnBarlow on May 15, 2013, 07:54:46 PM
Quote from: coutts on May 15, 2013, 07:28:09 PM
and who will write the new firmware for DryOS to communicate with the changed slot?

I appreciate what your sayin, but maybe the existing code just needs a tweak? No one really know at this juncture.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 08:01:07 PM
I know... I read the datasheet for the IO chip trying to overclock the SD slot as is done on some android. Data integrity be damned :)

You'd honestly be better off finding the traces for CF. The code for CF and SD is inside the FW for all cameras and that USB hard disk stuff too... These all seem like better avenues than trying to do something thats not possible. Even with UHS the max speed  we're getting is like 40MB/s.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 15, 2013, 08:01:46 PM
I also do experiments with RAW
This is very interesting, but I need a fast flash drive!
and some video:
http://www.youtube.com/watch?v=ONIKL8s6uVw
http://www.youtube.com/watch?v=OPuVHyEyc6U
http://www.youtube.com/watch?v=NbCnVhckcYg
Title: Re: uncompressed 14-bit RAW video recording
Post by: xaled on May 15, 2013, 08:21:22 PM
@tackyattack
Look at the raw to ssd thread, there is some info on cf adapters.

Tho the wft usb path seams to be more promising.
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 15, 2013, 09:07:58 PM
Does anyone know why I continue getting this error when I boot the camera(5D3)
camera was not shut down cleanly. skipping module loading

I get that message after I boot the camera. but it only happens after I've recorded a raw video, then shut the camera down.
Only way I know to get rid of the message is to format the card and reinstall ML
Title: 600D 960x540 video is promising (pending tearing and blocking fixes)
Post by: mattmvpmedia on May 15, 2013, 09:13:01 PM
I'm using a SanDisk Extreme Pro 95MB/s 16GB card on my 600D (T3i). When shooting 960x540 raw I get a few dropped frames but not bad. I shot a comparison test of a scene with 1. Normal Canon 1080p h.264 quicktime movie settings and then 2. Magic Lantern cropped sensor 960x540 raw. I used raw2dng on Mac and then After Effects to process the dng files for the 960x540 and exported a ProResHQ movie.

There is currently too much tearing and blocking of magenta blocks for the raw footage to be usable, but the detail in the frames that don't have the tearing is an incredible improvement over the h.264 quicktime movie. Even when I blow up the 960x540 ProRes to 1920x1080 and compare it with the standard Canon 1920x1080 h.264 file the footage shot 960x540 raw blows the h.264 away in terms of detail and dynamic range. Love it.

I'm definitely looking to get a 5D mark iii now, but if the tearing and blocking can be fixed on the 600D it is a huge improvement to what it currently shoots in h.264, even when shooting 960x540 raw. Fantastic job by the ML team! 
Title: Re: 600D 960x540 video is promising (pending tearing and blocking fixes)
Post by: ashtrai on May 15, 2013, 09:17:36 PM
Quote from: mattmvpmedia on May 15, 2013, 09:13:01 PM
I'm using a SanDisk Extreme Pro 95MB/s 16GB card on my 600D (T3i). When shooting 960x540 raw I get a few dropped frames but not bad. I shot a comparison test of a scene with 1. Normal Canon 1080p h.264 quicktime movie settings and then 2. Magic Lantern cropped sensor 960x540 raw. I used raw2dng on Mac and then After Effects to process the dng files for the 960x540 and exported a ProResHQ movie.

There is currently too much tearing and blocking of magenta blocks for the raw footage to be usable, but the detail in the frames that don't have the tearing is an incredible improvement over the h.264 quicktime movie. Even when I blow up the 960x540 ProRes to 1920x1080 and compare it with the standard Canon 1920x1080 h.264 file the footage shot 960x540 raw blows the h.264 away in terms of detail and dynamic range. Love it.

I'm definitely looking to get a 5D mark iii now, but if the tearing and blocking can be fixed on the 600D it is a huge improvement to what it currently shoots in h.264, even when shooting 960x540 raw. Fantastic job by the ML team!

Can you post the ProRes sample on mega?
Title: Re: 600D 960x540 video is promising (pending tearing and blocking fixes)
Post by: payne on May 15, 2013, 09:36:50 PM
Quote from: mattmvpmedia on May 15, 2013, 09:13:01 PM
I'm using a SanDisk Extreme Pro 95MB/s 16GB card on my 600D (T3i). When shooting 960x540 raw I get a few dropped frames but not bad. I shot a comparison test of a scene with 1. Normal Canon 1080p h.264 quicktime movie settings and then 2. Magic Lantern cropped sensor 960x540 raw. I used raw2dng on Mac and then After Effects to process the dng files for the 960x540 and exported a ProResHQ movie.

There is currently too much tearing and blocking of magenta blocks for the raw footage to be usable, but the detail in the frames that don't have the tearing is an incredible improvement over the h.264 quicktime movie. Even when I blow up the 960x540 ProRes to 1920x1080 and compare it with the standard Canon 1920x1080 h.264 file the footage shot 960x540 raw blows the h.264 away in terms of detail and dynamic range. Love it.

I'm definitely looking to get a 5D mark iii now, but if the tearing and blocking can be fixed on the 600D it is a huge improvement to what it currently shoots in h.264, even when shooting 960x540 raw. Fantastic job by the ML team!

Do we know if this build will run on the 60D as well or are the 600D and 60D (although very very similar) not similar enough on the inside?
Title: Re: uncompressed 14-bit RAW video recording
Post by: sicetime on May 15, 2013, 09:41:35 PM
is there a place that has compiled all different cards against the resolution they are capable of reaching (with out dropped frames)?
Title: Re: uncompressed 14-bit RAW video recording
Post by: tonybeccar on May 15, 2013, 09:47:53 PM
Has anyone tried with the Toshiba 1066x Excercia Pro compact flash cards?? They're supposed to be 30 MB/s faster! They say they're prepared to record 4k video...
Title: Vimeo links to 600D 960x540 raw versus standard h.264 1080p comparison
Post by: mattmvpmedia on May 15, 2013, 10:08:55 PM
Here's the standard Canon h.264 1080 24p file:


Here's the 960x540 raw file (shot with ML raw, converted to DNG files, processed with Adobe AE, exported to ProResHQ).

Title: Updated Vimeo links to 600D files
Post by: mattmvpmedia on May 15, 2013, 10:17:01 PM
Sorry about that, forgot to remove the "s" from https -

Here's the standard Canon h.264 1080 24p file:


Here's the 960x540 raw file (shot with ML raw, converted to DNG files, processed with Adobe AE, exported to ProResHQ).
Title: Re: uncompressed 14-bit RAW video recording
Post by: Vegandelight on May 15, 2013, 10:18:02 PM
Quote from: tonybeccar on May 15, 2013, 09:47:53 PM
Has anyone tried with the Toshiba 1066x Excercia Pro compact flash cards?? They're supposed to be 30 MB/s faster! They say they're prepared to record 4k video...

I don't think you'll be able to break 100mb/s with that card either, it's hardware related limit at the moment
Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 15, 2013, 10:23:07 PM
Sell your 600D's and save up for Mark III's - the whole thing is jarring lol
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 10:24:50 PM
Try some different resolutions and modes.. You seem to be getting more dropped frames than I am. But this middle is gone dealie is happening on 5d2 as well sometimes. Simply it has to mature a bit more...
Title: Re: uncompressed 14-bit RAW video recording
Post by: ajay on May 15, 2013, 10:29:04 PM
Can someone please set me straight....I have been trying to get 1:1 cropping while recording RAW and I have scoured everywhere trying to figure out how to do this so I don't have to bug people, but I cannot for the life of me figure out how to record RAW in anything other than full-frame. What do I need to do to crop/pan while recording?

Sorry to be a bother...

AJ
Title: Re: uncompressed 14-bit RAW video recording
Post by: Dash on May 15, 2013, 10:31:11 PM
4k. But not in RAW mode.

Quote from: tonybeccar on May 15, 2013, 09:47:53 PM
Has anyone tried with the Toshiba 1066x Excercia Pro compact flash cards?? They're supposed to be 30 MB/s faster! They say they're prepared to record 4k video...
Title: Re: uncompressed 14-bit RAW video recording
Post by: marcb on May 15, 2013, 10:33:49 PM
Rob Galbraith has posted a 57.5MB/s write speed for the Canon 7D with FW 1.2.5 and the Lexar Professional 1000X 32GB CF. I'm curious to see what kind of RAW video capabilities the 7D could offer with Magic Lantern. Has anyone achieved higher write speeds on the 7D? http://www.robgalbraith.com/bins/camera_wb_multi_page.asp?cid=6007-10294
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 15, 2013, 10:40:46 PM
7d probably == 5d2


But g3ggo said the raw image is on the other proc so harder to get at.. which is why its not done.
Title: Re: uncompressed 14-bit RAW video recording
Post by: jpgentry on May 15, 2013, 10:42:40 PM
Hi Guys.  A few observations if you can confirm them:

It seems that on all cameras/card combinations frame skipping is an issue including the 5d3.  I have the 5d3 with 128GB 1000x KomputerBay CF card.  I start skipping frames about 10 seconds in at 1920x1080.  I also skip frames at: 1920x960, 1920x840.  I do not skip frames at 1920x720.  I'm still wondering how others are reporting that they are recording 1920x1080 video on the 5d3.  I'm thinking the announcements around the internet are premature about 1080p RAW on the 5d3 being a reality.

The encoding of RAW into a finalized usable clip is very slow and tedious.  While the workflow may improve, I don't see the encoding time improving drastically when working with large amounts of footage.

Is the development process showing signs of cracking these issues?  Are we even sure that we're not bumping into limitations of the camera controller on the 5d3 causing frame skipping?  I guessuntil someone gets the 1066 card we will not know...

I've been holding back on the c100 until I hear where this is going, but this does not yet seem quite the holy grail that Cinema5d and nofilmschool are putting out on the street... 
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 15, 2013, 10:52:12 PM
Quote from: jpgentry on May 15, 2013, 10:42:40 PM
Hi Guys.  A few observations if you can confirm them:

It seems that on all cameras/card combinations frame skipping is an issue including the 5d3.  I have the 5d3 with 128GB 1000x KomputerBay CF card.  I start skipping frames about 10 seconds in at 1920x1080.  I also skip frames at: 1920x960, 1920x840.  I do not skip frames at 1920x720.  I'm still wondering how others are reporting that they are recording 1920x1080 video on the 5d3.  I'm thinking the announcements around the internet are premature about 1080p RAW on the 5d3 being a reality.

The encoding of RAW into a finalized usable clip is very slow and tedious.  While the workflow may improve, I don't see the encoding time improving drastically when working with large amounts of footage.

Is the development process showing signs of cracking these issues?  Are we even sure that we're not bumping into limitations of the camera controller on the 5d3 causing frame skipping?  I guessuntil someone gets the 1066 card we will not know...

I've been holding back on the c100 until I hear where this is going, but this does not yet seem quite the holy grail that Cinema5d and nofilmschool are putting out on the street...

Have you tried with an older firmware?
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 15, 2013, 11:03:25 PM
Quote from: jpgentry on May 15, 2013, 10:42:40 PM
Hi Guys.  A few observations if you can confirm them:

It seems that on all cameras/card combinations frame skipping is an issue including the 5d3.  I have the 5d3 with 128GB 1000x KomputerBay CF card.  I start skipping frames about 10 seconds in at 1920x1080.  I also skip frames at: 1920x960, 1920x840.  I do not skip frames at 1920x720.  I'm still wondering how others are reporting that they are recording 1920x1080 video on the 5d3.  I'm thinking the announcements around the internet are premature about 1080p RAW on the 5d3 being a reality.

The encoding of RAW into a finalized usable clip is very slow and tedious.  While the workflow may improve, I don't see the encoding time improving drastically when working with large amounts of footage.

Is the development process showing signs of cracking these issues?  Are we even sure that we're not bumping into limitations of the camera controller on the 5d3 causing frame skipping?  I guessuntil someone gets the 1066 card we will not know...

I've been holding back on the c100 until I hear where this is going, but this does not yet seem quite the holy grail that Cinema5d and nofilmschool are putting out on the street...
Your card is too slow. Thats a cheaper card which really doesnt have the write speed that the Lexar(I believe) has
Title: Re: uncompressed 14-bit RAW video recording
Post by: pravdomil on May 15, 2013, 11:05:51 PM
Today I have been playing with raw_rec and there is result.
(http://uploadpic.org/storage/2011/thumb_0jtDYY48CC0uIFtRTedrsbLv6.png) (http://uploadpic.org/v.php?img=9E3TcHSqiI)
I can send you module file, if you wish (no warranty). You don't want to see source.

But I cannot compute seconds dividing frames by fps. Time goes faster. Any ideas?
Nice work!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 15, 2013, 11:07:41 PM
@mattmvpmedia - Thanks for posting the comparisons. To my eyes there is hardly anything in it. I think anyone criticizing the hardware limitations of these lower-end cameras should remember where we were a week ago. We (600d users) now have massively increased post production flexibility and virtually no loss in quality. I'll take that anyday! :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: payne on May 15, 2013, 11:17:49 PM
Quote from: Danne on May 15, 2013, 10:52:12 PM
Have you tried with an older firmware?

I second that question. I had heard that the older firmware Andrew is using allows 1920x1280.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Vegandelight on May 15, 2013, 11:51:17 PM
I find it really hard to know what im actually shooting at 5x and 10x. The frame isnt correct i guess cause im not getting what i think i am ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Nspa32 on May 15, 2013, 11:51:51 PM
Can someone please let me know if we can get this working on the 600D would it also work on the 550D? I mean, why wouldn't it? Aren't the internals identical?
Title: Re: uncompressed 14-bit RAW video recording
Post by: mixmastermike on May 15, 2013, 11:56:14 PM
Quote from: aaphotog on May 15, 2013, 11:03:25 PM
Your card is too slow. Thats a cheaper card which really doesnt have the write speed that the Lexar(I believe) has

Yeah i think its becoming evident that the "cheapo" Komputerbay cards are not giving write speeds the same as Lexar.

I believe in trying to get the best deal out there but your card is where you store your work, without it you have nothing. I don't want to even have the worry of a card going down thats why Im willing to pay a little more fore the Lexar.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 16, 2013, 12:12:43 AM
Quote from: mixmastermike on May 15, 2013, 11:56:14 PM
Yeah i think its becoming evident that the "cheapo" Komputerbay cards are not giving write speeds the same as Lexar.

I believe in trying to get the best deal out there but your card is where you store your work, without it you have nothing. I don't want to even have the worry of a card going down thats why Im willing to pay a little more fore the Lexar.

I,m not so sure about that. Unless some more tests we can,t exclude that it could be firmware related.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Vegandelight on May 16, 2013, 12:16:53 AM
Quote from: Nspa32 on May 15, 2013, 11:51:51 PM
Can someone please let me know if we can get this working on the 600D would it also work on the 550D? I mean, why wouldn't it? Aren't the internals identical?

It's been said it probably wont work. Read a few pages back.
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 16, 2013, 12:42:55 AM
Quote from: jpgentry on May 15, 2013, 10:42:40 PM
Hi Guys.  A few observations if you can confirm them:

It seems that on all cameras/card combinations frame skipping is an issue including the 5d3.  I have the 5d3 with 128GB 1000x KomputerBay CF card.  I start skipping frames about 10 seconds in at 1920x1080.  I also skip frames at: 1920x960, 1920x840.  I do not skip frames at 1920x720.  I'm still wondering how others are reporting that they are recording 1920x1080 video on the 5d3.  I'm thinking the announcements around the internet are premature about 1080p RAW on the 5d3 being a reality.

The encoding of RAW into a finalized usable clip is very slow and tedious.  While the workflow may improve, I don't see the encoding time improving drastically when working with large amounts of footage.

Is the development process showing signs of cracking these issues?  Are we even sure that we're not bumping into limitations of the camera controller on the 5d3 causing frame skipping?  I guessuntil someone gets the 1066 card we will not know...

I've been holding back on the c100 until I hear where this is going, but this does not yet seem quite the holy grail that Cinema5d and nofilmschool are putting out on the street...


I just ordered and received the same CF card as you... the KomputerBay 128gb 1000x CF card and have been running tests with each of the various Raw builds over the last 3 days.  The latest build for 5D3 someone just posted from today seems to be the best in terms of speed... see http://www.magiclantern.fm/forum/index.php?topic=5413.25 (Reply #30).  This includes some small memory tweaks that definitely helped me get a little bit longer runtimes as well as moved me up to the next resolution without dropped frames.

Here has been my experience with the KomputerBay 128 GB 1000x CF card...

For my tests, I tried Global draw turned ON and OFF and there was no significant difference except maybe a second or two extra of recording time but the extra time was not consistent.  So I decided to leave Global Draw = ON (with just peaking and crop marks enabled).


My tests using the ML benchmark utility shows read speeds that range anywhere from 52MB/s to 72MB/s.  Using CrystalDiskMark and ATTO Disk Benchmark, I get similar write speeds so my particular card seems to have a write speed that varies a bit.  I would say the average write speed that I get most of the time is about 65MB/s which is completely in line with the ML benchmarks and the listed speed needed by each resolution listed under the RAW video section of ML.  So I would trust the ML benchmarks and recommended write speeds for each resolution.  I have also confirmed that at least one other person is getting similar write speed using this card.

That said, the buffer/memory tweak Alex made today did have a significant impact when using my card. 

Here are my tests Before today's build...
1920 x 720 or less resolution (no skipped frames for complete 4gb file)
1920 x 840 (no skipped frames for complete 4 gb file)... before today's build, this was the best I could do.
1920 x 900 (no skipped frames for complete 4 gb file)... tweaks added to today's build allowed me to use resolution for first time
1920 x 960 (maybe 10 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (maybe 1-2 seconds before frame skipping begins... unusabe beyond that)

After using today's build...
1920 x 900 (no skipped frames for complete 4 gb file) before today's build, I could not use this resolution
1920 x 960 (went from 10 seconds to 20 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (went from 1-2 seconds to 5 seconds before frame skipping begins... unusabe beyond that)

So the tweaks made by Alex today have definitely helped so I encourage further memory/buffer optimizations... they are making a difference.

I don't know how EOSHD, Nuemann Films, and Cinema 5D are getting 1080P and greater without dropped frames. They say they are using the same card.  Maybe some of these cards are getting faster write times.  Or maybe they are just getting that resolution for a limited time but a bit longer than me before frame skipping appears.  If any of them can chime in here to clarify, it would helpful for those considering the Komputerbay cards.  This card is by far the most realistic in terms of pricing for the vast majority of people who want to take advantage of this RAW update and plan on using it beyond test videos and personal work.  None of the other cards comes close in terms of price/gb.  The other solutions by Lexar and Toshiba (soon) are more like $650 for 128gb vs $178 or KomputerBay. I'm sure this will eventually change, but that could take a year or longer before cheaper alternatives are available.  But there is no getting around the fact that the less than ideal write speed of these cards may prevent continuous 1080p for some (depending on their particular card?).  But continued memory optimization and other techniques by the ML team may be able to extend 1080p recording from 5 seconds to 30 seconds or possibly more when using this card.  If that can be attained, the vast majority of folks will be able use this resolution in many shooting scenarios, short of documentary or event work.  I do realize that smaller resolutions can be scaled up in post nicely, but I tend to avoid that as do many others I'm sure.

Lastly, on the topic of workflow mentioned above... it's completely worth the time as far as I'm concerned.  The difference is night and day.  There are a lot of test videos coming out now, some of which don't fully capture the quality improvement gained here.  EOSHD, Nuemann Films, and Cinema 5D did great job of showing what's possible.  Just wait until some shorts and features by other skillful DOPs start to appear online.  While I don't love the added work required in post, I find it impossible to go back to H.264 after getting used to the look of my footage now...it's improved that much.

Thank you Alex and team for this... also to g3gg0 who apparently had a huge role in making this particular breakthrough happen.




Title: Re: uncompressed 14-bit RAW video recording
Post by: Nspa32 on May 16, 2013, 12:58:53 AM
Quote from: Vegandelight on May 16, 2013, 12:16:53 AM
It's been said it probably wont work. Read a few pages back.

I don't understand why. If it's ported to the 600D and maybe the 7D, what makes it not usable on the T2i? I mean, doesn't the 600D have almost the exact same internals as the T2i? I don't get it. Someone please chime in here before I decide if I need to buy a new camera.

Also, good job to the people that made this happen. Amazing work.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 01:14:53 AM
@A1ex - Just trying out your new raw_rec.mo (from the 5d3). It's helping a lot on the 600d. Longer record times with hardly any corrupt frames and I'm still using a poorly performing card (10MB/s max).
Title: Re: uncompressed 14-bit RAW video recording
Post by: mattmvpmedia on May 16, 2013, 01:17:58 AM
Andy600 - forgive my newb question, but did you have to compile a build using the new raw_rec.mo for use with the 600D, or is it simple to just add it into the modules folder?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Rush on May 16, 2013, 01:20:42 AM
I agree, last changes are very good! Thanks, a1ex!

Can I ask to add 434px height?
1280x434 will work for 10-20 sec max I think, but it will result in nice 16:9 1280x720 after restoring correct aspect ratio (when recording in 720p mode of 600D). I think that it will be my favorite resolution for 600D.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 01:27:31 AM
Quote from: mattmvpmedia on May 16, 2013, 01:17:58 AM
Andy600 - forgive my newb question, but did you have to compile a build using the new raw_rec.mo for use with the 600D, or is it simple to just add it into the modules folder?

http://www.magiclantern.fm/forum/index.php?topic=5413.25 go to reply #30 I assumed the raw_rec.mo was a unified module so I tried it. Works. Works well :) Just replace the one you are using with the one from the .zip

I forgot to thank the devs for it in my previous post so thanks guys!!  8)
Title: Re: uncompressed 14-bit RAW video recording
Post by: mattmvpmedia on May 16, 2013, 01:29:06 AM
Thanks Andy600!
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 16, 2013, 01:34:18 AM
Quote from: y3llow on May 15, 2013, 06:56:10 PM
There's no color profile assigned because its raw, color primaries and therefore gamut are not even defined, when raw is 'developed' in whatever app used you define the color space, dcraw for example lets you develop as XYZ, Prophoto, Adobe RGB or sRGB, lets you choose any gamma to be applied, so use a color managed app, define your working color space, choose different working spaces, import dng's, compare histograms and channel clipping, go from there.

I'm working on a Canon standard replicant sLUT for adobe camera raw.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 16, 2013, 01:36:55 AM
Quote from: tonybeccar on May 15, 2013, 09:47:53 PM
Has anyone tried with the Toshiba 1066x Excercia Pro compact flash cards?? They're supposed to be 30 MB/s faster! They say they're prepared to record 4k video...

Could be a load of bullshit marketing hype.... but I bought it. Should be here in a week or so.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 16, 2013, 01:43:23 AM
Quote from: aaphotog on May 15, 2013, 09:07:58 PM
Does anyone know why I continue getting this error when I boot the camera(5D3)
camera was not shut down cleanly. skipping module loading

I get that message after I boot the camera. but it only happens after I've recorded a raw video, then shut the camera down.
Only way I know to get rid of the message is to format the card and reinstall ML

Yeah same here, started happening on day 2 with the May 13 build.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 02:00:58 AM
Looks to be fixed from latest commit. I have to try it.. its just a lock file in the dir.. no need to format anything.
Title: Re: uncompressed 14-bit RAW video recording
Post by: tonybeccar on May 16, 2013, 02:11:30 AM
Quote from: squig on May 16, 2013, 01:36:55 AM
Could be a load of bullshit marketing hype.... but I bought it. Should be here in a week or so.

Great!! Let us know what happens! :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 02:18:08 AM
The new module is the shit.

Beep should be on/off for now I guess turn off beeps.

My write appears to be lower, dunno if its calculated better but despite the numbers it appears way more stable.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 02:20:01 AM
Just did a quick and dirty 10 second test (on a slow SD card) to test ISO and scaling. It's not very pretty but...

http://www55.zippyshare.com/v/57521426/file.html (40mb file)

600d using the new raw_rec module - 960x360 scaled to 1080p / 2.35:1 (almost) shot at ISO2500 (which I would never use). I just made a few tweaks to levels and color in AE. I'm gonna try some proper shots tomorrow and maybe a green screen test.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 16, 2013, 02:20:22 AM
Quote from: tonybeccar on May 16, 2013, 02:11:30 AM
Great!! Let us know what happens! :)

Costs about the same as the Lexar so once I post the benchmarks we'll know which is the one to get.
Title: New raw_rec module does help the 600D
Post by: mattmvpmedia on May 16, 2013, 02:35:37 AM
Fo shizzle - the new raw_rec module helps a bunch. I'm still dropping a few frames at 960x540, still getting a little bit of tearing and magenta blocks, but much improved. Thanks!
Title: Re: uncompressed 14-bit RAW video recording
Post by: hirethestache on May 16, 2013, 02:37:34 AM
Quote from: hirethestache on May 15, 2013, 02:13:42 AM


I was about to ask these exact same questions...beat me to it!


Since the last I inquired about recording to the HDMI-out, the developers implied that you did not even thing continuous recording would be possible...but your magical ingenuity has broken through! Is the idea of running through the HDMI's [through an external field monitor such as the Atomos Ninja2] limit of 10gpbs (realistically 3.2) even a feasible approach, or is it completely unrealistic? This would reduce the strain on the camera, and heat would be much less of an issue...

Im going to catch flack for asking this again, but I did not see a response in the pages since posting.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ease on May 16, 2013, 02:39:30 AM
i now just want to say thanks a lot for your amazing work here.


Title: Re: uncompressed 14-bit RAW video recording
Post by: Kabuto1138 on May 16, 2013, 02:42:18 AM
Quote from: 1% on May 16, 2013, 02:18:08 AM
The new module is the shit.

Beep should be on/off for now I guess turn off beeps.

My write appears to be lower, dunno if its calculated better but despite the numbers it appears way more stable.

Hey guys, where is the new module?

thanks!
Title: reply #904 has info on new raw_rec module
Post by: mattmvpmedia on May 16, 2013, 02:48:04 AM
Go to reply #904 from Andy600 for details on where to get the new raw_rec module
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 03:00:05 AM
New stuff should be up for 6D/600D. New module and all.. .can hit rec to start stop movies.

It beeps twice because there is an unpress/press event and something is up with how its defined on 600D.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 03:05:35 AM
@1% - thanks man. I was just reading the new commits. Looks interesting :)

Just a thought. If performance takes a hit from Global Draw being on could the raw_rec module switch off things before recording is started and back on when recording is stopped?  :)
Title: Updated build for 6d 600D
Post by: mattmvpmedia on May 16, 2013, 03:12:52 AM
Thanks 1%!

FYI, if you are new like me you can find the new [600D] Beeps Twice.zip file here:
https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads (https://bitbucket.org/OtherOnePercent/tragic-lantern-2.0/downloads)
Title: Re: uncompressed 14-bit RAW video recording
Post by: jasonp on May 16, 2013, 03:54:33 AM
Komputer Bay 1000x 64Gb
The latest raw module posted previously by Andy in this thread works perfectly in 1920x1080 with a 5D3, drops 1 frame in x1152 and several in x1180. Let me know if you need more tests with different resolutions. (https://dl.dropboxusercontent.com/u/51781673/VRAM1.BMP)
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 04:02:20 AM
Global draw can be on for indicators. Mainly no peaking/zebras.
Title: Re: uncompressed 14-bit RAW video recording
Post by: KMA_WWC on May 16, 2013, 04:20:23 AM
Quote from: platu on May 16, 2013, 12:42:55 AM

I just ordered and received the same CF card as you... the KomputerBay 128gb 1000x CF card and have been running tests with each of the various Raw builds over the last 3 days.  The latest build for 5D3 someone just posted from today seems to be the best in terms of speed... see http://www.magiclantern.fm/forum/index.php?topic=5413.25 (Reply #30).  This includes some small memory tweaks that definitely helped me get a little bit longer runtimes as well as moved me up to the next resolution without dropped frames.

Here has been my experience with the KomputerBay 128 GB 1000x CF card...

For my tests, I tried Global draw turned ON and OFF and there was no significant difference except maybe a second or two extra of recording time but the extra time was not consistent.  So I decided to leave Global Draw = ON (with just peaking and crop marks enabled).


My tests using the ML benchmark utility shows read speeds that range anywhere from 52MB/s to 72MB/s.  Using CrystalDiskMark and ATTO Disk Benchmark, I get similar write speeds so my particular card seems to have a write speed that varies a bit.  I would say the average write speed that I get most of the time is about 65MB/s which is completely in line with the ML benchmarks and the listed speed needed by each resolution listed under the RAW video section of ML.  So I would trust the ML benchmarks and recommended write speeds for each resolution.  I have also confirmed that at least one other person is getting similar write speed using this card.

That said, the buffer/memory tweak Alex made today did have a significant impact when using my card. 

Here are my tests Before today's build...
1920 x 720 or less resolution (no skipped frames for complete 4gb file)
1920 x 840 (no skipped frames for complete 4 gb file)... before today's build, this was the best I could do.
1920 x 900 (no skipped frames for complete 4 gb file)... tweaks added to today's build allowed me to use resolution for first time
1920 x 960 (maybe 10 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (maybe 1-2 seconds before frame skipping begins... unusabe beyond that)

After using today's build...
1920 x 900 (no skipped frames for complete 4 gb file) before today's build, I could not use this resolution
1920 x 960 (went from 10 seconds to 20 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (went from 1-2 seconds to 5 seconds before frame skipping begins... unusabe beyond that)

So the tweaks made by Alex today have definitely helped so I encourage further memory/buffer optimizations... they are making a difference.

I don't know how EOSHD, Nuemann Films, and Cinema 5D are getting 1080P and greater without dropped frames. They say they are using the same card.  Maybe some of these cards are getting faster write times.  Or maybe they are just getting that resolution for a limited time but a bit longer than me before frame skipping appears.  If any of them can chime in here to clarify, it would helpful for those considering the Komputerbay cards.  This card is by far the most realistic in terms of pricing for the vast majority of people who want to take advantage of this RAW update and plan on using it beyond test videos and personal work.  None of the other cards comes close in terms of price/gb.  The other solutions by Lexar and Toshiba (soon) are more like $650 for 128gb vs $178 or KomputerBay. I'm sure this will eventually change, but that could take a year or longer before cheaper alternatives are available.  But there is no getting around the fact that the less than ideal write speed of these cards may prevent continuous 1080p for some (depending on their particular card?).  But continued memory optimization and other techniques by the ML team may be able to extend 1080p recording from 5 seconds to 30 seconds or possibly more when using this card.  If that can be attained, the vast majority of folks will be able use this resolution in many shooting scenarios, short of documentary or event work.  I do realize that smaller resolutions can be scaled up in post nicely, but I tend to avoid that as do many others I'm sure.

Lastly, on the topic of workflow mentioned above... it's completely worth the time as far as I'm concerned.  The difference is night and day.  There are a lot of test videos coming out now, some of which don't fully capture the quality improvement gained here.  EOSHD, Nuemann Films, and Cinema 5D did great job of showing what's possible.  Just wait until some shorts and features by other skillful DOPs start to appear online.  While I don't love the added work required in post, I find it impossible to go back to H.264 after getting used to the look of my footage now...it's improved that much.

Thank you Alex and team for this... also to g3gg0 who apparently had a huge role in making this particular breakthrough happen.

I feel like you were siting next to me and seeing what I saw :) I have/had same issues like you did. I got my Komputerbay card and the speed vary between 60-72mb/s. I haven't tried the new build yet, but that's what I'm going to do. Hopefully its as good as other people say or at least better.


My personal observation about this whole thing:

I think right now not just everyday, but almost every hour or so there are new improvements in the firmware. I like the fact that most of the bugs can be fixed or worked around. I personally think installation and whole post workflow isn't as hard as some people describe it. It only took me 10min to downgrade from 1.2.1 to 1.1.3 and then install everything needed. Yes, post workflow is time consuming, but overall i think within a certain time that would be worked out too. I really like where everything goes and hope to contribute and help.

Also, I just tested the SanDisk extreme pro 32gb CF card and I was actually able to shoot at 1080p (1920-1080 exact) with that card until it reached 4gb. I had only 9 skipped frames (to be honest not 100% if it skipped any frames at all). Test shows the write speed around 74-88mb/s. Its much faster than my brand new komputerbay 128gb 1000x even though on paper komputerbay supposed to be better. All those test were done with an old build and I'm pretty sure after I install new build it would be much better.

QUESTION: Where do I send/attach the list of bugs I encountered also log infos?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yomommassis on May 16, 2013, 04:22:38 AM
Is there a link we can go to for donating to the developers?
Lets not forget to support the developers, they are really doing some amazing things here

and with the recent increase in attention and the current project there is no reason not to donate
Title: Re: uncompressed 14-bit RAW video recording
Post by: mixmastermike on May 16, 2013, 04:27:55 AM
Quote from: jasonp on May 16, 2013, 03:54:33 AM
Komputer Bay 1000x 64Gb
The latest raw module posted previously by Andy in this thread works perfectly in 1920x1080 with a 5D3, drops 1 frame in x1152 and several in x1180. Let me know if you need more tests with different resolutions. (https://dl.dropboxusercontent.com/u/51781673/VRAM1.BMP)

Nice test, I just ran the same test on a 32Gb x1000 Lexar, 1920 x 1180 now works without dropping frames. 1920 x 1280 drops a few frames.

Today's raw_rec file really improves the reliability. Much respect to all the coders!

The work over the last few days has given me a lot more confidence in filming 1920 x 1080
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 16, 2013, 04:39:15 AM
5D3 with 1000x 32 gig Lexar .  New Compile TEST

New compile gives me problems with producing cropped digital video.  If I record at 5X or 10X zoom, the RAW video can be separated by raw2dng.exe converter but the dng files cannot be opened in ACR 7.1 photoshop.

Also sometimes the raw2dng.exe converter does not put out any frames.

Also there is always a new RAW video on the card at 0Bites.

Maybe it is my compile or maybe not!

The regular video (not zoomed) is ok and I can now record to 1920x1152 without frame drops so the module clears the buffer a bit faster.

The new start - stop method works fine!

Anyway you guys are awesome!!
Title: Re: uncompressed 14-bit RAW video recording
Post by: mlrocks on May 16, 2013, 04:50:38 AM
Is there a way to record the upstream raw frames right after the sensor acquiring data? That will be 7 k.
Title: Re: uncompressed 14-bit RAW video recording
Post by: KMA_WWC on May 16, 2013, 04:52:13 AM
Tried to install new build and I always get "please copy al ML files" even though I did copied everything. Also all the menus are blured and look weird (small blocks). Anyone knows why? I will try to install an old build for now.

UPD: Just to confirm, old build works perfectly. Not sure why new one freaks out.

UPD 2: Looks like in new build missing some fonts etc. will try to fix it now and test it out. Am i the only one who has missing fonts in the new build?
Title: Re: uncompressed 14-bit RAW video recording
Post by: jpgentry on May 16, 2013, 05:06:36 AM
Thoughtful reply, thanks.

OK, so regarding the issue of 5d3 skipping and KomputerBay cards being junk as some have said...  It seems from what I'm gathering that the guys who have been successful at 1920x1080 were using the KomputerBay cards of 64 and 32GB size.  I have the 128GB and myself and you (platu) we are not getting 1920x1080 without almost immediate line skipping.

Let's all put our collective heads together to find out if this is the card size that is the issue.  How many of you 5d3 guys get no line skipping with 1920x1080 and what size card do you have?

Quote from: platu on May 16, 2013, 12:42:55 AM

I just ordered and received the same CF card as you... the KomputerBay 128gb 1000x CF card and have been running tests with each of the various Raw builds over the last 3 days.  The latest build for 5D3 someone just posted from today seems to be the best in terms of speed... see http://www.magiclantern.fm/forum/index.php?topic=5413.25 (Reply #30).  This includes some small memory tweaks that definitely helped me get a little bit longer runtimes as well as moved me up to the next resolution without dropped frames.

Here has been my experience with the KomputerBay 128 GB 1000x CF card...

For my tests, I tried Global draw turned ON and OFF and there was no significant difference except maybe a second or two extra of recording time but the extra time was not consistent.  So I decided to leave Global Draw = ON (with just peaking and crop marks enabled).


My tests using the ML benchmark utility shows read speeds that range anywhere from 52MB/s to 72MB/s.  Using CrystalDiskMark and ATTO Disk Benchmark, I get similar write speeds so my particular card seems to have a write speed that varies a bit.  I would say the average write speed that I get most of the time is about 65MB/s which is completely in line with the ML benchmarks and the listed speed needed by each resolution listed under the RAW video section of ML.  So I would trust the ML benchmarks and recommended write speeds for each resolution.  I have also confirmed that at least one other person is getting similar write speed using this card.

That said, the buffer/memory tweak Alex made today did have a significant impact when using my card. 

Here are my tests Before today's build...
1920 x 720 or less resolution (no skipped frames for complete 4gb file)
1920 x 840 (no skipped frames for complete 4 gb file)... before today's build, this was the best I could do.
1920 x 900 (no skipped frames for complete 4 gb file)... tweaks added to today's build allowed me to use resolution for first time
1920 x 960 (maybe 10 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (maybe 1-2 seconds before frame skipping begins... unusabe beyond that)

After using today's build...
1920 x 900 (no skipped frames for complete 4 gb file) before today's build, I could not use this resolution
1920 x 960 (went from 10 seconds to 20 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (went from 1-2 seconds to 5 seconds before frame skipping begins... unusabe beyond that)

So the tweaks made by Alex today have definitely helped so I encourage further memory/buffer optimizations... they are making a difference.

I don't know how EOSHD, Nuemann Films, and Cinema 5D are getting 1080P and greater without dropped frames. They say they are using the same card.  Maybe some of these cards are getting faster write times.  Or maybe they are just getting that resolution for a limited time but a bit longer than me before frame skipping appears.  If any of them can chime in here to clarify, it would helpful for those considering the Komputerbay cards.  This card is by far the most realistic in terms of pricing for the vast majority of people who want to take advantage of this RAW update and plan on using it beyond test videos and personal work.  None of the other cards comes close in terms of price/gb.  The other solutions by Lexar and Toshiba (soon) are more like $650 for 128gb vs $178 or KomputerBay. I'm sure this will eventually change, but that could take a year or longer before cheaper alternatives are available.  But there is no getting around the fact that the less than ideal write speed of these cards may prevent continuous 1080p for some (depending on their particular card?).  But continued memory optimization and other techniques by the ML team may be able to extend 1080p recording from 5 seconds to 30 seconds or possibly more when using this card.  If that can be attained, the vast majority of folks will be able use this resolution in many shooting scenarios, short of documentary or event work.  I do realize that smaller resolutions can be scaled up in post nicely, but I tend to avoid that as do many others I'm sure.

Lastly, on the topic of workflow mentioned above... it's completely worth the time as far as I'm concerned.  The difference is night and day.  There are a lot of test videos coming out now, some of which don't fully capture the quality improvement gained here.  EOSHD, Nuemann Films, and Cinema 5D did great job of showing what's possible.  Just wait until some shorts and features by other skillful DOPs start to appear online.  While I don't love the added work required in post, I find it impossible to go back to H.264 after getting used to the look of my footage now...it's improved that much.

Thank you Alex and team for this... also to g3gg0 who apparently had a huge role in making this particular breakthrough happen.
Title: Re: uncompressed 14-bit RAW video recording
Post by: kaoshotbeatz on May 16, 2013, 05:13:08 AM
I got the may 12th version working fine on my 5dmk3  i see more detail and inside i see less noise at the higher iso im using  promasters 64 gig highspeed udma 1000 card the files write super big would be great to just get a 14 bit or even a 12 bit prores  that would be fine i havent been able to get past the 2 gig limit but thats ok out side i see a big color difference . still no sound and hope they figure out a 1.2.1 version as i have th ninja 2 and want to use that . great alex i get paid on friday i will most def donate some money again
Title: Re: uncompressed 14-bit RAW video recording
Post by: jpgentry on May 16, 2013, 05:17:14 AM
OK Platu.  I just downloaded the build you referenced and got the same results as you.  1920x960 went about 650 frames and then started skipping.  I think that numann and cinema5d must be using the 32 and 64gb cards where you and I are using the 128gb.  I don't know why that would have an effect, but I get no love from the 1920x1080 (full 1080p) setting on the 5d3 with the 128gb 1000x KomputerBay card. 

Quote from: platu on May 16, 2013, 12:42:55 AM

Here has been my experience with the KomputerBay 128 GB 1000x CF card...

After using today's build...
1920 x 900 (no skipped frames for complete 4 gb file) before today's build, I could not use this resolution
1920 x 960 (went from 10 seconds to 20 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (went from 1-2 seconds to 5 seconds before frame skipping begins... unusabe beyond that)

So the tweaks made by Alex today have definitely helped so I encourage further memory/buffer optimizations... they are making a difference.

I don't know how EOSHD, Nuemann Films, and Cinema 5D are getting 1080P and greater without dropped frames. They say they are using the same card.  Maybe some of these cards are getting faster write times.  Or maybe they are just getting that resolution for a limited time but a bit longer than me before frame skipping appears.  If any of them can chime in here to clarify, it would helpful for those considering the Komputerbay cards.  This card is by far the most realistic in terms of pricing for the vast majority of people who want to take advantage of this RAW update and plan on using it beyond test videos and personal work.  None of the other cards comes close in terms of price/gb.  The other solutions by Lexar and Toshiba (soon) are more like $650 for 128gb vs $178 or KomputerBay. I'm sure this will eventually change, but that could take a year or longer before cheaper alternatives are available.  But there is no getting around the fact that the less than ideal write speed of these cards may prevent continuous 1080p for some (depending on their particular card?).  But continued memory optimization and other techniques by the ML team may be able to extend 1080p recording from 5 seconds to 30 seconds or possibly more when using this card.  If that can be attained, the vast majority of folks will be able use this resolution in many shooting scenarios, short of documentary or event work.  I do realize that smaller resolutions can be scaled up in post nicely, but I tend to avoid that as do many others I'm sure.
Title: 600D 960x540 recording has improved with latest build
Post by: mattmvpmedia on May 16, 2013, 05:17:53 AM
Here's a sample from the 600D at 960x540 resolution with the latest build:


There is still some tearing and magenta blocks showing up, but it's definitely getting better and moving in the right direction. Great job ML team.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 16, 2013, 05:20:50 AM
Someone stated that the 4GB file limit is a constraint the Magic Lantern people put on it while its in experimental stage. They will remove it at some point.

Is this true or not?
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 16, 2013, 05:23:12 AM
jpgentry,

It seems like the 64GB KomputerBay card version may be faster than the 128GB.  Someone else mentioned this possibility either here or on another forum but I had discounted it until now.  We'll have to see what other folks experience to know for sure.
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 16, 2013, 05:28:39 AM
Quote from: jpgentry on May 16, 2013, 05:17:14 AM
OK Platu.  I just downloaded the build you referenced and got the same results as you.  1920x960 went about 650 frames and then started skipping.  I think that numann and cinema5d must be using the 32 and 64gb cards where you and I are using the 128gb.  I don't know why that would have an effect, but I get no love from the 1920x1080 (full 1080p) setting on the 5d3 with the 128gb 1000x KomputerBay card.

It actually makes a lot of sense now.  There are many examples of different speeds from the same series of card but just different capacity.
Title: Re: uncompressed 14-bit RAW video recording
Post by: kaoshotbeatz on May 16, 2013, 05:28:52 AM
http://youtu.be/g8ydDFewyGI

raw 1080 p
Title: Re: uncompressed 14-bit RAW video recording
Post by: Audionut on May 16, 2013, 05:29:22 AM
Consistent err 70 when half pressing shutter (silent pic) after RAW video.

Latest changeset doesn't work with old RAW2DNG, API version mismatch 29030400
Title: Re: uncompressed 14-bit RAW video recording
Post by: jpgentry on May 16, 2013, 05:30:26 AM
In the meantime 1920x900 is my friend.  I kind of like the wider format, though I'm not sure if it fits into any standard.  I guess we can always uprez a bit.

It's good to see that they made such progress today.  That holds out great hope for this making it to full 1080P even with our 128GB card.  I would like to see them raise the limit to 8GB for a single clip in the build for tomorrow.

Quote from: platu on May 16, 2013, 05:23:12 AM
jpgentry,

It seems like the 64GB KomputerBay card version may be faster than the 128GB.  Someone else mentioned this possibility either here or on another forum but I had discounted it until now.  We'll have to see what other folks experience to know for sure.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Colemar on May 16, 2013, 05:34:58 AM
just a guess, but it may be that the 128GB cards are formatted as exFAT, while the <64GB cards are FAT32 and the Canon's firmware is optimized for FAT32..somewhat analogous to the possible 32bit v. 64bit problem with raw2dng.
Title: Re: uncompressed 14-bit RAW video recording
Post by: jpgentry on May 16, 2013, 05:44:03 AM
If we are formatting the cards in the camera before using them wouldn't the camera dictate the format of the card? 

Quote from: Colemar on May 16, 2013, 05:34:58 AM
just a guess, but it may be that the 128GB cards are formatted as exFAT, while the <64GB cards are FAT32 and the Canon's firmware is optimized for FAT32..somewhat analogous to the possible 32bit v. 64bit problem with raw2dng.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Colemar on May 16, 2013, 05:46:35 AM
Quote from: jpgentry on May 16, 2013, 05:44:03 AM
If we are formatting the cards in the camera before using them wouldn't the camera dictate the format of the card?

I haven't found that to be the case, rather the firmware of the card dictates its native formatting schema and size allotment, etc.

also, 1080p23.98 14bit raw footage: https://vimeo.com/66268193  with more to follow
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 16, 2013, 05:46:42 AM
Quote from: jpgentry on May 16, 2013, 05:30:26 AM
In the meantime 1920x900 is my friend.  I kind of like the wider format, though I'm not sure if it fits into any standard.  I guess we can always uprez a bit.

It's good to see that they made such progress today.  That holds out great hope for this making it to full 1080P even with our 128GB card.  I would like to see them raise the limit to 8GB for a single clip in the build for tomorrow.

1920x900 is a pretty good resolution if that's the best we can get.... you can crop down in post to 1920x817 for perfect 2:35:1 Cinemascope.  As a bonus, you'll have 83 pixels of extra vertical space for recomposing.  I'm sure that further optimizations will get us closer to 1080p, but I'm not sure if it will get us all the way there.  The reality is that 1080p requires 80MB/s+ of sustained writing, and our cards just don't do that... not even in short bursts have I ever measured  80+ on my card.  But given how clever the ML folks have been thus far, it wouldn' t surprise me if they figured something out.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Digital Corpus on May 16, 2013, 05:48:49 AM
Just consolidating some information here from what I've read as I find this thread highly intoxicating, despite not owning a 5D3.

Correct me if I'm wrong here, but a RAW frame will be composed of R, G, & B channels. However since the data is not debayered, R & B are 1/2 the horizontal and vertical resolution of a debayered frame, thus 1/4 of the frame's resolution is color data for either R or B. And since G is twice as pixels than either R or B, it'd consume 1/2 of the frame's resolution is color data. Overall, this add's us to just the pixel count of what a frame is. Now, this is 14-bits per pixel data, 8-bits in a byte, 23.976 fps (simplified to 24 fps for the sake of simplicity), frame_res*14/8*24= datarate in bytes/sec.

If my understanding of the data format is sound then this is what follows so far...

For the 5D3 with the KomputerBay 1000x cards being used, we've seen:
1920x900 @ 24 fps --> 1,728,000 pixels, 14 bpp @ 24 fps --> ~69.2 MB/sec write speed requirement
1920x1080 @ 24 fps --> 2,073,600 pixels, 14 bpp @ 24 fps --> ~83.1 MB/sec write speed requirement
1920x1152 @ 24 fps --> 2,211,840 pixels, 14 bpp @ 24 fps --> ~88.6 MB/sec write speed requirement

For the 600D, the reported card I happened to miss, we've seen:
1280x400 @ 24 fps --> 512,000 pixels, 14 bpp @ 24 fps --> ~20.5 MB/sec write speed
960x540 @ 24 fps --> 518,400 pixels, 14 bpp @ 24 fps --> ~20.76 MB/sec write speed

CF and SD cards are Flash memory. There is a bit of a principle of Flash memory that *will* affect  your write speeds, and though these memory cards are not what we consider SSD's, they have to abide by the same rules (http://"http://www.anandtech.com/show/2738/8").

In order to provide a proper test, also to ensure your CF/SD card wears evenly, which not all manufacturers use or advertise if they have wear leveling, you should format your memory card in a computer with a "Full" format to make sure you get real world results. Initial writes may be faster, but that won't be how the rest of the card performs during the rest of it's life.
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 16, 2013, 05:52:20 AM
Quote from: Colemar on May 16, 2013, 05:34:58 AM
just a guess, but it may be that the 128GB cards are formatted as exFAT, while the <64GB cards are FAT32 and the Canon's firmware is optimized for FAT32..somewhat analogous to the possible 32bit v. 64bit problem with raw2dng.

I already tried that.  The card came originally formatted Fat32 and when all my attempts to get faster speed failed, I did a low level reformat to exFAT to see if it faired any better... but there was no significant change in speed (sort of what I expected).
Title: Re: uncompressed 14-bit RAW video recording
Post by: eyeland on May 16, 2013, 05:57:23 AM
Quote from: squig on May 16, 2013, 01:34:18 AM
I'm working on a Canon standard replicant sLUT for adobe camera raw.
Please do share :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 16, 2013, 06:02:39 AM
Quote from: Colemar on May 16, 2013, 05:46:35 AM
I haven't found that to be the case, rather the firmware of the card dictates its native formatting schema and size allotment, etc.

also, 1080p23.98 14bit raw footage: https://vimeo.com/66268193  with more to follow

That's correct... the camera will respect whatever compatible format the card is using.  So formatting my exFAT card in-camera maintains exFAT.  FYI... the benefit of exFAT will be when the ML folks enable >4GB raw files.  Fat32 only supports files up to 4GB...  exFAT has no such restriction.  This may be a moot point if the ML folks decide to create separate 4GB files instead of one larger file >4GB for long takes.  I see the pros and cons of either approach so ideally it would be a configurable option.
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 16, 2013, 06:06:59 AM
Quote from: Digital Corpus on May 16, 2013, 05:48:49 AM
Just consolidating some information here from what I've read as I find this thread highly intoxicating, despite not owning a 5D3.

Correct me if I'm wrong here, but a RAW frame will be composed of R, G, & B channels. However since the data is not debayered, R & B are 1/2 the horizontal and vertical resolution of a debayered frame, thus 1/4 of the frame's resolution is color data for either R or B. And since G is twice as pixels than either R or B, it'd consume 1/2 of the frame's resolution is color data. Overall, this add's us to just the pixel count of what a frame is. Now, this is 14-bits per pixel data, 8-bits in a byte, 23.976 fps (simplified to 24 fps for the sake of simplicity), frame_res*14/8*24= datarate in bytes/sec.

If my understanding of the data format is sound then this is what follows so far...

For the 5D3 with the KomputerBay 1000x cards being used, we've seen:
1920x900 @ 24 fps --> 1,728,000 pixels, 14 bpp @ 24 fps --> ~69.2 MB/sec write speed requirement
1920x1080 @ 24 fps --> 2,073,600 pixels, 14 bpp @ 24 fps --> ~83.1 MB/sec write speed requirement
1920x1152 @ 24 fps --> 2,211,840 pixels, 14 bpp @ 24 fps --> ~88.6 MB/sec write speed requirement

For the 600D, the reported card I happened to miss, we've seen:
1280x400 @ 24 fps --> 512,000 pixels, 14 bpp @ 24 fps --> ~20.5 MB/sec write speed
960x540 @ 24 fps --> 518,400 pixels, 14 bpp @ 24 fps --> ~20.76 MB/sec write speed

CF and SD cards are Flash memory. There is a bit of a principle of Flash memory that *will* affect  your write speeds, and though these memory cards are not what we consider SSD's, they have to abide by the same rules (http://"http://www.anandtech.com/show/2738/8").

In order to provide a proper test, also to ensure your CF/SD card wears evenly, which not all manufacturers use or advertise if they have wear leveling, you should format your memory card in a computer with a "Full" format to make sure you get real world results. Initial writes may be faster, but that won't be how the rest of the card performs during the rest of it's life.

Interesting analysis... thank you.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 06:45:24 AM
Silent pic with raw_rec armed will kill silent pics... all memory is allocated.

4gb limit already off in my modules but files won't convert.

If you format your whatever GB cf card to exfat then the 4gb limit doesn't apply to you either. You wouldn't gain speed... just lose the limit.



Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 16, 2013, 06:56:06 AM
Anyway to to format the camera in camera, while keeping Magic Lantern?
Im talking about the latest compiled May 16th build
Title: Re: uncompressed 14-bit RAW video recording
Post by: Colemar on May 16, 2013, 08:31:00 AM
beautiful "test" here: https://vimeo.com/66296381
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 16, 2013, 08:31:55 AM
Quote from: platu on May 16, 2013, 06:02:39 AM
That's correct... the camera will respect whatever compatible format the card is using.  So formatting my exFAT card in-camera maintains exFAT.  FYI... the benefit of exFAT will be when the ML folks enable >4GB raw files.  Fat32 only supports files up to 4GB...  exFAT has no such restriction.  This may be a moot point if the ML folks decide to create separate 4GB files instead of one larger file >4GB for long takes.  I see the pros and cons of either approach so ideally it would be a configurable option.

I actually need to correct what I just said.  It seems that when formatting my CF card in-camera, the 5D3 uses Fat32 for cards up to 128gb, but switches to exFAT for cards larger than this.  It's in canon's documentation and I verified this behavior myself by first formatting my card in my computer using exFAT and then formatted in-camera... the file format was switched to Fat32.

This can be quite problematic for all people using cards up to 128gb (the vast majority) who need to record Raw files larger than 4gb and still be able to format in-camera.  Doing that would revert the card back to Fat32.  I am unaware of any method of changing this camera behavior.  So for future prospects of recording 1080p longer than 30 secords or so, you would have to format via computer using exFAT and be sure not to ever format in-camera.  Unless the ML team could override this behavior when formatting, the best option would probably be to automatically create additional 4gb raw files on the fly for longer takes.   Anyway, just thought I'd share this observation.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Fergus Kennedy on May 16, 2013, 09:42:08 AM
Hi Folks,

First of all- massive thanks to the developers- I'm so excited about this and can't wait to get it running.
Apologies for what may be a stupid newb question, but I'm having trouble getting the RAW capable ML on my 5D Mark III. In the past I've had previous alpha version on it no probs.
This time I've been following all posted instructions to the letter (I think!). Canon firmware 1.1.3, camera on M, etc. When I load the bootflag.fir I get an  error message - could not find ML files, followed by a series of other messages, then some garbage half way up the screen, which stays there. Is this normal behaviour?
Moving onto the next steps, I have deleted the bootflag file and added the ML folder and autoexec file. Plus made the card bootage with EOScard, following the instructions for that. When I load it up, nothing happens when I press the trash button.
I have tried several different SD cards and have cleared all camera settings and have even tried reinstalling the Canon firmware.
HELP- it would be fantastic if someone had any idea where I am going wrong, or anything else to try as it's driving me nuts!
Cheers
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 16, 2013, 09:59:22 AM
Quote from: platu on May 16, 2013, 08:31:55 AMrecord Raw files larger than 4gb

as only few cameras support files > 4GiB, we will probably not use that feature, but use file splitting instead.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Vegandelight on May 16, 2013, 10:13:02 AM
Quote from: KMA_WWC on May 16, 2013, 04:20:23 AM
I feel like you were siting next to me and seeing what I saw :) I have/had same issues like you did. I got my Komputerbay card and the speed vary between 60-72mb/s. I haven't tried the new build yet, but that's what I'm going to do. Hopefully its as good as other people say or at least better.


My personal observation about this whole thing:

I think right now not just everyday, but almost every hour or so there are new improvements in the firmware. I like the fact that most of the bugs can be fixed or worked around. I personally think installation and whole post workflow isn't as hard as some people describe it. It only took me 10min to downgrade from 1.2.1 to 1.1.3 and then install everything needed. Yes, post workflow is time consuming, but overall i think within a certain time that would be worked out too. I really like where everything goes and hope to contribute and help.

Also, I just tested the SanDisk extreme pro 32gb CF card and I was actually able to shoot at 1080p (1920-1080 exact) with that card until it reached 4gb. I had only 9 skipped frames (to be honest not 100% if it skipped any frames at all). Test shows the write speed around 74-88mb/s. Its much faster than my brand new komputerbay 128gb 1000x even though on paper komputerbay supposed to be better. All those test were done with an old build and I'm pretty sure after I install new build it would be much better.

QUESTION: Where do I send/attach the list of bugs I encountered also log infos?

I think it might be the bigger Komputerbay cards that has problems with speed. I got the 32gb and havent had any dropped frames on 1920x1080. And im running the May13 build.  I havent been able to do bigger than 4gb files tho.
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 16, 2013, 10:23:35 AM
Quote from: g3gg0 on May 16, 2013, 09:59:22 AM
as only few cameras support files > 4GiB, we will probably not use that feature, but use file splitting instead.

Makes sense... thanks.
Title: Re: uncompressed 14-bit RAW video recording
Post by: maxpyane_2k6 on May 16, 2013, 10:55:54 AM
Does anyone able to shoot raw video at 23.976 on a 5D Mark II??
Title: Re: uncompressed 14-bit RAW video recording
Post by: KenFTW on May 16, 2013, 11:06:49 AM
I'm using a sandisk extreme pro 95mb card (just in hopes of getting peak performance from my cam) on my 600d and this new build is far better than the one from May 13th! Shooting at 1280 x 320: 11 of 544 frames were unusable and that in and of itself makes me drool. I'm more than confident that the ML and new found contributors will make this one for the history books! Congrats and DONATE!!!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Africashot on May 16, 2013, 11:09:33 AM
Quote from: maxpyane_2k6 on May 16, 2013, 10:55:54 AM
Does anyone able to shoot raw video at 23.976 on a 5D Mark II??
There are several threads about this, like this one: http://www.magiclantern.fm/forum/index.php?topic=5419.msg35576;topicseen#msg35576
Mine does continuous 23.976 only at 1280x720, but between 280 - 350 frames in 1880x720. In any case I only have tried on a SanDisk 60mb/s card, thus faster cards may do 1880x840 or more.
On the bottom line I am quite happy about this already, I am used to compromising  on 720p if I have to shoot higher frame rates with other cameras, ones upscaled it looks just as good (or bad) as h.264 but with the extra dynamic range and color depth, if the full width (1880) could be rendered at an aspect ration of 2.35 rather then the current 2.6 I'd be a happy camper shooting anamorphic style cinemascope till either I or my good old 5D2 die!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 16, 2013, 11:45:14 AM
I do not have the 1000-х  flash card. But on 400-x  working well with 1920х720!
500мм f6.3,  SIGMA150-500...


If you are interested in an uncompressed download here:
http://yadi.sk/d/YJZ82AMJ4t7Ac
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kuky on May 16, 2013, 12:20:54 PM
Quote from: Peter Linov on May 16, 2013, 11:45:14 AM
I do not have the 1000-х  flash card. But on 400-x  working well with 1920х720!
500мм f6.3,  SIGMA150-500...


If you are interested in an uncompressed download here:
http://yadi.sk/d/YJZ82AMJ4t7Ac

How ?? On a 5Dmk3? You are not skipping any frame on long shots?

Using Commit b6c2ae6 on a 5Dmk3 I'm able to do max 1920 x 680 on a Lexar Professional 600x UDMA 16Gb card. From 1920 x 700 it begins to skip frames.

But DNG files can't be opened! Help!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 12:21:47 PM
TIP - This is useful for calculating aspect ratios (if you don't know them) http://www.silisoftware.com/tools/screen.php

It's very easy. Just select the aspect ratio i.e. 2.35 = Cinemascope, enter either the width OR height (not both) and it will calculate the missing dimension for you.  :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Mr. Propaganda Guy on May 16, 2013, 12:24:51 PM
Quote from: 1% on May 15, 2013, 10:40:46 PM
7d probably == 5d2


But g3ggo said the raw image is on the other proc so harder to get at.. which is why its not done.

Real Amazing work Magic Lantern Crew!
If you guys get this running on the 7D, "Hope you do" you think it'll be possible to use the 2nd processor to add compression to the RAW? If so, what compression, Mjpeg, Prores, DNxHD? 
That would be Shweet.
Title: Re: uncompressed 14-bit RAW video recording
Post by: xvince1 on May 16, 2013, 12:31:17 PM
Hi to everyone there, and off course, a massive thank's to the ML team... for what's coming, and for what's allready done since the first ML releases...

I Just register to clarify some things :

- So if I've well understand : if the CF is in exFAT formated, we can record files beyond 4Gb ? is that correct ?
- To everyone here, if it can be more precise : actual benchs on CF rates RAW are using audio or not ?
- with H264, I got breaks up around 2.3x (without audio) on a 600x komputerbay CF, is it coming from digic or buffer overwhelming since the debt are much lower than RAW ?
- The picture styles (cinestyle / landscape etc etc) are still used on RAW footage or not ?

As a noob, I'm waiting for a stable release to test on my mk2... So happy to read this threads... THANK'S AGAIN !!!  ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 16, 2013, 12:34:30 PM
Quote from: Kuky on May 16, 2013, 12:20:54 PM
How ?? On a 5Dmk3? You are not skipping any frame on long shots?

Using Commit b6c2ae6 on a 5Dmk3 I'm able to do max 1920 x 680 on a Lexar Professional 600x UDMA 16Gb card. From 1920 x 700 it begins to skip frames.

Yes, I have no problem with the 400x at 1920х720.
I use a flash card Transcend 64Gb
Here's another video on my street:


If you are interested in an uncompressed download here:
http://yadi.sk/d/G0lQEtqk4tG56
Title: Re: uncompressed 14-bit RAW video recording
Post by: xvince1 on May 16, 2013, 12:39:52 PM
Quote from: Peter Linov on May 16, 2013, 12:34:30 PM
Yes, I have no problem with the 400x at 1920х720.
I use a flash card Transcend 64Gb
Here's another video on my street:


If you are interested in an uncompressed download here:
http://yadi.sk/d/G0lQEtqk4tG56

RAW dynamics are very impressive... No differences between RAW / H264 shooting parameters, I presume ?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 16, 2013, 12:59:22 PM
parameters are the same for both videos!
there is no difference
diaphragm 8
  shutter 500
  iso 125
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kuky on May 16, 2013, 01:01:32 PM
Quote from: Peter Linov on May 16, 2013, 12:59:22 PM
parameters are the same for both videos!
there is no difference
diaphragm 8
  shutter 500
  iso 125

shutter 500 ?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Alia5 on May 16, 2013, 01:12:43 PM
Absulutely ZERO(!) magenta stuff when shooting 720x400 on my 600D with new raw_rec.mo
fantastic!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yomommassis on May 16, 2013, 01:14:44 PM
I cant really contribute much to the cause..but if it helps a way to help automate the raw2dng process:

1. create a .bat
2. paste in the location of your raw2dng.exe   (ie. mine is C:\Users\John\Desktop\RawFootage\raw2dng.exe)
3. add %~n1.raw to the end of the line   (ie. C:\Users\John\Desktop\RawFootage\raw2dng.exe %~n1.raw)

this allows you to simply drag n drop the .raw onto the .bat and immediately convert without having to open a cmd and type anything

Hope this helps
Title: Re: uncompressed 14-bit RAW video recording
Post by: Alia5 on May 16, 2013, 01:18:52 PM
Quote from: Yomommassis on May 16, 2013, 01:14:44 PM
I cant really contribute much to the cause..but if it helps a way to help automate the raw2dng process:

1. create a .bat
2. paste in the location of your raw2dng.exe   (ie. mine is C:\Users\John\Desktop\RawFootage\raw2dng.exe)
3. add %~n1.raw to the end of the line   (ie. C:\Users\John\Desktop\RawFootage\raw2dng.exe %~n1.raw)

this allows you to simply drag n drop the .raw onto the .bat and immediately convert without having to open a cmd and type anything

Hope this helps

It even works without the batch-file ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yomommassis on May 16, 2013, 01:23:21 PM
Quote from: Alia5 on May 16, 2013, 01:18:52 PM
It even works without the batch-file ;)

Really? it wasn't working for me, although it might be because the .RAW files I tried were corrupted...
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yomommassis on May 16, 2013, 01:25:23 PM
but if you try to convert a corrupted file directly to the .exe and it fails it just flashes up and closes, if you add a pause to the .bat you can see the error code
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kuky on May 16, 2013, 01:30:36 PM
@Peter Linov

Can you pls post a link or just post the build you are using for 5dmk3?
I'm trying the ommit b6c2ae6 commit but can't open dng files.

It appears that a Sandisk Extreme Pro 90MB/s 16 gb is able to do 1920x1080 without skipping with this commit but I can't confirm because I'm unable to see the recording on the PC.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 16, 2013, 01:33:45 PM
Quote from: Kuky on May 16, 2013, 01:30:36 PM
@Peter Linov

Can you pls post a link or just post the build you are using for 5dmk3?
I'm trying the ommit b6c2ae6 commit but can't open dng files.

Sorry, my English is not very good.
The settings in the camera 5DMIII?
Or how I process RAW, what .... I'm sorry, I do not understand.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 01:41:14 PM
What we need is a basic app to copy each .RAW file from the card to it's own folder along with a copy of raw2dng.exe, run raw2dng.exe in each folder then delete the .RAWs plus maybe the option to delete the .RAWs from the card. I'm no programmer but this sounds relatively simple no?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kuky on May 16, 2013, 01:45:51 PM
Quote from: Peter Linov on May 16, 2013, 01:33:45 PM
Sorry, my English is not very good.
The settings in the camera 5DMIII?
Or how I process RAW, what .... I'm sorry, I do not understand.

The firmware. A zip file containing autoexec.bin and ML folder.

Thank you,
Cristian
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 16, 2013, 02:05:37 PM
Quote from: Kuky on May 16, 2013, 01:45:51 PM
The firmware. A zip file containing autoexec.bin and ML folder.

Thank you,
Cristian

http://yadi.sk/d/xLeAyYky4qek0
Title: Re: uncompressed 14-bit RAW video recording
Post by: s---70 on May 16, 2013, 02:22:07 PM
Hey there
this question has probably already been asked, but whats the great advantage of 14 bit vs. 12 bit RAW recording? If people have problems with 1080p on some cards but get it working nicely with 900p, why can't you just reduce the bitsize and therefore storage / datarate requirements? Sure 14 bit give more colors to work with but I can't imagine any situation where 12 bit is not enough. Am I missing something?
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 16, 2013, 02:27:55 PM
Quote from: s---70 on May 16, 2013, 02:22:07 PM
Hey there
this question has probably already been asked, but whats the great advantage of 14 bit vs. 12 bit RAW recording? If people have problems with 1080p on some cards but get it working nicely with 900p, why can't you just reduce the bitsize and therefore storage / datarate requirements? Sure 14 bit give more colors to work with but I can't imagine any situation where 12 bit is not enough. Am I missing something?

Take a look at this ...

http://www.cambridgeincolour.com/tutorials/bit-depth.htm
Title: Re: uncompressed 14-bit RAW video recording
Post by: s---70 on May 16, 2013, 02:33:12 PM
Quote from: mageye on May 16, 2013, 02:27:55 PM
Take a look at this ...

http://www.cambridgeincolour.com/tutorials/bit-depth.htm

are you referring to the "bit depth visualization"? That "12 bit" image shows a 4 bit per channel bit depth and we're talking about 12 bit/channel here ;-)
Title: Re: uncompressed 14-bit RAW video recording
Post by: deleted.account on May 16, 2013, 02:59:07 PM
http://www.cambridgeincolour.com/forums/thread81.htm

http://www.bythom.com/qadraw.htm
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 16, 2013, 03:29:16 PM
Quote from: s---70 on May 16, 2013, 02:33:12 PM
are you referring to the "bit depth visualization"? That "12 bit" image shows a 4 bit per channel bit depth and we're talking about 12 bit/channel here ;-)

Not following you there?

The images (*.dng) files that are being discussed on this forum are 14-bit (apparently). I posted the link to the Cambridge in Colour site just as a good reference to explain bit depth. It makes it pretty clear (at least to me) about the differences in bit depth. I posted the link purely as an information source. I think that such a site is a good resource as regards to digital photography. Some of the fundamental stuff is important for reference (in my opinion anyway)
Title: Re: uncompressed 14-bit RAW video recording
Post by: s---70 on May 16, 2013, 03:50:15 PM
Of course there's a slight advantage of 14 bit over 12 bit, but I'd definitely prefer to have 1080 instead of 900 lines over having 14 bit instead of 12 bit recording depth. The Black Magic Cinema Camera also records with 12 bit and I haven't heard anyone complaining about that. Maybe an option to choose between 14 bit and 12 bit would be a good thing to add.

Edit: I've read that the Raw data comes in a 14 bit linear format. If I understand correctly, it would be necessary to convert to a log format when using 12 bit to avoid losing dynamic range. Is that the problem?
Title: Re: uncompressed 14-bit RAW video recording
Post by: jpgentry on May 16, 2013, 04:09:02 PM
Quote from: s---70 on May 16, 2013, 02:22:07 PM
Hey there
this question has probably already been asked, but whats the great advantage of 14 bit vs. 12 bit RAW recording? If people have problems with 1080p on some cards but get it working nicely with 900p, why can't you just reduce the bitsize and therefore storage / datarate requirements? Sure 14 bit give more colors to work with but I can't imagine any situation where 12 bit is not enough. Am I missing something?

You don't understand, we are getting data straight off the sensor in raw format. Trying to compress that information is an additional step for the developers here. So it's not that there's an advantage or disadvantage, it's just that this is where we are at this particular stage of development.

You are getting 24P RAW photos collected into a single file, then breaking them apart into multiple DNG files, 24 per second, and recompiling them into a video clip.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 04:14:13 PM
@jpgentry - exactly! and we can bet the developers have had these ideas themselves and know the limitations of the hardware. What they are doing is remarkable and we should all let them just do their thing. I think we would all be better off helping them iron out any bugs first.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 04:15:48 PM
I think for 600D we need to try a different read edmac and a different connection. I have a feeling from earlier testing that something is using them sporadically... the pink frames don't come at regular intervals and look like something tried to write there.
Title: Re: uncompressed 14-bit RAW video recording
Post by: s---70 on May 16, 2013, 04:16:24 PM
Quote from: s---70 on May 16, 2013, 03:50:15 PM
Edit: I've read that the Raw data comes in a 14 bit linear format. If I understand correctly, it would be necessary to convert to a log format when using 12 bit to avoid losing dynamic range. Is that the problem?

did I get it right there?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Stedda on May 16, 2013, 04:17:36 PM
Quote from: Andy600 on May 16, 2013, 04:14:13 PM
What they are doing is remarkable and we should all let them just do their thing. I think we would all be better off helping them iron out any bugs first.

Amen, this feature has been around for less than a week... I say they've already provided an awesome feature set and they owe us nothing this is a free. Give them a break, I'm sure by the time this is included in a full release we'll all have more than we could have ever imagined.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 04:21:27 PM
@1% - I've shot quite a few things at different frames sizes to see if there is a pattern to when pink frames appear and it seems totally random. One theory might be that the card itself has issues? Bad sectors or something. All the pink frames I see look like the cropped area jumps to another part of the sensor. Could the autopan feature be a cause?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 04:29:33 PM
I don't think its auto pan... I don't have these issues on 6D and had a lot of trouble with early edmac_memcpy on 600D.

Its either sync or something is messing with the edmac.
Title: Re: uncompressed 14-bit RAW video recording
Post by: haavard on May 16, 2013, 04:32:25 PM
Using the "Build 16 Mai by forum member "a.d."" from the "Raw video on 5DMK2" thread, the 5D Mk2 does not update the space left between recorded clips. I have tried going in and out of live view mode, but it does not change the space left.
The only option is rebooting the camera.
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 16, 2013, 05:05:50 PM
Quote from: 1% on May 16, 2013, 04:29:33 PM
I don't think its auto pan... I don't have these issues on 6D and had a lot of trouble with early edmac_memcpy on 600D.

Its either sync or something is messing with the edmac.
I only get the pink frames once or twice in the first second or two of footage, then the rest is just fine.

This is with the Sandisk 95, if that makes a shit.



Did this video with the newest 600D build, 1280x720 at 24fps for the h264, and 640x480 at 24 fps with raw resolution set at 1280x400.

Better to use raw in 720p or 480? Seems like the images are cropped differently in each setting, but scaling the 400 up to 720 barely has any effect on quality.

Title: Re: uncompressed 14-bit RAW video recording
Post by: Shawn_Lights on May 16, 2013, 05:09:33 PM
Quote from: N/A on May 16, 2013, 05:05:50 PM
I only get the pink frames once or twice in the first second or two of footage, then the rest is just fine.

This is with the Sandisk 95, if that makes a shit.



Did this video with the newest 600D build, 1280x720 at 24fps for the h264, and 640x480 at 24 fps with raw resolution set at 1280x400.

Better to use raw in 720p or 480? Seems like the images are cropped differently in each setting, but scaling the 400 up to 720 barely has any effect on quality.

The second shot to my eye had more detail and looked great. Was that the 480 setting?

If we can get a 550D build I'll engage in testing as well.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 05:09:43 PM
I'm noticing a couple of little green dots on my DNGs. I thought they might be dead/hot pixels but they are not there on jpegs, CR2 or .mov files

(http://i.imgbox.com/aczXFmLI.jpg)
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 05:14:12 PM
Probably defects...

6D stuff upscales to 2K wide and 1K high without any noticeable loss from 1/2 of that pretty much.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 05:21:00 PM
@1% - yes, my image viewer says the DNGs are 300dpi so upscaling 4x shouldn't degrade when compared to H.264 (72dpi), especially if 1x1 crop
Title: Re: uncompressed 14-bit RAW video recording
Post by: sicetime on May 16, 2013, 05:37:47 PM
I feel like I get so lost in this forum, in a beautiful way.

Can someone point me to the latest module/nightly build for the 5diii?

Is it safe to assume that the 128gb komputerbay is slower compared to the 32 and 64?

Anyone running a 64gb komputerbay 1000x able to record constant 1080p?

I know that cinema5d/eoshd both used the 128gb on their tests, so are they lying about getting 1920x1080?

Or is it possible that older builds were able to pull that info?

As always, thanks!
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 16, 2013, 05:39:25 PM
Quote from: Shawn_Lights on May 16, 2013, 05:09:33 PM
The second shot to my eye had more detail and looked great. Was that the 480 setting?

If we can get a 550D build I'll engage in testing as well.

First was normal h264, second was 640x480 with raw resolution set to 1280x400.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shawn_Lights on May 16, 2013, 05:44:09 PM
Quote from: N/A on May 16, 2013, 05:39:25 PM
First was normal h264, second was 640x480 with raw resolution set to 1280x400.

Awesome. The raw blew the H264 away in my opinion. Also the aspect ratio was fine with me. Then again I'm an anamorphic fan.

Framing wise, does the liveview show 640x480?
Title: Re: uncompressed 14-bit RAW video recording
Post by: KahL on May 16, 2013, 05:47:19 PM
Any chance you guys have a few 720x480 DNG frames on hand (from the 600D)? I'd like to do some resolution and color tests here while I wait :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 05:49:45 PM
How are the 600D 32M buffers working? Raw speed with 32M went down for 6D, 600D originally had 31M buffers so maybe better or no change.
Title: Re: uncompressed 14-bit RAW video recording
Post by: tackyattack on May 16, 2013, 05:51:41 PM
Quote from: Shawn_Lights on May 16, 2013, 05:09:33 PM
If we can get a 550D build I'll engage in testing as well.

Yeah, if this comes to the 550D, I'd love to do some testing too  ;)
Really hope the 550D is actually capable of doing what the 600D has been doing.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kuky on May 16, 2013, 05:52:50 PM
I can confirm that a 5D mk 3 with Peter Linov posted firmware (from 12May) and a Sandisk Extrem Pro 90MB/s 16 Gb CF card    *CAN*    record 1920x1080 without skipping frames.

I don't know what commit number is that firmware but it does not appear to have tearing/stuttering. But didn't do many tests.

I had the impression that both my CF card had roughly the same speed:

- Lexar Professional 600x UDMA 16 Gb
- Sandisk Extreme Pro 90MB/s 16 Gb

Here are both cards benchmarked with commit b6c2ae6:

(https://lh6.googleusercontent.com/-fVfpntwXQ7c/UZT_0f00CgI/AAAAAAAAAhc/cjaHjKWT-vI/s800/lexar.jpg)

(https://lh3.googleusercontent.com/-_IAiYVdWkKU/UZT_0Vc5PZI/AAAAAAAAAhc/sFOxKc1e8Yk/s800/sandisk.jpg)


Bottom line:

Lexar GO HOME!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 05:56:50 PM
@1% - who are you asking? I'm not sure what build had what buffers but your latest one is the best so far.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 06:03:57 PM
I'm still only using a card with a 10MB/s write speed (half of what the buffer can handle on the 600d) but it writes 1280x400 RAW (ok, it's only 4 seconds lol). What are the faster cards getting with respect to frame sizes?
Title: To add to others, latest build for 600D is much better
Post by: mattmvpmedia on May 16, 2013, 06:04:20 PM
Just wanted to chime in that the latest build for 600D is so much better - thank you.

I did a 960x540 test and only had 1 or 2 frames of tearing/magenta blocks on a 10 second clip. Of course, the camera can't quite keep up with speed and drops a few frames on 960x540, so that may be related to the issue.

I did a 960x500 test and had a quick magenta block frame at the beginning, but after that it ran like a champ on my 20 second clip and looks great. No dropped frames at 900x500.

Thanks ML team- you rock.

Title: Andy600 - 1280x400 with no dropped frames?
Post by: mattmvpmedia on May 16, 2013, 06:08:09 PM
Hi Andy600,

can you record 1280x400 on the 600D with no dropped frames? I've got a SanDisk Extreme Pro 95MB/s card and I drop a few frames at that resolution. Thanks,

Matt
Title: Re: Andy600 - 1280x400 with no dropped frames?
Post by: ashtrai on May 16, 2013, 06:11:52 PM
Quote from: mattmvpmedia on May 16, 2013, 06:08:09 PM
Hi Andy600,

can you record 1280x400 on the 600D with no dropped frames? I've got a SanDisk Extreme Pro 95MB/s card and I drop a few frames at that resolution. Thanks,

Matt

If I've understood it correctly, the card speed doesn't matter as the XXXD cameras have a maximum SD-interface speed of ~21Mb/s.
Title: Re: Andy600 - 1280x400 with no dropped frames?
Post by: Andy600 on May 16, 2013, 06:15:07 PM
Quote from: mattmvpmedia on May 16, 2013, 06:08:09 PM
Hi Andy600,

can you record 1280x400 on the 600D with no dropped frames? I've got a SanDisk Extreme Pro 95MB/s card and I drop a few frames at that resolution. Thanks,

Matt

Yes, only 4 seconds though. I was looking at the wrong folder where I'd shot about 1600 frames continuously but at lowest dimensions. I've had a few shots where I didn't get any pink frames at all and others where they pop up randomly.

TW have you tried a low level reformat of your card? I did and it seemed to help a bit. I was also testing if formatting with bigger block sizes made a difference. The block size does affect things but not in the camera. Larger block sizes mean it takes ages to copy the small ML files back to the card after formatting so I'm gonna go back to the defaults.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 06:16:00 PM
Yep, but 21MB/s still depends on a card that can pull that off. Write size is important... 6D 32MB writes = 27MB and 30MB writes produced 28-29MB/s.. so on 600D if you were getting 19 previous build and now 17 or 18... thats what I'm asking about.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 06:18:01 PM
@ashtrai - yes that's right but my card only writes at half that (10MB/s). I'm just wondering if the faster cards are getting twice the performance?
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 16, 2013, 06:18:50 PM
Quote from: 1% on May 16, 2013, 05:49:45 PM
How are the 600D 32M buffers working? Raw speed with 32M went down for 6D, 600D originally had 31M buffers so maybe better or no change.
The 32M buffers definitely helped, 1280x480 runs longer until it skips frames, up from 5 seconds to 9 or 10 seconds. 1280x400 works just fine, a frame or two in the first couple seconds get corrupted then its smooth sailing from there. Still gotta test it out with running for a few minutes but as of now its much more stable than even a few days ago.

Its still stopping once it gets up to 21 MB/s continuous so even with a 95 MB/s card I'm thinking thats the limit.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 06:21:20 PM
@1% - what's the best way to test it? What should I be looking at? I have both builds

EDIT: Ignore that. I can see. Just loading previous build to compare
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 06:28:00 PM
Pick the same res and record.. see what the write indicator says.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 06:34:07 PM
@1% - Ok, big difference

shot both at 960x320

Latest build/latest raw_rec.mo = 316 frames before skipping 9.9MB/s
Previous build/previous raw_rec.mo = 70 frames before skipping 7MB/s
Title: Info on shooting raw with 600D at 960x500 with no dropped frames
Post by: mattmvpmedia on May 16, 2013, 06:53:18 PM
So far I've found that I can shoot at 960x500 raw with no dropped frames with Global Draw off. I'm using a SanDisk Pro Extreme 95MB/s SD card.

- If I turn Global Draw on with focus peaking and/or zebras on, it drops frames.
- However, I can have Global Draw on with the Histogram and waveform monitor on and it records with no dropped frames - just as long as focus peaking and zebras are turned off.
-Also, if I plug in the mini HDMI cable and connect to an external monitor I can continue to shoot with Global Draw on, histogram and waveform monitor on, and no dropped frames.

I think it may be time to shoot some real stuff :) Very pleased - thanks so much to the ML team.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 07:03:03 PM
Yup after testing, 32MB buffers aren't making a negative difference.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 16, 2013, 07:04:24 PM
Title: Re: uncompressed 14-bit RAW video recording
Post by: sicetime on May 16, 2013, 07:10:36 PM
Quote from: squig on May 16, 2013, 07:04:24 PM


Hey squig, which build are you working with for this? Which card are you writing to? What's your experience with it all, what are you able to pump out of the 600x?

Quote from the vimeo link
"5D MK3 Magic Lantern RAW May 13 build + Iscorama 1.5x anamorphic lens 1280x720 Lexar 600x card."
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 07:14:24 PM
Just switched EVERYTHING off in the menu. No dropped frames up to until I try 960x500 which is twice my cards write capability although no dropped frames at wider dimensions (just shorter busts). (I changed to 1080p 24fps in the canon menu btw). Slowly getting towards an optimal setting for my 600d/card. I can live happily with 10-20 sec bursts.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 16, 2013, 07:16:18 PM
May 13 (tear fix build) Lexar 600x 16Gb.. It's pretty solid, no dropped frames @ 1280x720 and no tearing or any other weirdness. I have to hit the menu button to stop recording which screws up the last 5 frames but other than that it's all good.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ernestoeduardo on May 16, 2013, 07:20:39 PM
Hi guys, can you please post the link for the latest 5d2 build ?


I have been getting a black line on the right side of my frame at 1880x720 on 5d2, it´s 16px wide, not dead black, but kind of purple-black. Will post images later. Also a pretty bad false color on a fence with a bright backlight.


Anyone can point to a free service where I can upload images to embed on my posts ?

Thanks
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 07:25:23 PM
Next thing to try now that I'm getting some nice footage is to convert to LOG intermediaries (in AE) for grading in Resolve. No one has mentioned doing that yet. Were still effectively dealing with REC709 images. LOG should be better for extreme grading and with the extra bit depth the color won't suffer. Fun time :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Redrocks on May 16, 2013, 07:40:57 PM
Quote from: hirethestache on May 16, 2013, 02:37:34 AM
Im going to catch flack for asking this again, but I did not see a response in the pages since posting.

I think the devs have moved away from HDMI hacking due to limitations of the tech, but they surprise us constantly. If you have an external recorder it might be worth selling it and buying a few fast CF cards.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 16, 2013, 07:46:37 PM
Has anyone tried 5D3's new firmware with ninja2? I am wondering if it's possible to record internally as well as through hdmi on ninja2.
When 5D3 firmware 1.2.1 is supported by ML, it would be nice to record prores proxy to ninja2 and to record 14bit raw internally. I may be wishing too much.
Title: Re: uncompressed 14-bit RAW video recording
Post by: CFP on May 16, 2013, 07:51:48 PM
Quote from: Andy600 on May 16, 2013, 06:18:01 PM
I'm just wondering if the faster cards are getting twice the performance?
On my EOS 600D with a card that reaches the 21 MB/s limit I was able to get:

- 1280 X 720 | stops after  71 frames | ~ 10 of them are purple frames
- 1280 X 400 | over 1500 frames | ~ 40 of them are purple frames
- 720 X 720 | over 1500 frames| ~ 30 of them are purple frames
- 960 X 540 | over 1500 frames | ~ 30 of them are purple frames

Settings: Golbal Draw = Off, 1080 24p mode, fps override = Off

The card was fresh formated. Quite nice, isn't it?

I also have that problem with these colorful dots if I check the images with Google Picasa ...
But when I open them in Lightroom 4 they disappear ???
Title: Re: uncompressed 14-bit RAW video recording
Post by: olik on May 16, 2013, 07:56:07 PM
Quote from: Andy600 on May 16, 2013, 07:25:23 PM
Next thing to try now that I'm getting some nice footage is to convert to LOG intermediaries (in AE) for grading in Resolve. No one has mentioned doing that yet. Were still effectively dealing with REC709 images. LOG should be better for extreme grading and with the extra bit depth the color won't suffer. Fun time :)

how do you mean? raw is always Logarithic. I converted the adobe-dngs with AFX to DPX Full range "log" and imported those into resolve. You will lose alot of the flexibility of raw (my afx does not support cinema-dng yet) but still mindblowing what you can do in terms of grading with it. Oh I have not tried it yet, but can cs5 export .R3D? maybe that is the best road to go.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shawn_Lights on May 16, 2013, 07:59:50 PM
Quote from: squig on May 16, 2013, 07:16:18 PM
May 13 (tear fix build) Lexar 600x 16Gb.. It's pretty solid, no dropped frames @ 1280x720 and no tearing or any other weirdness. I have to hit the menu button to stop recording which screws up the last 5 frames but other than that it's all good.

How many secs do you get on a take?
Title: Re: uncompressed 14-bit RAW video recording
Post by: iaremrsir on May 16, 2013, 08:02:26 PM
Quote from: Andy600 on May 16, 2013, 07:25:23 PM
Next thing to try now that I'm getting some nice footage is to convert to LOG intermediaries (in AE) for grading in Resolve. No one has mentioned doing that yet. Were still effectively dealing with REC709 images. LOG should be better for extreme grading and with the extra bit depth the color won't suffer. Fun time :)

VisionColor is developing VisionColor, VisionLog, and CineTech for Adobe Camera Raw and Lightroom.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shawn_Lights on May 16, 2013, 08:04:39 PM
Quote from: CFP on May 16, 2013, 07:51:48 PM
On my EOS 600D with a card that reaches the 21 MB/s limit I was able to get:

- 1280 X 720 | stops after  71 frames | ~ 10 of them are purple frames
- 1280 X 400 | over 1500 frames | ~ 40 of them are purple frames
- 720 X 720 | over 1500 frames| ~ 30 of them are purple frames
- 960 X 540 | over 1500 frames | ~ 30 of them are purple frames

Settings: Golbal Draw = Off, 1080 24p mode, fps override = Off

The card was fresh formated. Quite nice, isn't it?

I also have that problem with these colorful dots if I check the images with Google Picasa ...
But when I open them in Lightroom 4 they disappear ???

That's damn good. purple frames popping up randomly or only in the beginning?
Title: Re: uncompressed 14-bit RAW video recording
Post by: danielnix on May 16, 2013, 08:18:53 PM
Im using the mark iii and would like to shoot in RAW video, I keeps seeing video posting so I know it can be done. I will pay someone VIA pay-pal to guide me through the process. Im having a problem. when I put the autobin file on the card and I boot the disc nothing happens when I put it in my camera.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 16, 2013, 08:21:53 PM
600d users - try switching everything OFF in the menu. I'm not getting pink frames after doing it.

@iaremrsir - they might well be but I'm not a fan of their colors or presets in general ;) The whole point of shooting with a stylised Picture Style was because we were limited to 8bit H.264 and getting the look right in camera was an easier option. The 8bit video image would fall apart in post when pushed too far. RAW gives you freedom to grade how you want.

BTW you can match any picture style if you know how to make and use LUTs. I've already made a couple of LUTs to match the Canon profiles for my own flat PS.

@Olik - I thought RAW is Linear as it's straight from the sensor? We can't currently import DNG into Resolve as is so it would need an intermediary. I'm thinking convert to LOG in AE then output DNxHD.
Title: Re: uncompressed 14-bit RAW video recording
Post by: CFP on May 16, 2013, 08:24:28 PM
Quote from: Shawn_Lights on May 16, 2013, 08:04:39 PM
purple frames popping up randomly or only in the beginning?
No. It's more like:

First 120 frames are fine. Then every sixth frame is purple for about 3 seconds. Then the next 250 frames are fine again. Then again every every sixth is purple for about 3 seconds. It's strange, but way better than it was with the first versions.

The numbers vary but that seems to be the scheme.

I was using todays Nightly Build and "BeepsTwice" raw_rec.

But I will try what Andy600 suggested ... Unless I'm not sure if that'll have any affect since Global Draw is already off.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mujart on May 16, 2013, 08:37:30 PM
Dear All

i was installed ML fro my 5d MK iii and he is istalled but say"Script dir missing" and i press "Load Modules" and nothing happens, there is no RAW Video line basically,
i was tried to repeat the installation, BUT the camera shut down so i chenged the battery (its full) and its work several times till the camera still not working >> when make the camera ON there is nothing happen !!

i just downloaded ML files from this: https://docs.google.com/file/d/0B7QlH_BH2m32dWpOOEpkWktXMXM/edit and follow this instructions http://www.cinema5d.com/news/?p=17898


please is there any solution????
Title: Re: uncompressed 14-bit RAW video recording
Post by: haysuess on May 16, 2013, 09:15:40 PM
This is so awesome to play with!

I have a Mark II, and a Transcend 400x 32GB card. When I run the benchmarks, my write speed is around 37.5-50MB/s, only at 16384k buffer is it under 40, and with the rest it's usually closer to 50. Read speeds are around 65MB/s consistently.

The anomaly is at 128k buffer it's only 25.5MB/s write speed :(

I'm trying to record at 1280x720 at 24fps, and it says i only need 36.9MB/s to record, however, I only get a few seconds before it starts skipping bad and automatically stops.

Even at 1280 x 540, which says it only requires 27.6MB/s it starts skipping within 10 seconds.

I can record without skipping at 960 x 540, but I still get a lot of magenta blocking.

I tried with both the May 14, and May 16 nightly builds and the 5D Mark II record module and autoexec.bin that someone else posted.

Is there something I'm missing, or is this normal?
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 16, 2013, 09:17:30 PM
Quote from: iaremrsir on May 16, 2013, 08:02:26 PM
VisionColor is developing VisionColor, VisionLog, and CineTech for Adobe Camera Raw and Lightroom.

Amazing! Hooray for visioncolor :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: JackDaniel412 on May 16, 2013, 09:24:41 PM


Test on 5d2
Title: Re: uncompressed 14-bit RAW video recording
Post by: silvertonesx24 on May 16, 2013, 09:27:34 PM
Idea: It would be nice to automatically toggle/disable global draw upon recording. Then you could set all of your levels etc and when you hit record have maximum performance.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shawn_Lights on May 16, 2013, 09:32:51 PM
Quote from: silvertonesx24 on May 16, 2013, 09:27:34 PM
Idea: It would be nice to automatically toggle/disable global draw upon recording. Then you could set all of your levels etc and when you hit record have maximum performance.

Definitely. Perhaps a shortcut defined button.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 16, 2013, 09:34:03 PM
It's not very clear where I can find the latest version to test on my 5D Mark III? Anyone plz?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 09:34:33 PM
On other cameras you don't need global draw off... this still sounds like something is drawing where it shouldn't or something uses that edmac channel.
Title: Re: uncompressed 14-bit RAW video recording
Post by: vitos1k on May 16, 2013, 09:40:33 PM
Quote from: haysuess on May 16, 2013, 09:15:40 PM
This is so awesome to play with!

I have a Mark II, and a Transcend 400x 32GB card. When I run the benchmarks, my write speed is around 37.5-50MB/s, only at 16384k buffer is it under 40, and with the rest it's usually closer to 50. Read speeds are around 65MB/s consistently.

The anomaly is at 128k buffer it's only 25.5MB/s write speed :(

I'm trying to record at 1280x720 at 24fps, and it says i only need 36.9MB/s to record, however, I only get a few seconds before it starts skipping bad and automatically stops.

Even at 1280 x 540, which says it only requires 27.6MB/s it starts skipping within 10 seconds.

I can record without skipping at 960 x 540, but I still get a lot of magenta blocking.

I tried with both the May 14, and May 16 nightly builds and the 5D Mark II record module and autoexec.bin that someone else posted.

Is there something I'm missing, or is this normal?

global draw off, and also set FPS override to 24
Title: Re: uncompressed 14-bit RAW video recording
Post by: robert_b on May 16, 2013, 09:42:34 PM
 this is what I'm getting with a 45MB/s Sandisk memory card
  at 1120 by 480 using a 600D, the settings are at default, I'll do some test with the global draw turned off
Title: Re: uncompressed 14-bit RAW video recording
Post by: vipmask on May 16, 2013, 09:49:45 PM
If you first do a write benchmark card and then start recording there is no drop-down shots! Probably during the test memory is freed. 
Title: Re: uncompressed 14-bit RAW video recording
Post by: KenFTW on May 16, 2013, 10:11:01 PM
camera: 600D
card: Sandisk Extreme Pro 95mb
Global draw: off
bitrate set at 20x

1280x360: 1 of 840 frames was unusable (completely black), no magenta blocking whatsoever, no skipped frames, and I stopped the recording

CRAZY!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 16, 2013, 10:19:28 PM
Just from filming the concert.
And I found a few minutes to record RAW, My boss will be very sad, but I did it!
But now the video in RAW, I was not able to record more than 20 seconds without dropping frames.
I just need a high-speed  flash !
And...By the way, I do a little correction on RAW, and it's not very good ....
You can see this for yourself:



If you are interested in an uncompressed download here:
http://yadi.sk/d/OAtKYdwJ4uPav
Title: Re: uncompressed 14-bit RAW video recording
Post by: AnotherDave on May 16, 2013, 10:26:12 PM
Sadly, I've upgraded to 1.2.1 and my 5D3 is being used with an Atomos for a project... so I can't test this builds out until it is done.

But, I was thinking... the main problem it seems we're dealing with here is the write speed of the CF cards, right?

There are plenty of companies that make CF to SSD converters... there HAS to be a way to adapter SSD to CF.  Right?

That would solve all the issues of speed, and storage with raw shooting.  Some sort of external module with a cable connector
and a pin converter...

Or am I just being hopeful?

Dave
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 16, 2013, 10:29:11 PM
These are my little tests. The very short edits are because this is how long I can record at 1880x720. You may also notice some jumping too. Of course this is far from perfect.

http://www.youtube.com/watch?feature=player_detailpage&v=p9uZXUGDJSA

Recorded on the 5DMKII with DURACELL Pro Photo 600x UDMA 16GB CF card. I think the build I was using was the one from the 13th?

I have ordered a Komputerbay 1000x 32Gb and am hoping that it will improve the situation (reading the posts on here I am glad I chose the 32Gb one instead of the 128Gb)
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 10:32:50 PM
QuoteAtomos for a project.

The HDMI recording at this point is a joke. As long as your card is fast enough even under sized/upscaled raws will be better than what the HDMI puts out. I think we kinda peed on  their "upgrade".

Someone should do a test... record raw 24P... record HDMI YUV (we could take a crop of the regular YUV and record too, couldn't we?) uprez to the same size and compare :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 16, 2013, 10:35:31 PM
Just chuckled to myself. Uploading these lovely RAW videos to YouTube is like sending it back through the 5D h264 algorithm!!!

(actually it probably is h264 compression anyway!?!)

But its still funny getting the very best quality footage to essentially degrade it to bits :D
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 16, 2013, 10:41:15 PM
we must look at the picture. you see the difference? see everything. is it good? or is it not necessary to lay out, because who thinks otherwise?
My clients do not need to RAW, it is important to the quality of accessible format to watch at home.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 10:49:22 PM
While its funny that they are getting H264'd again its much better to start with a non 8 bit image when post processing.

Clients may not need raw but they may need corrections that would fall apart at 8bit... if any of your clients need chroma key and you have to do it in 8 bit  :o
Title: Re: uncompressed 14-bit RAW video recording
Post by: mujart on May 16, 2013, 10:49:56 PM
i use this files in my 5D MK III,
https://docs.google.com/file/d/0B7QlH_BH2m32dWpOOEpkWktXMXM/edit

the problem there is the line of RAW VIDEO  NOT exist basically

like this photos:

(http://s16.postimg.org/4m6bpqqv9/photo.jpg)


and when press on load modules nothing happend !!

is there any solution??
Title: Re: uncompressed 14-bit RAW video recording
Post by: Alia5 on May 16, 2013, 10:52:07 PM
1%: Like i said a few pages ago, with the new "beeps twice" (wich doesn beep twice for me) i get ZERO magenta frames when recording in 720x400
going to higher resolutions, and the buffer fills up more, i get this magenta stuff again!

i really think it has something to do with sd-writespeeds/the buffers

global draw is always off
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 10:53:09 PM
Yup, could be that too. I think we need even more buffer :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: AnotherDave on May 16, 2013, 10:53:41 PM
I don't want this post to get off topic, but yes HDMI out is indeed an upgrade.

It may not be as clean as RAW, but storage-wise and being able to go straight into an NLE is pretty amazing.

I'd rather shoot raw, but when your camera is working for a living... speed is everything.
Title: Re: uncompressed 14-bit RAW video recording
Post by: vipmask on May 16, 2013, 11:06:38 PM
I was able to write without dropping frames
1920x960 25 frames per second!
1920x672 50  frames per second!
Transcend  32gb 1000x
I just wrote above how I got it.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Mei Lewis on May 16, 2013, 11:14:59 PM
Quote from: AnotherDave on May 16, 2013, 10:26:12 PM
Sadly, I've upgraded to 1.2.1 and my 5D3 is being used with an Atomos for a project... so I can't test this builds out until it is done.

But, I was thinking... the main problem it seems we're dealing with here is the write speed of the CF cards, right?

There are plenty of companies that make CF to SSD converters... there HAS to be a way to adapter SSD to CF.  Right?

That would solve all the issues of speed, and storage with raw shooting.  Some sort of external module with a cable connector
and a pin converter...

Or am I just being hopeful?

Dave

How about recording to the CF and SC cards in parallel? Twice the capacity and ???x the speed?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Alia5 on May 16, 2013, 11:18:10 PM
okay i checked a bit more on 600D
no magenta frames on 640x480 mode... even when recording in 1120x400

same size in 720p mode, 1080p mode or photo-mode

so... its not the buffer...(?)
Title: Re: uncompressed 14-bit RAW video recording
Post by: haysuess on May 16, 2013, 11:18:47 PM
Quote from: vitos1k on May 16, 2013, 09:40:33 PM
global draw off, and also set FPS override to 24

I have global draw off, and FPS override set to 24 exact FPS. Any other ideas? :-/
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 16, 2013, 11:25:34 PM
Quote from: mujart on May 16, 2013, 10:49:56 PM
i use this files in my 5D MK III,
https://docs.google.com/file/d/0B7QlH_BH2m32dWpOOEpkWktXMXM/edit

the problem there is the line of RAW VIDEO  NOT exist basically


and when press on load modules nothing happend !!

is there any solution??

That is an old build. I just replaced those old files with newer ones from yesterday. You would want re-download that.  I will be posting a newer one later tonight given new updates just came out. 
Title: Re: uncompressed 14-bit RAW video recording
Post by: vipmask on May 16, 2013, 11:28:35 PM
http://yadi.sk/d/gx75Q5t04uUQ0
1920x1080 25fps
Frame skip: 1
5D Mark III
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 16, 2013, 11:31:34 PM
Quoteso... its not the buffer...(?)

CPU and memory then? Or too much edmac usage on other channels... YUV buffer is much smaller in 640x480.

So I've tried the HDR and while you wouldn't know it from the screen it seems to work. Preview was flashy.


Indicators had to be removed from "faking" recording... I did it for my 6D stuff. I still have some sound stuff to rework. Have to come up with something... I have FPS wav, H264 wav and now raw_rec wav.... so I only have input changing/audio meters set up for H264 wav which is kind of funny. I think we need a unified "record wav" flag to handle all of these modes. Raw histos and spot meter showing up too.. I'm not seeing a hit with the write or anything.

Ok, slight hit to write from the spot meter. Not the histo tho.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Chris F on May 16, 2013, 11:39:38 PM
Kind of hard to follow all this. From what I think I'm reading, there likely won't be a way to record raw on the Rebel cameras in a resolution much better than SD. While that dynamic range is great, it doesn't look a whole lot better in resolution after it has been upscaled than the H264. Is there likely a way that we will be able to record a better quality 1080p (or close to it) video than what the H264 can do? MJPEG? JPEG sequence?
Title: Re: uncompressed 14-bit RAW video recording
Post by: vipmask on May 16, 2013, 11:41:27 PM
Quote from: lourenco on May 16, 2013, 11:25:34 PM
That is an old build. You should try a newer build.  I will be posting a newer one later tonight given new updates just came out. 

Here is one from yesterday you can update to. You just want to overwrite the old files with these new ones. 

5D3:  https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/ML/ML-5D3-RAW_VIDEO.zip
I'll try this new one. It frames fall!

It is on this I was able to record 1920 x 1080 25 fps and 1920 x 1280 25 fps :D
http://yadi.sk/d/0ogqyTkb4uUvL
Title: Re: uncompressed 14-bit RAW video recording
Post by: KMA_WWC on May 16, 2013, 11:43:33 PM
Just returned my Komputerbay 128gb card and ordered 3x 64gb cards. Will be testing it and see if there is any deference between them.

Actually what I realized is that even komputerbay tests show that 64gb is a lot faster than 128gb.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Alia5 on May 16, 2013, 11:48:20 PM
if i record with sound on my 600d (ml audiorecorder), i either loose wav, get skipped frames, or camera is crashing instantly!
except i record in really tight crops....

if im in 640x480 mode i still dont get any magenta frame
i feel like photo-mode works best after 640x480, but dont rely on that! just a feeling didnt test this much

Edit: can confirm this! photo mode (m) and 640x480 work both without any magenta frame! /Edit

besides an interesting fact:
the higher the iso the more fixed (noise?) dots i get when viewing the dngs with picasa...
just like beeing said it doesnt matter in acr
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 12:20:18 AM
Yep.. not enough to record audio with most resolutions. On 6D too.. get a ton of wav drops unless I lower the res. On 600D it did actually crash if the res was too high.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 17, 2013, 12:23:18 AM
Quote from: 1% on May 16, 2013, 10:49:22 PM
While its funny that they are getting H264'd again its much better to start with a non 8 bit image when post processing.

Clients may not need raw but they may need corrections that would fall apart at 8bit... if any of your clients need chroma key and you have to do it in 8 bit  :o

Totally agree!!! still funny  ;D ::)
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 17, 2013, 12:30:12 AM
OK a better rendered version of my previous YouTube 5DMKII RAW test upload:

YouTube here:

http://www.youtube.com/watch?v=u47aj1h0m6k&list=PLWUV9WfHlRNNBbQ98y3J99YbxWiK-AMSq&feature=player_detailpage

and for the actual file here at mega:

https://mega.co.nz/#!Ut0mxSzJ!ZfuzVXUykfAmarhxQXZqhCRUagFTVBbZFbxwlDE-wsE
Title: Re: uncompressed 14-bit RAW video recording
Post by: tonybeccar on May 17, 2013, 12:32:50 AM
Quote from: vipmask on May 16, 2013, 09:49:45 PM
If you first do a write benchmark card and then start recording there is no drop-down shots! Probably during the test memory is freed.

Excuse me, I read correctly? 50fps at almost 1080p??????! Is this how you said you accomplished it?

Why did your comment passed unseen? It is freaking awesome!!!!!!!!!
Title: Re: uncompressed 14-bit RAW video recording
Post by: vipmask on May 17, 2013, 01:09:48 AM
Quote from: tonybeccar on May 17, 2013, 12:32:50 AM
Excuse me, I read correctly? 50fps at almost 1080p??????! Is this how you said you accomplished it?

Why did your comment passed unseen? It is freaking awesome!!!!!!!!!
Tomorrow afternoon I will take the test at 50 frames sun. :-) And let's see. When unpacking the information that was not missed shots. I'll be shocked. Could not understand how this is possible ...
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 17, 2013, 01:31:30 AM
Quote from: platu on May 16, 2013, 12:42:55 AM

I just ordered and received the same CF card as you... the KomputerBay 128gb 1000x CF card and have been running tests with each of the various Raw builds over the last 3 days.  The latest build for 5D3 someone just posted from today seems to be the best in terms of speed... see http://www.magiclantern.fm/forum/index.php?topic=5413.25 (Reply #30).  This includes some small memory tweaks that definitely helped me get a little bit longer runtimes as well as moved me up to the next resolution without dropped frames.

Here has been my experience with the KomputerBay 128 GB 1000x CF card...

For my tests, I tried Global draw turned ON and OFF and there was no significant difference except maybe a second or two extra of recording time but the extra time was not consistent.  So I decided to leave Global Draw = ON (with just peaking and crop marks enabled).


My tests using the ML benchmark utility shows read speeds that range anywhere from 52MB/s to 72MB/s.  Using CrystalDiskMark and ATTO Disk Benchmark, I get similar write speeds so my particular card seems to have a write speed that varies a bit.  I would say the average write speed that I get most of the time is about 65MB/s which is completely in line with the ML benchmarks and the listed speed needed by each resolution listed under the RAW video section of ML.  So I would trust the ML benchmarks and recommended write speeds for each resolution.  I have also confirmed that at least one other person is getting similar write speed using this card.

That said, the buffer/memory tweak Alex made today did have a significant impact when using my card. 

Here are my tests Before today's build...
1920 x 720 or less resolution (no skipped frames for complete 4gb file)
1920 x 840 (no skipped frames for complete 4 gb file)... before today's build, this was the best I could do.
1920 x 900 (no skipped frames for complete 4 gb file)... tweaks added to today's build allowed me to use resolution for first time
1920 x 960 (maybe 10 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (maybe 1-2 seconds before frame skipping begins... unusabe beyond that)

After using today's build...
1920 x 900 (no skipped frames for complete 4 gb file) before today's build, I could not use this resolution
1920 x 960 (went from 10 seconds to 20 seconds before frame skipping begins.. unusable beyond that)
1920 x 1080 (went from 1-2 seconds to 5 seconds before frame skipping begins... unusabe beyond that)

So the tweaks made by Alex today have definitely helped so I encourage further memory/buffer optimizations... they are making a difference.

I don't know how EOSHD, Nuemann Films, and Cinema 5D are getting 1080P and greater without dropped frames. They say they are using the same card.  Maybe some of these cards are getting faster write times.  Or maybe they are just getting that resolution for a limited time but a bit longer than me before frame skipping appears.  If any of them can chime in here to clarify, it would helpful for those considering the Komputerbay cards.  This card is by far the most realistic in terms of pricing for the vast majority of people who want to take advantage of this RAW update and plan on using it beyond test videos and personal work.  None of the other cards comes close in terms of price/gb.  The other solutions by Lexar and Toshiba (soon) are more like $650 for 128gb vs $178 or KomputerBay. I'm sure this will eventually change, but that could take a year or longer before cheaper alternatives are available.  But there is no getting around the fact that the less than ideal write speed of these cards may prevent continuous 1080p for some (depending on their particular card?).  But continued memory optimization and other techniques by the ML team may be able to extend 1080p recording from 5 seconds to 30 seconds or possibly more when using this card.  If that can be attained, the vast majority of folks will be able use this resolution in many shooting scenarios, short of documentary or event work.  I do realize that smaller resolutions can be scaled up in post nicely, but I tend to avoid that as do many others I'm sure.

Lastly, on the topic of workflow mentioned above... it's completely worth the time as far as I'm concerned.  The difference is night and day.  There are a lot of test videos coming out now, some of which don't fully capture the quality improvement gained here.  EOSHD, Nuemann Films, and Cinema 5D did great job of showing what's possible.  Just wait until some shorts and features by other skillful DOPs start to appear online.  While I don't love the added work required in post, I find it impossible to go back to H.264 after getting used to the look of my footage now...it's improved that much.

Thank you Alex and team for this... also to g3gg0 who apparently had a huge role in making this particular breakthrough happen.

A little update regarding the KomputerBay cards...

You may want to avoid recommending the 128GB version of the KomputerBay CF card. Evidence is building that the 128 version had slower real world write speeds than the 64. This prevents 1080p raw recording since the 128 doesn't seem to attain the 83MB/s minimum write required for 1080p raw recording. The 64GB version seems to test at 90+ MB/s write while the 128GB version tests at 60-70 MB/s write.  It turns out that Andrew from EOSHD was using the 64gb variety of the KomputerBay card for his Raw tests, which is why he was able to utilize even greater than 1080p raw without dropped frames.  This is all still unproven but I just returned my 128gb and ordered two 64gb to see for myself.
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 17, 2013, 01:36:21 AM
Current 5D3 update files only : https://drive.google.com/folderview?id=0B7QlH_BH2m32cW5mUW80VFlzSWc&usp=sharing

Complete Current 5D3 : https://docs.google.com/file/d/0B7QlH_BH2m32dWpOOEpkWktXMXM/edit?usp=sharing

Current as of last commit : fixed edmac_memcpy to clean up CBRs correctly.
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 17, 2013, 01:40:43 AM
i just committed another fix (rev 6919)
Title: Re: uncompressed 14-bit RAW video recording
Post by: cinema5D on May 17, 2013, 01:53:58 AM
Quote from: platu on May 17, 2013, 01:31:30 AM
You may want to avoid recommending the 128GB version of the KomputerBay CF card. Evidence is building that the 128 version had slower real world write speeds than the 64.

Hey there. We're using 2 Komputerbay 128GB cards and they're both holding up great, we never get dropped frames except for the first recording after formatting usually and we shot lots and lots of things already, on both cards. We also tried the higher resolutions, but the firmware limits it to not much more than 1080p.

Here's the simplest post workflow in OSX we found around here, and we think it's most easy after going through hell two days ago: https://vimeo.com/66355682 (https://vimeo.com/66355682)

After that we tried to do some nightshots but unfortunately we couldn't manage to get the shutter locked. If there is any info you can give us, or if that is a feature not yet enabled please let us know.
Specifically since we're in Europe we need a 1/50th shutter, but the system keeps jumping back to 1/46th no matter what we do/change/select/lock in the menu. Thanks for any hints.

Also 25p would be tremendous. We had a discussion today about wether the Europeans would be taken care of. We've had very bad experience there with the manufacturers in the past  :-X.

Keep up the amazing work, you guys are building something thats exciting for all movieshooters :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 17, 2013, 01:54:31 AM
Quote from: lourenco on May 17, 2013, 01:36:21 AM
Current 5D3 update files only : https://drive.google.com/folderview?id=0B7QlH_BH2m32cW5mUW80VFlzSWc&usp=sharing

Complete Current 5D3 : https://docs.google.com/file/d/0B7QlH_BH2m32dWpOOEpkWktXMXM/edit?usp=sharing

Current as of last commit : fixed edmac_memcpy to clean up CBRs correctly.
I have a question. Is there a reason as to why you add the 'raw2dng' file in all of the uploads?
Im just curious if thats something thats supposed to be placed on my memory stick, or is that simply for the computer to convert the recorded raw files and you only upload so the downloaders will have it in case they never got it?
Title: Re: uncompressed 14-bit RAW video recording
Post by: KMA_WWC on May 17, 2013, 01:57:45 AM
Quote from: platu on May 17, 2013, 01:31:30 AM
A little update regarding the KomputerBay cards...

You may want to avoid recommending the 128GB version of the KomputerBay CF card. Evidence is building that the 128 version had slower real world write speeds than the 64. This prevents 1080p raw recording since the 128 doesn't seem to attain the 83MB/s minimum write required for 1080p raw recording. The 64GB version seems to test at 90+ MB/s write while the 128GB version tests at 60-70 MB/s write.  It turns out that Andrew from EOSHD was using the 64gb variety of the KomputerBay card for his Raw tests, which is why he was able to utilize even greater than 1080p raw without dropped frames.  This is all still unproven but I just returned my 128gb and ordered two 64gb to see for myself.
Hey there! That's what I did today! Check reply #1067
I also noticed that komputerbay tests itself show that 64gb is much faster than 128gb, I don't know how I missed that before buying.
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 17, 2013, 01:58:47 AM
Another 600D Raw video.



Rokinon Cine, Sandisk 95. Rendering the DNG's out of After Effects into ProRes 4444, loading them up into FCPX and doing the rest of the editing.

Compressed with Mpeg StreamClip from ProRes 4444 to mp4 for Youtube uploading.
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 17, 2013, 01:59:32 AM
Quote from: aaphotog on May 17, 2013, 01:54:31 AM
I have a question. Is there a reason as to why you add the 'raw2dng' file in all of the uploads?
Im just curious if thats something thats supposed to be placed on my memory stick, or is that simply for the computer to convert the recorded raw files and you only upload so the downloaders will have it in case they never got it?

You may not need to worry about the raw2dng  file if you have one. There was a update about 2 days ago. If you already have that current version, you do not need to replace it.
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 17, 2013, 02:04:13 AM
Quote from: lourenco on May 17, 2013, 01:59:32 AM
You may not need to worry about the raw2dng  file if you have one. There was a update about 2 days ago. If you already have that current version, you do not need to replace it.
O Ok, thanks for clarifying that.
Im using the one built by a forum member here.
the mac app version.
Hopefully it doesn't need an update. If you don't mind me asking, what was updated in it? Was it the ability to correctly split the 4gb files or something like that?
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 17, 2013, 02:06:19 AM
Quote from: aaphotog on May 17, 2013, 02:04:13 AM
O Ok, thanks for clarifying that.
Im using the one built by a forum member here.
the mac app version.
Hopefully it doesn't need an update. If you don't mind me asking, what was updated in it? Was it the ability to correctly split the 4gb files or something like that?
"a1ex committed 2 days ago (raw commit)
raw2dng: fix for files larger than 2GB
"
It works fine on 4GB files.
Title: Re: uncompressed 14-bit RAW video recording
Post by: KMA_WWC on May 17, 2013, 02:08:31 AM
Lourenco, first of all huge thanks for the new build. Just installed it. One question though. Is there any way to change the resolution like in old builds or is it 1080p only? thanks in advance!

UPD: Never mind found it :) thanks
Title: Re: uncompressed 14-bit RAW video recording
Post by: rossangeles on May 17, 2013, 02:22:36 AM
Did a quick test under less than ideal circumstances. The kind that I'm usually shooting in. Movement. Shadows and light. Highlight recovery. Skin tones. Canon 5d Mark III. 24 mm L series. 3.2 KomputerBay 128 mb. 1080 x 720  http://rossangeles.org/rosspile/trucking_web.mov 
Title: Re: uncompressed 14-bit RAW video recording
Post by: eatstoomuchjam on May 17, 2013, 02:57:16 AM
1920x1080p on my 5D Mark III (and 24-105/4L) from a few days ago at the celebration for legal marriage equality in Minnesota.  The footage is pretty crude since I was just  goofing around handheld.  This was done with a build known for light frame skipping so I was getting about 8 skipped frames per 4G file on a Lexar 32G 1000x card.  After tweaking the footage in AE, the intermediate renders in 10-bit DNxHD look good enough to keep (and Premiere plays them back like a dream).
Incidentally, it was about 90 degrees and I was shooting off and on for a couple of hours.  No overheating problems evident.

http://www.youtube.com/watch?v=zckSN5j5Cvo&feature=youtu.be
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 17, 2013, 02:58:43 AM
Quote from: Shawn_Lights on May 16, 2013, 07:59:50 PM
How many secs do you get on a take?

I haven't timed it but I can keep recording up to the 4.2Gb limit.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 17, 2013, 03:02:55 AM
Quote from: iaremrsir on May 16, 2013, 08:02:26 PM
VisionColor is developing VisionColor, VisionLog, and CineTech for Adobe Camera Raw and Lightroom.

Perfect! I guess I can stop working on the sLUT then. I really like visioncolor/tech, it's a good improvement on prolost.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 03:06:58 AM
I feel like the refactoring lowered write speed somehow.. I'm getting less frames.... either that or the card is getting slower?

Now in photo mode non LV it takes all the memory and you have to turn it off before LV will start.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Digital Corpus on May 17, 2013, 03:09:48 AM
@platu, glad I can give some insight.

Quote from: vipmask on May 16, 2013, 11:41:27 PM
I'll try this new one. It frames fall!

It is on this I was able to record 1920 x 1080 25 fps and 1920 x 1280 25 fps :D
http://yadi.sk/d/0ogqyTkb4uUvL
1920x1280 @ 25 fps --> 2,457,600 pixels, 14 bpp @ 25 fps --> 102.5 MB/sec
If it was at 24 fps, you'd only need about 98.4 MB/sec.

As demonstrated by real world testing of 1080p RAW video, on would need a card that performs few extra percent, ~3%-5%, above this number in order to safely record the video without dropped frames.

Quote from: mageye on May 16, 2013, 10:29:11 PM
These are my little tests. The very short edits are because this is how long I can record at 1880x720. You may also notice some jumping too. Of course this is far from perfect.

http://www.youtube.com/watch?feature=player_detailpage&v=p9uZXUGDJSA

Recorded on the 5DMKII with DURACELL Pro Photo 600x UDMA 16GB CF card. I think the build I was using was the one from the 13th?

I have ordered a Komputerbay 1000x 32Gb and am hoping that it will improve the situation (reading the posts on here I am glad I chose the 32Gb one instead of the 128Gb)
1880x720 @ 24 fps --> 1,353,600 pixels, 14 bpp @ 24 fsp --> ~54.2 MB/sec
This speed coincides directly with your memory card. That KomputerBay one will do the trick.

Quote from: 1% on May 17, 2013, 03:06:58 AM
I feel like the refactoring lowered write speed somehow.. I'm getting less frames.... either that or the card is getting slower?

Now in photo mode non LV it takes all the memory and you have to turn it off before LV will start.
If you're still having write speed issues, try a low level or full format of the card to see if that helps.
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 17, 2013, 03:15:23 AM
Quote from: 1% on May 17, 2013, 03:06:58 AM
I feel like the refactoring lowered write speed somehow.. I'm getting less frames.... either that or the card is getting slower?

Now in photo mode non LV it takes all the memory and you have to turn it off before LV will start.

Same here... getting less frames as well.   Also, the 900 vertical resolution was removed since the last build.  1920x900 was my maximum baseline resolution I was using that had no dropped frames.  Can this be added back?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 03:30:40 AM
I still kept 900. Now it seems write starts of fast and then drops off slowly. I was doing 1500 frames now doing way less same res.
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 17, 2013, 03:34:54 AM
Quote from: platu on May 17, 2013, 03:15:23 AM
Same here... getting less frames as well.   Also, the 900 vertical resolution was removed since the last build.  1920x900 was my maximum baseline resolution I was using that had no dropped frames.  Can this be added back?

https://drive.google.com/folderview?id=0B7QlH_BH2m32Q1FMeHBBQVVmb3M&usp=sharing 
Title: Re: uncompressed 14-bit RAW video recording
Post by: Luiz Roberto dos Santos on May 17, 2013, 03:35:29 AM
Quote from: N/A on May 17, 2013, 01:58:47 AM
Another 600D Raw video.



Rokinon Cine, Sandisk 95. Rendering the DNG's out of After Effects into ProRes 4444, loading them up into FCPX and doing the rest of the editing.

Compressed with Mpeg StreamClip from ProRes 4444 to mp4 for Youtube uploading.


What is your maximum recording time?
Very good, we are progressing here! :D
Title: Re: uncompressed 14-bit RAW video recording
Post by: KMA_WWC on May 17, 2013, 03:48:45 AM
Not trying to be a controversial, but new build is much improved for me.

Here are the results:

Old build May 14:

No drop frames before 1080p
1920x1080 7-9 dropped frames
1920x1152 3-5 seconds of recording and after that drop frames all over the place

New build, today's:
1920x1080 no drop frames at all.
1920x1152 15-20 second of recording/vary

I use SanDisk 32BG Extreme Pro card which performs btw pretty good.
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 17, 2013, 03:51:47 AM
Quote from: lourenco on May 17, 2013, 03:34:54 AM
https://drive.google.com/folderview?id=0B7QlH_BH2m32Q1FMeHBBQVVmb3M&usp=sharing

Thanks.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 17, 2013, 03:58:34 AM
Does anyone know when we may expect to be able to test file spanning?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Roars on May 17, 2013, 04:19:49 AM
so i have a couple of thought to throw out hear

what is the Colour Chroma Subsampling when shooting in Raw is it 4:2:2 or 4:4:4 the 5d is 4:2:0 as standerd

could some of the issues with the Sd cards be solved by using a SSd, ie a ninja (cos you can get a better bit rate [10-12mb] out of the hdmi export onto a ninja than you can standerd [8mb]) but i dont think that would work cos you cant export raw data down the hdmi maybe if you could retro fit a ssd direct to the 5d.

and the bit rate is that hardwere limeted to 14 bits or the hack or the card

more over how do these raw files stack up to red as to my knowledge there the only people to use raw (i think arri might to)

why does all the raw test footage look abit darker is that simply a byproduct of more detail

so raw on dslrs has been proven so why has cannon not thought of this

also what are you guy editing on final cut 7 cant realy handle HD let alone raw. i was told only avid could truly edit .raw but you can convert it [woudnt you loos the ability to edit the exposure and stuff if you did that]

are you useing redcode edding ?
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 17, 2013, 04:21:44 AM
Quote from: Luiz Roberto dos Santos on May 17, 2013, 03:35:29 AM

What is your maximum recording time?
Very good, we are progressing here! :D
The colors are mind blowing compared to h264. Haven't tested maximum yet, Ive done a solid 20 secs of frames though.
Title: Re: uncompressed 14-bit RAW video recording
Post by: andersonchagas on May 17, 2013, 04:24:33 AM
Any chance of raw in T2i at some resolution?

Title: Re: uncompressed 14-bit RAW video recording
Post by: paramon on May 17, 2013, 04:28:33 AM
i don't know what's going on,
on the May 14 build i was able to film full 1920x1080 4Gb files, but with some dropped frames, on today's build the largest rez i can do that is 1920 x720, but no dropped frames,
i am on Mkiii Lexar x1000,
is this normal?
Title: Re: uncompressed 14-bit RAW video recording
Post by: tridluu on May 17, 2013, 04:33:39 AM
Quote from: Roars on May 17, 2013, 04:19:49 AM
so i have a couple of thought to throw out hear

what is the Colour Chroma Subsampling when shooting in Raw is it 4:2:2 or 4:4:4 the 5d is 4:2:0 as standerd

could some of the issues with the Sd cards be solved by using a SSd, ie a ninja (cos you can get a better bit rate [10-12mb] out of the hdmi export onto a ninja than you can standerd [8mb]) but i dont think that would work cos you cant export raw data down the hdmi maybe if you could retro fit a ssd direct to the 5d.

and the bit rate is that hardwere limeted to 14 bits or the hack or the card

more over how do these raw files stack up to red as to my knowledge there the only people to use raw (i think arri might to)

why does all the raw test footage look abit darker is that simply a byproduct of more detail

so raw on dslrs has been proven so why has cannon not thought of this

also what are you guy editing on final cut 7 cant realy handle HD let alone raw. i was told only avid could truly edit .raw but you can convert it [woudnt you loos the ability to edit the exposure and stuff if you did that]

are you useing redcode edding ?


These questions have already been answered in earlier posts or similar threads.

Kudos to the ML team on this development. It will help them if people testing would post bugs and issues rather than have people asking the same questions over and over. Just my thought. It doesn't take long to read the full thread and understand what is going on.
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 17, 2013, 04:36:54 AM
Quote from: paramon on May 17, 2013, 04:28:33 AM
i don't know what's going on,
on the May 14 build i was able to film full 1920x1080 4Gb files, but with some dropped frames, on today's build the largest rez i can do that is 1920 x720, but no dropped frames,
i am on Mkiii Lexar x1000,
is this normal?

You need to check your settings for Raw video.  The buffer full option see if it set to stop recording. You can change it to skip frames. Before it would always skip frames. Now there is option, which is normally set to stop recording when buffer is full.
Title: Re: uncompressed 14-bit RAW video recording
Post by: EOSHD on May 17, 2013, 04:47:32 AM
Quote from: platu on May 17, 2013, 01:31:30 AM
A little update regarding the KomputerBay cards... You may want to avoid recommending the 128GB version of the KomputerBay CF card. Evidence is building that the 128 version had slower real world write speeds than the 64. This prevents 1080p raw recording since the 128 doesn't seem to attain the 83MB/s minimum write required for 1080p raw recording. The 64GB version seems to test at 90+ MB/s write while the 128GB version tests at 60-70 MB/s write.  It turns out that Andrew from EOSHD was using the 64gb variety of the KomputerBay card for his Raw tests, which is why he was able to utilize even greater than 1080p raw without dropped frames.  This is all still unproven but I just returned my 128gb and ordered two 64gb to see for myself.

73MB/s on my 128GB card (best case).
95MB/s on my 64GB card peak sustained rate in ML benchmark - 85MB/s reliable during raw recording.

Barely 20MB/s on my fastest SD cards. They are: Sandisk SDHC-I 95MB/s (16GB) and Sony 94MB/s (32GB). They are unusable for raw, at least for me. Had expected better from those.

I am going to try painting go faster stripes on my 128GB card and see if it improves ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 17, 2013, 04:49:38 AM
Quote from: squig on May 17, 2013, 03:02:55 AM
Perfect! I guess I can stop working on the sLUT then. I really like visioncolor/tech, it's a good improvement on prolost.
If you have premiere, why not use the log setting after converting to pro res?
it's in effect>cineon.
or something like that.
Title: Re: uncompressed 14-bit RAW video recording
Post by: jpgentry on May 17, 2013, 04:49:52 AM
Wow, it took 45 pages to answer every question you just asked and more...  Everything you stated shows you didn't really read any of them.

Quote from: Roars on May 17, 2013, 04:19:49 AM
so i have a couple of thought to throw out hear

what is the Colour Chroma Subsampling when shooting in Raw is it 4:2:2 or 4:4:4 the 5d is 4:2:0 as standerd

could some of the issues with the Sd cards be solved by using a SSd, ie a ninja (cos you can get a better bit rate [10-12mb] out of the hdmi export onto a ninja than you can standerd [8mb]) but i dont think that would work cos you cant export raw data down the hdmi maybe if you could retro fit a ssd direct to the 5d.

and the bit rate is that hardwere limeted to 14 bits or the hack or the card

more over how do these raw files stack up to red as to my knowledge there the only people to use raw (i think arri might to)

why does all the raw test footage look abit darker is that simply a byproduct of more detail

so raw on dslrs has been proven so why has cannon not thought of this

also what are you guy editing on final cut 7 cant realy handle HD let alone raw. i was told only avid could truly edit .raw but you can convert it [woudnt you loos the ability to edit the exposure and stuff if you did that]

are you useing redcode edding ?
Title: Re: uncompressed 14-bit RAW video recording
Post by: EOSHD on May 17, 2013, 04:54:13 AM
Is it due to fragmentation?

With latest build I had strange issue where on my faster 64GB card it dropped back to 70MB/s and dropped frames at the start of the recording, then gradually sped up around the 1GB mark during recording, until it was back at 80MB/s+ again.

Rebooted camera, tried 1920x1280. 92MB/s sustained to the card. Under the magic 98MB/s really needed for that resolution but still - why such variation in card writing speeds going on here?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 05:06:09 AM
I wrote a few more files and got some speed back... maybe this is from the cards?
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 17, 2013, 05:09:04 AM
Quote from: aaphotog on May 17, 2013, 04:49:38 AM
If you have premiere, why not use the log setting after converting to pro res?
it's in effect>cineon.
or something like that.

I do have premiere but I'm a long time final cut user and I haven't had time to get my head around premiere yet but thanks for the tip.
Title: Re: uncompressed 14-bit RAW video recording
Post by: EOSHD on May 17, 2013, 05:13:55 AM
Sometimes my 128GB card is slow off the marks - 50MB/s, then slowly rises to reach 70MB/s after around 10 seconds of recording.

In benchmark it always achieves around 73MB/s no matter what mood it is in  ;D
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 17, 2013, 05:23:10 AM
Quote from: EOSHD on May 17, 2013, 05:13:55 AM
Sometimes my 128GB card is slow off the marks - 50MB/s, then slowly rises to reach 70MB/s after around 10 seconds of recording.

In benchmark it always achieves around 73MB/s no matter what mood it is in  ;D

On your 128gb, what is the highest resolution you can use (1x mode, 24fps) and still record the full 4 gig file without skipping frames?  On my KomputerBay 128gb, I can only do 1080x900.  Anything higher will only go for 10-20 seconds before frame skipping begins.
Title: Re: uncompressed 14-bit RAW video recording
Post by: EOSHD on May 17, 2013, 05:27:08 AM
Highest reliable res on the 128GB card is 1920x840, around 65MB/s.

1920x960 isn't too bad... but right on the limit of the card at 73MB/s. Sometimes it can't even do that.

Still the 64GB card pushes on for 92MB/s peak and can do 1080p no sweat.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Digital Corpus on May 17, 2013, 05:33:23 AM
Quote from: Digital Corpus on May 16, 2013, 05:48:49 AM...CF and SD cards are Flash memory. There is a bit of a principle of Flash memory that *will* affect  your write speeds, and though these memory cards are not what we consider SSD's, they have to abide by the same rules (http://"http://www.anandtech.com/show/2738/8")...
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 17, 2013, 05:43:56 AM
Quote from: EOSHD on May 17, 2013, 05:27:08 AM
Highest reliable res on the 128GB card is 1920x840, around 65MB/s.

1920x960 isn't too bad... but right on the limit of the card at 73MB/s. Sometimes it can't even do that.

Still the 64GB card pushes on for 92MB/s peak and can do 1080p no sweat.

Our cards are behaving almost exactly the same in terms of speed.  You should also be able to get 1920x900 WITHOUT frame skipping for the full 4gb file.  The build from a little earlier today omitted that resolution but lourenco sent me a link adding it back...

https://drive.google.com/folderview?id=0B7QlH_BH2m32Q1FMeHBBQVVmb3M&usp=sharing

If you are missing 1080x900, just overwrite these files in the ML/MODULES folder (I'm assuming you are using the latest build).
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 17, 2013, 06:36:23 AM
An attempt to do hdr, but I'm not very good with PP:



It's kinda surreal being able to capture clouds in the sky without underexposing the shadowy areas too much!

Though it doesn't show on Youtube for obvious reasons, most of the things my brain instinctively looks out for doesn't exist anymore! Blocky, muddy or clipped shadows and blotchy textures are now very, very clear and detailed.

edit: The video compressed to BD specs: https://mega.co.nz/#!zQVCQQCS!czx6vEM2zxZqcNH-OE9dQTgbyUBeqG6sFXfulxQlUBo
Title: Re: uncompressed 14-bit RAW video recording
Post by: kgv5 on May 17, 2013, 06:51:47 AM
Yoshiyuki Blade, nice try, did you use twixtor to make frame blending (I assume you had 12 light and 12 dark frames)? Could you please briefly describe your twixtor workflow? I am using it with normal clips with no problem but somehow I cannot blend DNG's.
Title: Re: uncompressed 14-bit RAW video recording
Post by: swinxx on May 17, 2013, 07:16:53 AM
Alex, isnt it possible to split the recorded file in 2 seperate files, and then use cf AND sd card slot at the same time to get better resolution? Like a raid system??

Greets and a big thank you to the whole ml team you for your great stuff
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 17, 2013, 07:27:07 AM
Quote from: kgv5 on May 17, 2013, 06:51:47 AM
Yoshiyuki Blade, nice try, did you use twixtor to make frame blending (I assume you had 12 light and 12 dark frames)? Could you please briefly describe your twixtor workflow? I am using it with normal clips with no problem but somehow I cannot blend DNG's.

No frameblending was used. I just threw together a bunch of nonsensical settings in Raw Therapee until it looked cool and applied the settings to all the frames in a batch. I'm uploading a compressed-but-way-better-than-Youtube clip atm and will edit my post with the link.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mjneubrander on May 17, 2013, 07:28:43 AM
Finished playing around and decided to do something. Here is my expose on leaves, flowers, and slow focus pulls. https://www.youtube.com/watch?v=FYFTaIslaSo

Using a transcend x400 32gb which tested at 35 Mb/s and getting continuous 1280x720 with a may 13 build. All the newer builds seem to be dropping many more frames.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 17, 2013, 08:05:39 AM
I have a problem with the last firmware update.
If you use the record in RAW, and then turn it off
then when I enable basic video(H264) shooting and then turn off, the camera freezes. Displays the last frame and stop everything.
  Turning off the camera does not work, still image is still visible.
  It was only when the camera is off remove AKM and then work fine!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Audionut on May 17, 2013, 08:21:23 AM
Some footage. (https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/ML/ML-RAW.mkv)

1600x840 in camera @ 5x zoom, best I can do with a Sandisk Extreme 60MB/s CF.
Resized to 1920x1008 with borders added to bring it to 1080 high and a touch of sharpening and encoded with x264

--preset placebo --tune grain --crf 12
Close enough to lossless.

A source frame (https://dl.dropboxusercontent.com/u/34113196/Camera%20stuff/ML/movie_51.tif) before resize and sharpening.
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 17, 2013, 09:41:16 AM
Quote from: Andy600 on May 16, 2013, 05:09:43 PM
I'm noticing a couple of little green dots on my DNGs. I thought they might be dead/hot pixels but they are not there on jpegs, CR2 or .mov files

(http://i.imgbox.com/aczXFmLI.jpg)

well, the dead pixel detection isnt executed by us.
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 17, 2013, 09:43:14 AM
Quote from: Peter Linov on May 17, 2013, 08:05:39 AM
I have a problem with the last firmware update.
If you use the record in RAW, and then turn it off
then when I enable basic video(H264) shooting and then turn off, the camera freezes. Displays the last frame and stop everything.
  Turning off the camera does not work, still image is still visible.
  It was only when the camera is off remove AKM and then work fine!

alex found the issue and worked it around.
https://bitbucket.org/hudson/magic-lantern/commits/d6bca45d8f31554fd0ab71b628a6b04a8e56b3e9
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 17, 2013, 10:09:14 AM
Quote from: g3gg0 on May 17, 2013, 09:43:14 AM
alex found the issue and worked it around.
https://bitbucket.org/hudson/magic-lantern/commits/d6bca45d8f31554fd0ab71b628a6b04a8e56b3e9
Please forgive me, what should I do?
Title: Re: uncompressed 14-bit RAW video recording
Post by: dfsgfeiuhd on May 17, 2013, 10:09:30 AM
On the 600d there is no Menu to set the resolution for me, yet people reported that was working before. I am a bit confused. Also, the buffer always fills first before writing the files to card, no co-recording while buffering.

I use the latest nightly build and the latest BeepsTwice.zip.

Is there more files required? I tried two different cards and the feature is not hidden in ML Menu either. Might be a bug.
Title: Re: uncompressed 14-bit RAW video recording
Post by: DjJuvan on May 17, 2013, 11:28:10 AM
Developers... this is maybe just my wild idea, but what about using SD card as backup temporary buffer for writing RAW video on CF cards? It could lower the chances of dropping frames because of too slow writing on CF cards and filling the in-camera buffer to quick. It could be than possible to record 1080p videos with slower CF cards too.

Just a thought?
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 17, 2013, 11:30:37 AM
Quote from: dfsgfeiuhd on May 17, 2013, 10:09:30 AM
On the 600d there is no Menu to set the resolution for me, yet people reported that was working before. I am a bit confused. Also, the buffer always fills first before writing the files to card, no co-recording while buffering.

I use the latest nightly build and the latest BeepsTwice.zip.

Is there more files required? I tried two different cards and the feature is not hidden in ML Menu either. Might be a bug.

Did you put raw_rec module into the modules folder of the sd card? Works great on my 600D.
Title: Re: uncompressed 14-bit RAW video recording
Post by: swinxx on May 17, 2013, 11:31:26 AM
@DjJuvan

check Reply #1117, nearly identical thoughts..

greets
Title: Re: uncompressed 14-bit RAW video recording
Post by: jpgentry on May 17, 2013, 11:35:23 AM
The bigger incentive for this would be pushing past 1080P to 2 or 2.5K resolution...

Quote from: DjJuvan on May 17, 2013, 11:28:10 AM
Developers... this is maybe just my wild idea, but what about using SD card as backup temporary buffer for writing RAW video on CF cards? It could lower the chances of dropping frames because of too slow writing on CF cards and filling the in-camera buffer to quick. It could be than possible to record 1080p videos with slower CF cards too.

Just a thought?
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 17, 2013, 11:43:27 AM
Benchmarks show a speed increase of about 20MB/s. The workflow and coding will be a bit more complex though.

For now, you have a CF+SD benchmark in Debug menu.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Grunf on May 17, 2013, 11:50:20 AM
Quick question (please ignore if I'm off kilter here):

Is it feasible to use CPU to shift away 4 bits and generate 8-bit 47MB/s 1080p24 raw file?
It would tax the CPU but it will lower the write speed to "manageable" 47MB/sec for those who don't need full 14-bit of colour?

CPU-assisted memcopy() should be good for approx 73MB/sec, so maybe it would suffice to copy the buffer to memory, do the quick "compression" by shifting away least significant 4 bits in CPU, and then using EDMAC to dump it to CF-card? (if this ARM-implementation has vector OP's that can do shift ops on wide-registers then it might not get so hot :) )

It would allow 1080p24 4:4:4 8-bit "raw" as many of us sought after for long time and still fit into reasonably quick CF-card bandwith :)


Title: Re: uncompressed 14-bit RAW video recording
Post by: savale on May 17, 2013, 12:07:52 PM
As developer (but not yet involved in the raw_rec recording dev) I see some options to improve the raw recording or add an option for slower cards:

1) if there is enough RAM / Cpu cycles left: I think the best way would be to use lossless compression to do the job. I can imagine this will move the problem to the processor, but there is a chance...

2) add an option to convert to 8bit raw? This will most probably introduces some nasty banding effects, but I am not sure if that is noticeable in real situations? This is quite easy to test and code I guess.
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 17, 2013, 12:11:56 PM
Quote from: savale on May 17, 2013, 12:07:52 PM
As developer (but not yet involved in the raw_rec recording dev) I see some options to improve the raw recording or add an option for slower cards:

1) if there is enough RAM / Cpu cycles left: I think the best way would be to use lossless compression to do the job. I can imagine this will move the problem to the processor, but there is a chance...

2) add an option to convert to 8bit raw? This will most probably introduces some nasty banding effects, but I am not sure if that is noticeable in real situations? This is quite easy to test and code I guess.

both impossible due to CPU being way too slow
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 17, 2013, 12:13:49 PM
Quote from: savale on May 17, 2013, 12:07:52 PM
As developer (but not yet involved in the raw_rec recording dev) I see some options to improve the raw recording or add an option for slower cards:

1) if there is enough RAM / Cpu cycles left: I think the best way would be to use lossless compression to do the job. I can imagine this will move the problem to the processor, but there is a chance...

2) add an option to convert to 8bit raw? This will most probably introduces some nasty banding effects, but I am not sure if that is noticeable in real situations? This is quite easy to test and code I guess.

I can't see why I would want either one of these options for myself. Just buy faster cards.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ajay on May 17, 2013, 12:33:04 PM
Another RAW video, but from a perspective as a bird photographer/videoagrapher. I was quite anxious to see how the 5DM3's RAW output would look using birds as a model. Here's a pair of Eastern Bluebirds preening. I filmed in 1920x840 perspective due to bandwidth limitations when using 5x zoom using Trancend's 1000x 32GB card. I have gotten it to work reliably now at 1920 x 1080 but only at 1x not 5x. 5x drops frames @ 1080p.

Also...I have no idea if I am correctly using the 5x mode. I set zoom to 5x, then record through ML's menu. Is this correct? I found that in 5x mode, what I see on the LCD screen is not what is being recorded. The subject that I want centered is usually off to the right or left of the zoomed display. (That's why in 5x mode in this video, the birds are chopped off frame. This test was really to see how the resolution looked at 5x.)

Anyhow...I am so elated to finally get excellent quality video from the 5DM3. I can't thank you guys enough for doing this. I will make a donation today!

Here's the clip...I did color correction and a bit of sharpening to the DNG's.



Thank you ML team!
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 17, 2013, 12:39:10 PM
I've added live preview in 5x mode, so you can see exactly what you are recording. Just wait for next build.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ajay on May 17, 2013, 12:53:11 PM
Quote from: a1ex on May 17, 2013, 12:39:10 PM
I've added live preview in 5x mode, so you can see exactly what you are recording. Just wait for next build.

Thanks Alex. I don't know if you realize this, but you guys are making A LOT of frustrated Canon users very happy.  :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: savale on May 17, 2013, 12:56:08 PM
Quote from: a1ex on May 17, 2013, 12:39:10 PM
I've added live preview in 5x mode, so you can see exactly what you are recording. Just wait for next build.

Thanks! I suggested probably impossible raw_rec features: (8bit raw or lossless compression). But do you see any other improvement possible for the 5d2 users (because the compact flash card interface seems to be maxed out...)
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 17, 2013, 01:01:48 PM
Quote from: savale on May 17, 2013, 12:56:08 PM
Thanks! I suggested probably impossible raw_rec features: (8bit raw or lossless compression). But do you see any other improvement possible for the 5d2 users (because the compact flash card interface seems to be maxed out...)

answered here:

Quote from: g3gg0 on May 17, 2013, 12:11:56 PM
both impossible due to CPU being way too slow

and we had inspected 14->12 bit reduction hardware, but its too complex yet.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Mr. Propaganda Guy on May 17, 2013, 01:12:09 PM
If you guys get this running on the 7D, "Hope you do" you think it'll be possible to use the 2nd processor to add compression to the RAW? If so, what compression would be possible, Mjpeg, ProRes, DNxHD?
Title: Re: uncompressed 14-bit RAW video recording
Post by: tihon on May 17, 2013, 01:13:26 PM
Do you mean: recording with 12bit dng raw?

Gonna be really cool! 14 bit is too much as  i think and make the file size to big to rec
Title: Re: uncompressed 14-bit RAW video recording
Post by: mnememedia on May 17, 2013, 01:16:36 PM
Hi all,

First, many thanks to the team and all the work they've done so far. Keep it up! The quality afforded by RAW capture is simply stunning. Let me know what you think of my quick sunset shoot I did this evening. As always, be sure to download the original to see the full detail.

https://vimeo.com/66389956

Title: Re: uncompressed 14-bit RAW video recording
Post by: Mr. Propaganda Guy on May 17, 2013, 01:22:42 PM
What would be the file size of 1 frame at 1920x1080 12bit RAW?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Brian@202020 on May 17, 2013, 01:26:34 PM
Any chance any of the SD card based Canon DSLR's internal SD hardware can harness the fast data rates of the new Panasonic MicroP2 SD cards? At 2Gb/s they are currently the only UHS-II cards I know of.

http://www.videomaker.com/videonews/2013/04/nab-2013-panasonic-microp2-resizes-p2-to-sd
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 17, 2013, 01:28:29 PM
An easy to follow mac-tutorial focusing on fast conversion to AppleProRes422HQ and a bunch of other stuff like creating xmp-templates, setting upp after effects, premiere pro.

http://www.youtube.com/watch?v=MSHVi7Jx9r0&feature=youtu.be

https://vimeo.com/66390822

Thanks!
//D
Title: Re: uncompressed 14-bit RAW video recording
Post by: tihon on May 17, 2013, 01:29:43 PM
Quote from: Mr. Propaganda Guy on May 17, 2013, 01:22:42 PM
What would be the file size of 1 frame at 1920x1080 12bit RAW?

Approximate RAW file
size on a Nikon D7000   12 bit   14 bit
compressed   12.6 MB   15.7 MB
uncompressed   14.9 MB   18.8 MB

sooo. we can see the difference in %. The filesize is 20% less in 12 bit then 14 bit
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 17, 2013, 01:29:55 PM
Quote from: Mr. Propaganda Guy on May 17, 2013, 01:12:09 PM
If you guys get this running on the 7D, "Hope you do" you think it'll be possible to use the 2nd processor to add compression to the RAW? If so, what compression would be possible, Mjpeg, ProRes, DNxHD?
no, none
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 17, 2013, 01:30:48 PM
Quote from: tihon on May 17, 2013, 01:13:26 PM
Do you mean: recording with 12bit dng raw?

Gonna be really cool! 14 bit is too much as  i think and make the file size to big to rec

as said, its too complex yet. maybe in 1-2 years
Title: Re: uncompressed 14-bit RAW video recording
Post by: Grunf on May 17, 2013, 01:34:59 PM
OK, if using CPU is out of the question then only option is to try to offload the data to CF interface as quick as possible.

As far as I understand, EDMAC has >600MB/sec throughput so as long as CF can swallow it, there is nothing stopping you to record higher resolutions and/or framerates.

Theoretically there is nothing preventing someone from using for example , Ti TMS320VC54x DSP as "faux CF-card" and letting it shuffle data further to SSD. :D

Current CF UDMA7 raw-interface speed is 167MB/sec so that would suffice for 1080p50 raw :D
Title: Re: uncompressed 14-bit RAW video recording
Post by: mcnys on May 17, 2013, 01:39:38 PM
ok this will sound amazingly stupid...but still...after reading this:

"With a little guesswork, we were lucky to notice that on the inside, the little EOS-M is very similar to the 5D Mark III. So, with a few tricks we were able to dump the firmware and print the familiar "Hello World" message – the proof that Magic Lantern will work on the EOS-M."

so i'm thinking, what kind of sd card controller does eos-m have? maybe its faster than 600-650d? if yes then maybe its the ultimate cheap raw video camera?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yomommassis on May 17, 2013, 01:43:03 PM
I'm really surprised that the 5D mark 2 can handle any of this considering video was added in a firmware update and people have said the cpu was too slow to handle 60 FPS...yet here it is recording 24fps RAW video

Damn.

If the 5D mark 2 can handle this I have no doubt in my mind we will see the whole canon dslr line up shooting raw in the future
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 17, 2013, 01:57:59 PM
@mcnys - I think the EOS-M has a UHS-I slot but a lot also depends on the buffer size. UHS-I can write faster than SD but if the buffer isn't up to it there won't be much, if any, benefit over SD
Title: Re: uncompressed 14-bit RAW video recording
Post by: AnotherDave on May 17, 2013, 01:59:14 PM
Quote from: Grunf on May 17, 2013, 01:34:59 PM
OK, if using CPU is out of the question then only option is to try to offload the data to CF interface as quick as possible.

As far as I understand, EDMAC has >600MB/sec throughput so as long as CF can swallow it, there is nothing stopping you to record higher resolutions and/or framerates.

Theoretically there is nothing preventing someone from using for example , Ti TMS320VC54x DSP as "faux CF-card" and letting it shuffle data further to SSD. :D

Current CF UDMA7 raw-interface speed is 167MB/sec so that would suffice for 1080p50 raw :D

I think it is entirely possible to send the data that is offloading to the CF to a higher speed SSD via some sort of CF extender cable to SSD converter.  Fortunately, CF extender cables do exist... but currently there is no SSD to CF converter.

It has to be possible.  If one of you knows how to do it, go for it!  You'll be a millionaire thousandaire overnight!

This would solve all throughput problems and storage issues recording RAW from a DSLR if it worked, wouldn't it?
Title: Re: uncompressed 14-bit RAW video recording
Post by: t2scorp on May 17, 2013, 02:13:17 PM
Quote from: Yomommassis on May 17, 2013, 01:43:03 PM
I'm really surprised that the 5D mark 2 can handle any of this considering video was added in a firmware update and people have said the cpu was too slow to handle 60 FPS...yet here it is recording 24fps RAW video

Damn.

If the 5D mark 2 can handle this I have no doubt in my mind we will see the whole canon dslr line up shooting raw in the future

The slow CPU is not involved. The RAW output is just saving the files to an other storage...
Title: Re: uncompressed 14-bit RAW video recording
Post by: Alia5 on May 17, 2013, 02:17:38 PM
On my 600D i got zero magenta frames in videomode (720p) with 1120x400 crop (new version... HDR thingy)
with the linescipping factor of 1.66x that almost gives me a nice 16:9 ratio when using pretty much of the sensor

works like charm!
1% did an excellent job in porting!

excpect some testfootage!
Title: Re: uncompressed 14-bit RAW video recording
Post by: cinema5D on May 17, 2013, 02:21:19 PM
The 25p version of the RAW module works flawlessy with our 128GB (!) Kompterbay cards (http://www.amazon.com/KOMPUTERBAY-128GB-Professional-COMPACT-Extreme/dp/B009JS61UM/?_encoding=UTF8&camp=1789&creative=9325&linkCode=ur2&tag=cinema5d-20) in 1080p! ARTICLE LINK (http://c5d.at/1oc).

We just recorded a 4,23GB clip without any frame drop. The compile we used came from this post (http://www.magiclantern.fm/forum/index.php?topic=5247.msg36052#msg36052) earlier in this thread. If only we could convert the large RAW files to dng's with that Mac app...  :-*

We also updated our step-by-step guide (http://www.cinema5d.com/news/?p=17898). Looking forward to more amazingness coming from this development here. Thanks to everyone involved.
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 17, 2013, 02:22:27 PM
Here's some high ISO (2800) raw footage from 600D with the latest build.



1280x400 in 640x480 mode. Works like a charm now, no pink or skipped frames, only those weird blue/red/yellow pixelated areas at high ISO.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 17, 2013, 02:35:20 PM
Thanks to the ML crew and  lourenco for the new build. I'm having a few issues but I gotta suss whether it's the camera or something in post before I make a report.

Title: Re: uncompressed 14-bit RAW video recording
Post by: Grunf on May 17, 2013, 02:38:40 PM
Quote from: AnotherDave on May 17, 2013, 01:59:14 PM
I think it is entirely possible to send the data that is offloading to the CF to a higher speed SSD via some sort of CF extender cable to SSD converter.  Fortunately, CF extender cables do exist... but currently there is no SSD to CF converter.

It has to be possible.  If one of you knows how to do it, go for it!  You'll be a millionaire thousandaire overnight!

This would solve all throughput problems and storage issues recording RAW from a DSLR if it worked, wouldn't it?

Hmm... I don't think "cable" will suffice as there will me buffering involved in-between.

Basically, we are looking for DSP (or even better, SOC) with both SATA and CF UDMA-7 interfaces. But CF interface needs to be "inverted". Thus, such apparatus needs to emulate CF and send buffered data to SATA-interface.

there are lot's of SOC's providing interface *to* CF, but I wonder if there are any that can be reversed. I guess we have a little hardware project on our hands, no? ;)

Side 8 describes CF interface specs:

http://www.ti.com/lit/an/spra803/spra803.pdf

As far as I understand, we are talking 16-bit wide I/O bus.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Grunf on May 17, 2013, 02:50:46 PM
Hmm...thinking further, CF interface is noting else but old-style 16-bit wide "IDE" harddrive interface. So what we actually need is IDE (with UDMA-7 support) to SATA interface card.

support for SATA rev 1. would be enough as it supports up to 150MB/sec and UDMA-7 tops out at 157MB/sec.

Title: Re: uncompressed 14-bit RAW video recording
Post by: nanomad on May 17, 2013, 02:52:24 PM
You need an IDE <-> SATA converter that's actually capable of using UDMA7
Title: Re: uncompressed 14-bit RAW video recording
Post by: mcnys on May 17, 2013, 02:54:49 PM
Quote from: Andy600 on May 17, 2013, 01:57:59 PM
@mcnys - I think the EOS-M has a UHS-I slot but a lot also depends on the buffer size. UHS-I can write faster than SD but if the buffer isn't up to it there won't be much, if any, benefit over SD

lol, so basically if canon raised manufacturing costs by few dollars more, and made buffer size at least twice the size it would have been, amazing little raw camera...oh well too bad  :/
Title: Re: uncompressed 14-bit RAW video recording
Post by: Grunf on May 17, 2013, 03:00:17 PM
OK guys, I think this can be done!

What we need is Bi-Directional converter for serial and parallel ATA that supports UDMA-7 on PATA side. SATA side isn't important as even the lowest speed (rev.1 ) is 150MB/s which is almost as good as UDMA-7 157MB/s.

I found one here for 38 dollars:

http://www.cooldrives.com/saidecomisat1.html

Quick check of specs:
Serial ATA 1.0 Specification compliant
Automatic Serial ATA 1.5 Gbps speed negotiation
ATA / ATAPI PIO mode 0 to 4
ATA / ATAPI Ultra DMA of transfer rate 16.7, 25, 33, 48, 66, 100, 133, and 150MB/s.

It supports UDMA-7 on PATA-side!!!

Theoretically, only thing needed to make 5D III record to SATA drive would be female "faux CF"-to-PATA passive adapter. I don't know if such exist but I could make one by disassembling old CF card and soldering new lines. :D


Should we fork this into separate project in order not to fill up this thread?
Mods?
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 17, 2013, 03:01:00 PM
Quote from: Grunf on May 17, 2013, 03:00:17 PM
Should we fork this into separate project in order not to fill up this thread?

good idea, just create a thread about that ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: peter_f on May 17, 2013, 03:09:22 PM
Just an update regarding the Komputerbay 128GB speed: I can record 1920x1152 w/o skipping frames at 88.5 MB/s. I did turn global draw and sound recording off though.
Title: Re: uncompressed 14-bit RAW video recording
Post by: AnotherDave on May 17, 2013, 03:14:03 PM
There are CF extenders that were designed to move CF ports to different places that might be of help!

Google - CF extender.
Title: Re: uncompressed 14-bit RAW video recording
Post by: JohnBarlow on May 17, 2013, 03:55:17 PM
To the devs  --->

Is it possible to bit shift or similar bit trix the raw stream to get to 12 bit instead of 14?

I am thinking this could be good for 550d/600d shooters with card controller limitations, giving an effective 4x speed increase
Title: Re: uncompressed 14-bit RAW video recording
Post by: KMA_WWC on May 17, 2013, 04:01:34 PM
Guys PLEASE STOP asking the same questions here. You are filling the thread with the same questions that were asked before!!! Please read the thread and you'll find the answers to all your questions. I understand its 47 pages, but some of the question been answered a page before, so at least check one page in advance or make a new thread!!! Thank you!
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 17, 2013, 04:03:34 PM
Quote from: JohnBarlow on May 17, 2013, 03:55:17 PM
To the devs  --->

Is it possible to bit shift or similar bit trix the raw stream to get to 12 bit instead of 14?

I am thinking this could be good for 550d/600d shooters with card controller limitations, giving an effective 4x speed increase

If you just read posts in a previous page...
Title: Re: uncompressed 14-bit RAW video recording
Post by: JohnBarlow on May 17, 2013, 04:09:43 PM
Ok my bad, must have missed it
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 04:27:29 PM
EOS -M memory is too low.. only 50MB total. It will work at some point and give you some bursts. Right now the silent pic shooting is close enough.
Title: Re: uncompressed 14-bit RAW video recording
Post by: AnotherDave on May 17, 2013, 04:32:38 PM
I read yesterday that figuring out how to record past the 4gb file size limit "should be possible" but other things are a priority at the moment - like assigning a start/stop button for raw recording.

So.... (to the developers) which button were you thinking?  I'd like it to be the shutter release!  :-)  Can we vote?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 17, 2013, 04:48:40 PM
Quote from: AnotherDave on May 17, 2013, 04:32:38 PM
I read yesterday that figuring out how to record past the 4gb file size limit "should be possible" but other things are a priority at the moment - like assigning a start/stop button for raw recording.

So.... (to the developers) which button were you thinking?  I'd like it to be the shutter release!  :-)  Can we vote?

Actually, the latest build has a start/stop on the rec button. on my mark 3 that is
Title: Re: uncompressed 14-bit RAW video recording
Post by: savale on May 17, 2013, 05:20:15 PM
I looked into the code and do see some optimization (for cpu speed) possible, but it will cost a little mem. It might make room for possible new features but I don't know yet. Maybe this weekend I will look into how to setup a build environment so I can start developing. Is there a guide available how to setup a build env? Thnx
Title: Re: uncompressed 14-bit RAW video recording
Post by: Athiril on May 17, 2013, 05:53:59 PM
Quote from: AnotherDave on May 17, 2013, 01:59:14 PM
I think it is entirely possible to send the data that is offloading to the CF to a higher speed SSD via some sort of CF extender cable to SSD converter.  Fortunately, CF extender cables do exist... but currently there is no SSD to CF converter.

It has to be possible.  If one of you knows how to do it, go for it!  You'll be a millionaire thousandaire overnight!

This would solve all throughput problems and storage issues recording RAW from a DSLR if it worked, wouldn't it?

Not sure what the bus speed between buffer and CF card is in the 5D3. But the adapters are typically slow.

But there ARE bidirectional IDE<->SATA adapters that work in both directions, either IDE drive to SATA or SATA to IDE drive over the same adapter.

Hijack the IDE side into CF. Though not sure what the speed is of those devices.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 05:57:43 PM
4GB limit isn't really a limit.. but the converters won't decode the large files.
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 17, 2013, 05:58:52 PM
Huh? works for me for 5GB files (didn't try larger)
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 06:03:50 PM
On linux binary or windows binary?

Title: Re: uncompressed 14-bit RAW video recording
Post by: AnotherDave on May 17, 2013, 06:06:03 PM
And it isn't possible to get the camera to process each frame as an individual .RAW file?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 06:08:09 PM
Its slower. Wish it was fast enough for some sort of dumb compression like zip, etc.
Title: Re: uncompressed 14-bit RAW video recording
Post by: AnotherDave on May 17, 2013, 06:17:38 PM
I find that the average shot (on a non-hacked camera for work), for me, rolls for around 2-3 mins for each take.

That would be a 15-16GB .RAW file, wouldn't it? 

What is the issue with the converters not being able to convert past 4GB?  Could you break a larger file up into 4GB segments before converting it?

How likely do you think it would be that we will be able to roll for 2-3 mins?

Title: Re: uncompressed 14-bit RAW video recording
Post by: Nicolai1227 on May 17, 2013, 06:24:39 PM
Just tryed out the RAW function, (5D3 with a SanDisk 16GB 60MB/s) ML v.2.3.NEXT.2013May17.5D3113

It all seemed pretty good, I was trying out a few different resolutions not higher than 1280*720 when an err came up.

Now the Card doesnt show up on the Computer, or works in the Camera. Formating in the Kamera wont work, it freezes.

A dialog came up saying, writting bug report. But where can I find it so you Guys know what happened?

I took another CF Card (same modell) and recorded again, worked all fine. Also the footage is great.

Great Work love it :D
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 17, 2013, 06:32:47 PM
Quote from: 1% on May 17, 2013, 05:57:43 PM
4GB limit isn't really a limit.. but the converters won't decode the large files.

What about just trying to create a new raw movie file  at the 4GB limit?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 06:49:21 PM
I guess that would work.. but new files will slow things down. It can already record till the end of the card. I need to see why its not opening.
Title: Re: uncompressed 14-bit RAW video recording
Post by: MD87 on May 17, 2013, 07:06:47 PM
Greetings, gents. I have a problem in the 600D is at stake with a nightly build unpack [600D] Raw HDR.zip. However, when you start the camera module writes that "MAGIC.SYM notes exist". but the file is in the folder "modules". And not load rav_rets. What am I doing wrong? I'm sorry if this question has been, my English is very bad.
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 17, 2013, 07:07:19 PM
QuoteOn linux binary or windows binary?

Both of them; tried the windows one under wine.
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 17, 2013, 07:10:46 PM
Quote from: MD87 on May 17, 2013, 07:06:47 PM
Greetings, gents. I have a problem in the 600D is at stake with a nightly build unpack [600D] Raw HDR.zip. However, when you start the camera module writes that "MAGIC.SYM notes exist". but the file is in the folder "modules". And not load rav_rets. What am I doing wrong? I'm sorry if this question has been, my English is very bad.
Place the MODULES folder into the ML folder
Title: Re: uncompressed 14-bit RAW video recording
Post by: Felixlgr on May 17, 2013, 07:14:44 PM
Quote from: 1% on May 17, 2013, 06:49:21 PM
I guess that would work.. but new files will slow things down. It can already record till the end of the card. I need to see why its not opening.

whats the trick to let it fill up the card and not stop after 4gig ?

TIA and keep up the good work !!
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 07:20:10 PM
QuoteBoth of them; tried the windows one under wine.

So something wrong with the header... I set it to reset after 4gb but I thought that this stuff isn't written to it. Just res, # frames, frame size
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 17, 2013, 07:24:42 PM
Quote from: Felixlgr on May 17, 2013, 07:14:44 PM
whats the trick to let it fill up the card and not stop after 4gig ?

TIA and keep up the good work !!

I noticed he change the raw_rec.c code from being  if (written > 0xFFFFFFFFu - ((uint32_t)buffer_count + 1) * 32*1024*1024)
to if (0) .   This way it does not stop until the memory card is full or until the user decides to stop.   He might had changed more code since I last look at it a few days ago.
Title: Re: uncompressed 14-bit RAW video recording
Post by: MD87 on May 17, 2013, 07:29:55 PM
Quote from: N/A on May 17, 2013, 07:10:46 PM
Place the MODULES folder into the ML folder
Tuplyu...  Thank you!
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 07:41:40 PM
I can try to cast the int64 to int and do it that way. We'll see if the files open then.
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 17, 2013, 07:43:45 PM
I've also tried to remove the 4GB check, but I've got files without footer (which can't be converted). So it stays there until we have a better solution.
Title: Re: uncompressed 14-bit RAW video recording
Post by: iaremrsir on May 17, 2013, 07:47:47 PM
I've been reading through all of the pages in this thread and I can't remember seeing anything about this. On the newest 600d build (edmac popin) from 1% it seems as if the ISO is being baked into the dng files. Adjusting exposure in ACR didn't help at all.
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 17, 2013, 07:50:59 PM
Quote from: 1% on May 17, 2013, 07:41:40 PM
I can try to cast the int64 to int and do it that way. We'll see if the files open then.
Don't mean to change subjects, but the newest 600D build has a bug, 5x zoom in photo and movie mode works fine with either raw_rec disabled, or enabled with global draw off. With GD and raw_rec on, even with just a histogram, 5x zoom gets distorted and B&W.
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 17, 2013, 07:56:54 PM
It's a feature...
Title: Re: uncompressed 14-bit RAW video recording
Post by: Felixlgr on May 17, 2013, 07:58:23 PM
Quote from: lourenco on May 17, 2013, 07:24:42 PM
I noticed he change the raw_rec.c code from being  if (written > 0xFFFFFFFFu - ((uint32_t)buffer_count + 1) * 32*1024*1024)
to if (0) .   This way it does not stop until the memory card is full or until the user decides to stop.   He might had changed more code since I last look at it a few days ago.

so basically latest build (may17th) can record larger than 4GB files ? but we cant convert them yet right ?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 08:00:20 PM
Yep, that is the preview.

I got the indicator to work with casting.. still only shows size up to 4gb but now written is int64.. I'll see if the file opens..

If it works may be worth taking both halves and then displaying total
Title: Re: uncompressed 14-bit RAW video recording
Post by: Colemar on May 17, 2013, 08:05:56 PM
for those who are interested, I posted a blog yesterday with basics on what ML is and currently does, and a straightforward workflow for processing .raw files: http://shop.texasmediasystems.com/and-Canon-will-now-sell-a-grip-of-5DmkIIIs_b_5.html

for this forum's members, a lot will be redundant knowledge, but still worth a look.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 08:15:08 PM
Still says not an raw rec file.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kuky on May 17, 2013, 08:15:45 PM



Camera: 5D Mark III
Build: Magic Lantern May 17 (f686b8c)
Card: SanDisk Extreme Pro 90Mb/s 16Gb CF
Resolution: 1920x1080 23,976 fps
No droped frames, but there are some glitches in the recording. Actually this CF it's very reliable on recording 1920 x 1080. Never had dropped frames.
Minor CC done.
Download to see in real FullHD.
Title: Re: uncompressed 14-bit RAW video recording
Post by: AnotherDave on May 17, 2013, 08:20:33 PM
Camera tests = more cat and dog videos online...  :-\ 

Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 17, 2013, 08:27:04 PM
I tried to copy RAW videos using usb cable 5D3>pc but my pc does not see them?  I use win 7 64 bit.  I want to prevent some wear on door and CF card due to extensive  testing.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kuky on May 17, 2013, 08:36:02 PM
Quote from: AnotherDave on May 17, 2013, 08:20:33 PM
Camera tests = more cat and dog videos online...  :-\

I am lazy. Next time I will do the flowers ???
Title: Re: uncompressed 14-bit RAW video recording
Post by: tridluu on May 17, 2013, 08:40:28 PM
We have a lot of test videos on plants, birds, dogs, inanimate objects. There should be videos on quick movements, actual real life shooting. Do a bunch of fast pans and tilts.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 17, 2013, 08:43:31 PM
Quote from: RenatoPhoto on May 17, 2013, 08:27:04 PM
I tried to copy RAW videos using usb cable 5D3>pc but my pc does not see them?  I use win 7 64 bit.  I want to prevent some wear on door and CF card due to extensive  testing.

I'm having a tech support PTSD relapse.  :'(   Yes madam you have to plug the printer into the power socket.

Camera movement is not covered by the ML warranty. Powering on the camera voids the warranty.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Colemar on May 17, 2013, 08:54:57 PM
Quote from: tridluu on May 17, 2013, 08:40:28 PM
There should be videos on quick movements, actual real life shooting.

https://vimeo.com/66296381
https://vimeo.com/66299918
https://vimeo.com/66304472
https://vimeo.com/66310082
https://vimeo.com/66296375
https://vimeo.com/66294984
Title: Re: uncompressed 14-bit RAW video recording
Post by: Picture and Color on May 17, 2013, 09:09:57 PM
Still waiting for my Komputerbay 1000x cards to arrive, but couldn't resist testing on my Kingston 266x 32GB CFs. These only get just under 30MB/sec, leaving me with a 1280 x 540 crop. Even at this low res, the image is very good and extremely easy to work with. Loving the photo-esque flexibility in post, it is really something.

Cinema5D's photoshop/ quicktime workflow is the one to use. It's blazing fast and if using Premiere, you can even skip the quicktime to prores step. Just import the TIFF files as an image sequence, interpret as 23.98 and away you go.
This one here: http://vimeo.com/66355682 (http://vimeo.com/66355682)

Another crappy plant video below to add to the mix. I'll shoot an actual project with it this weekend, once I get my hands on the Komputerbay 1000x cards. Hopefully it won't be as slow as reported by others here.

Big thanks to Lourenco for continuing to compile latest builds for us. This is SUCH a huge help and we really appreciate it.

I'm using the build he provided in post 1075 here:
http://www.magiclantern.fm/forum/index.php?topic=5247.msg36099#msg36099 (http://www.magiclantern.fm/forum/index.php?topic=5247.msg36099#msg36099)

Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 17, 2013, 09:51:52 PM
Quote from: squig on May 17, 2013, 08:43:31 PM
I'm having a tech support PTSD relapse.  :'(   Yes madam you have to plug the printer into the power socket.

Camera movement is not covered by the ML warranty. Powering on the camera voids the warranty.

What is you point?  Just sarcasm?

The issue is that is is not possible to transfer video.RAW files using a USB cable.   Just wondering if anybody has experience the same problem.
Title: Re: uncompressed 14-bit RAW video recording
Post by: tridluu on May 17, 2013, 10:03:46 PM
Quote from: Colemar on May 17, 2013, 08:54:57 PM
https://vimeo.com/66296381
https://vimeo.com/66299918
https://vimeo.com/66304472
https://vimeo.com/66310082
https://vimeo.com/66296375
https://vimeo.com/66294984

Thank you! Definitely did not see these. Now I just need to order like 5 of these 64gb Komputerbay CF cards to help ML in testing.
Title: Re: uncompressed 14-bit RAW video recording
Post by: AnotherDave on May 17, 2013, 10:04:30 PM
So with the KomputerBay 64GB 1000X cards, you still have to turn the audio off?  Aren't the benchmarks showing they're running at about 95MB/sec?  Recording 1920X1080 produces frames @ 3.5MB, right?  Around 84MB/sec?  How much space is taken up by the audio?

Would it be possible to leave 1 channel of audio for reference... or is that not possible?  Can the audio be recorded to the SD?  *read some mumbles about that...
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 10:14:58 PM
5d3 can record audio to SD card in stereo.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 17, 2013, 10:18:02 PM
Quote from: Picture and Color on May 17, 2013, 09:09:57 PM
Cinema5D's photoshop/ quicktime workflow is the one to use. It's blazing fast and if using Premiere, you can even skip the quicktime to prores step. Just import the TIFF files as an image sequence, interpret as 23.98 and away you go.
This one here: http://vimeo.com/66355682 (http://vimeo.com/66355682)

If you have a copy of after effects you could skip the whole process of exporting to tiff and export directly from dng to ProRes422HQ. Saves a ton of time.
I chipped in with a workflow tutorial today
http://www.youtube.com/watch?v=MSHVi7Jx9r0&feature=youtu.be
vimeo.com/66390822
//D
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 17, 2013, 10:37:45 PM
Quote from: Kuky on May 17, 2013, 08:15:45 PM
[
Camera: 5D Mark III
Build: Magic Lantern May 17 (f686b8c)


This is the latest build?
And you can download it somewhere?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 17, 2013, 10:57:19 PM
Quote from: 1% on May 17, 2013, 10:14:58 PM
5d3 can record audio to SD card in stereo.

I tried that 2 firmware builds ago by following Lourencos tip. First I went in the canon camera menu and chose record to SD-card, than went into the magic lantern menu and started recording audio. Then back into the canon menu and chose save to CF - card, leaving the audio running on the SD card. Then started recording video to the CF - card. Didn,t fully get it to work, some strange hangups and sometimes it worked. Is there another way?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 17, 2013, 11:03:09 PM
Should be automatic in new builds. Sync beep -> seperate wav
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 17, 2013, 11:25:56 PM
Quote from: Kuky on May 17, 2013, 08:36:02 PM
I am lazy. Next time I will do the flowers ???

Its OK I have the flowers covered ;D
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 17, 2013, 11:38:20 PM
Quote from: 1% on May 17, 2013, 11:03:09 PM
Should be automatic in new builds. Sync beep -> seperate wav

Yes. It is automatic in the new builds. Make sure your main card is CF. It will automatically record to the audio to the SD card. You just need to change the synch beep to seperate wav in the raw record menu.  The build I posted last night supports this.

found posted in this post http://www.magiclantern.fm/forum/index.php?topic=5247.msg36099#msg36099
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 17, 2013, 11:50:49 PM
Told you I had the flowers covered  ;D ...

http://www.youtube.com/watch?v=oXle_MnR8OU (http://www.youtube.com/watch?v=oXle_MnR8OU)
Title: Re: uncompressed 14-bit RAW video recording
Post by: dfsgfeiuhd on May 18, 2013, 12:54:03 AM
Quote from: N/A on May 17, 2013, 11:30:37 AM
Did you put raw_rec module into the modules folder of the sd card? Works great on my 600D.

sure, and both lv_rec and raw_rec are loaded in the modules menu. However, I cannot set resolutions anywhere on my 600d. I updated to [600D] NewChannel.zip, still nothing. Weird...
Title: Re: uncompressed 14-bit RAW video recording
Post by: budafilms on May 18, 2013, 12:54:59 AM
New cards in the market 1050X
http://www.bhphotovideo.com/c/product/964008-REG/delkin_devices_ddcf1050_128gb_128gb_cf_1050x_udma.html
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kuky on May 18, 2013, 12:55:26 AM
Quote from: Peter Linov on May 17, 2013, 10:37:45 PM
This is the latest build?
And you can download it somewhere?


http://www.magiclantern.fm/forum/index.php?topic=5413.0 (http://www.magiclantern.fm/forum/index.php?topic=5413.0)
Title: 600D 960x500 raw with no tearing or magenta blocks
Post by: mattmvpmedia on May 18, 2013, 01:19:12 AM


This is with a Sandisk 95MB/s card and the 600D Edmac Popin build. Absolutely no tearing or magenta blocks in the footage that I can find - great work ML Team and thank you.

Also, this video isn't much to look at - didn't do any color grading or anything - it just verifies that the magenta blocks and tearing are gone.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 18, 2013, 01:27:24 AM
I tried edmac 0x13 on 6D like 650D was... that edmac is used... but I managed to kill the LV without the camera crashing.. so maybe we can just redirect edmacs to kill them? No need to do things at boot?
Title: Re: uncompressed 14-bit RAW video recording
Post by: CFP on May 18, 2013, 01:31:09 AM
Is it just me or does the "New Channel" build not work correctly? I only tried it briefly but every single frame was only purple noise ...
I used 3 different resolutions, but all files turned out to be unusable ...

Is it my fault or is something wrong with the build?
Title: Re: uncompressed 14-bit RAW video recording
Post by: mattmvpmedia on May 18, 2013, 01:41:40 AM
I haven't tried the 600D "NewChannel" build yet - the Edmac Popin one has been working great for me.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 18, 2013, 01:53:26 AM
Installed the 'NewChannel' build and I'm just getting frames of noise at all resolutions. Reverted to the previous build. Works nicely :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 18, 2013, 01:54:30 AM
So new edmac not working?
Title: Re: uncompressed 14-bit RAW video recording
Post by: noix222 on May 18, 2013, 01:55:16 AM
Just a quick H264 vs RAW comparison with my 5d mark ii... it looks good to me! Color correcting is so much easier... Can't do H264 anymore  :o



My card is a Delkin 600x so i could do something like 200 frames before it start to skip frames with 1880x720... with a stable release and a faster card im sure it will fly ! 

Thanks for all the AMAZING work you guys are doing...
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 18, 2013, 01:55:39 AM
It records raw files but they are just noise. No image.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Alia5 on May 18, 2013, 02:00:16 AM
NewChannel (600D) also doesnt work for me...
Edmac_popping is great though!
Title: Re: uncompressed 14-bit RAW video recording
Post by: nanomad on May 18, 2013, 02:00:27 AM
That means that the edmac is already in use by canon stuff
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 18, 2013, 02:11:01 AM
Yup, all files full of data but I dunno what it is. Strange nothing crashed and all seemed normal
Title: Re: uncompressed 14-bit RAW video recording
Post by: Alia5 on May 18, 2013, 02:24:47 AM
Did record some test footage with a lighter with the edmac_poppin...



you can read everything in the youtube description...

got 2 pink frames... but camera also skipped 2...

with the raw_hdr versions i think i never got any magenta frames

Edit: flickering comes from 2 crt monitores!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 18, 2013, 02:31:34 AM
@1% - Just noticed something strange. With the same settings (1120x400) and shooting exactly the same scene, I'm getting 109 frames on one shot then 145 on the next, then it goes back to 109, then 145, 109, 145 etc.. a predictable pattern (Edmac Popin).

Other resolutions I've tried so far...

1120x434 it's either 101 or 134 frames
960x540 either 94 or 125 frames
Title: Re: uncompressed 14-bit RAW video recording
Post by: Alia5 on May 18, 2013, 02:38:19 AM
Quote from: Andy600 on May 18, 2013, 02:31:34 AM
@1% - Just noticed something strange. With the same settings (1120x400) I'm getting 109 frames on one shot then 145 on the next, then it goes back to 109, then 145, 109, 145 etc.. a predictable pattern (Edmac Popin). I'll try other resolutions

I think it skipped 109, and 145, for me.. but not sure
2 magenta frames happend afterwards
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 18, 2013, 02:45:06 AM
I've got frame skipping off. It's set to stop recording.

Time for some zzzz's now but I'll check more tomorrow

Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 18, 2013, 03:24:00 AM
5D3 update files. Replace existing files. 
https://drive.google.com/folderview?id=0B7QlH_BH2m32QjRhRDhUeVBwZmM&usp=sharing
includes cf_acc: module as noted on http://www.magiclantern.fm/forum/index.php?topic=5471.msg36588#msg36588
Title: Re: uncompressed 14-bit RAW video recording
Post by: Colemar on May 18, 2013, 03:34:26 AM
lourenco,
  I get an error message when trying to load the modules:

[i] proprs    at:  0x00000000
[i] cbr       at:  0x00A97628
------------------------------------------
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 18, 2013, 03:47:12 AM
Quote from: lourenco on May 18, 2013, 03:24:00 AM
5D3 update files. Replace existing files. 
https://drive.google.com/folderview?id=0B7QlH_BH2m32QjRhRDhUeVBwZmM&usp=sharing
includes cf_acc: module as noted on http://www.magiclantern.fm/forum/index.php?topic=5471.msg36588#msg36588

is this a build with the issues that everyone is mentioning just a few posts above yours?
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 18, 2013, 03:53:31 AM
Quote from: aaphotog on May 18, 2013, 03:47:12 AM
is this a build with the issues that everyone is mentioning just a few posts above yours?
I just posted that new build. He was having issues with older one for some reason. It has worked fine for others. I just sent him a message to try to help him.
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 18, 2013, 04:01:54 AM
Quote from: lourenco on May 18, 2013, 03:53:31 AM
I just posted that new build. He was having issues with older one for some reason. It has worked fine for others. I just sent him a message to try to help him.
ok. I appreciate it. Ill edit the link in my newest compiled build thread, if you dont mind
Title: Re: uncompressed 14-bit RAW video recording
Post by: Colemar on May 18, 2013, 04:25:34 AM
confirmed, lourenco helped me out and the build works great.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 18, 2013, 04:38:08 AM
Quote from: a1ex on May 17, 2013, 05:58:52 PM
Huh? works for me for 5GB files (didn't try larger)

How did you record over 4GB?
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 18, 2013, 04:51:45 AM
Quote from: Alia5 on May 18, 2013, 02:24:47 AM
Did record some test footage with a lighter with the edmac_poppin...



you can read everything in the youtube description...

got 2 pink frames... but camera also skipped 2...

with the raw_hdr versions i think i never got any magenta frames

Edit: flickering comes from 2 crt monitores!
Try 1280x 400 in 720p mode, fps override at 23.98 optimized for low light, 1/24 shutter. Then just stretch the 400 to 644.

This vid was 1280x 400 in 640x480 mode, iso 6400 cleaned up with neat video. I think vimeo did noise reduction as well, the original file had WAY more noise...

Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 18, 2013, 05:17:46 AM
Quote from: RenatoPhoto on May 17, 2013, 09:51:52 PM
What is you point?  Just sarcasm?

No offence intended.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Colemar on May 18, 2013, 05:19:00 AM
@N/A,
   Dan Hudgins http://www.dancad3d.com/ (http://www.dancad3d.com/) is currently working on fixed-pattern noise reduction and figuring out what exactly is going on with Canon's 14bit buffer...I'll post back more info in the next couple days as to his findings.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 18, 2013, 05:31:37 AM
Quote from: lourenco on May 18, 2013, 03:24:00 AM
5D3 update files. Replace existing files. 
https://drive.google.com/folderview?id=0B7QlH_BH2m32QjRhRDhUeVBwZmM&usp=sharing
includes cf_acc: module as noted on http://www.magiclantern.fm/forum/index.php?topic=5471.msg36588#msg36588

Cool thanks. I'll take the kids out for a bike ride and put a1exs judder fix to the test.
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 18, 2013, 05:38:10 AM
Quote from: Colemar on May 18, 2013, 05:19:00 AM
@N/A,
   Dan Hudgins http://www.dancad3d.com/ (http://www.dancad3d.com/) is currently working on fixed-pattern noise reduction and figuring out what exactly is going on with Canon's 14bit buffer...I'll post back more info in the next couple days as to his findings.
Cool, that's the only issue I'm seeing now. Good times, good times...

Headed to the beach tomorrow, looks like I'll be lugging my cam and MacBook along with me.
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 18, 2013, 06:09:55 AM
I recorded a 12.7GB movie file. The raw to dng converter is not working correctly as 1% noted. I was able to play back all 3,592 frames in the camera by using the raw video menu option for playback. All the frames look good.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 18, 2013, 06:13:17 AM
Just tried the new build. The cf_acc_mo module isn't helping, it's actually filling the buffer sooner.
600x lexar with cf_acc_mo 812.9Mb without 1.07Gb
60MB/s  sandisk with cf_acc_mo 348.4Mb without 406.4Mb
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 18, 2013, 06:18:16 AM
Quote from: squig on May 18, 2013, 06:13:17 AM
Just tried the new build. The cf_acc_mo module isn't helping, it's actually filling the buffer sooner.
600x lexar with cf_acc_mo 812.9Mb without 1.07Gb
60MB/s  sandisk with cf_acc_mo 348.4Mb without 406.4Mb

You really do not need to have the cf_acc.mo module. That is just part of the cf card test.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 18, 2013, 06:21:07 AM
Quote from: lourenco on May 18, 2013, 06:18:16 AM
You really do not need to have the cf_acc.mo module. That is just part of the cf card test.

Yeah I know, I was testing it  ;D
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 18, 2013, 06:38:27 AM
Quote from: lourenco on May 18, 2013, 03:24:00 AM
5D3 update files. Replace existing files. 
https://drive.google.com/folderview?id=0B7QlH_BH2m32QjRhRDhUeVBwZmM&usp=sharing
includes cf_acc: module as noted on http://www.magiclantern.fm/forum/index.php?topic=5471.msg36588#msg36588
Thank you, everything works.
   Now there is no problem switching standard movie recording in H264 after turning off the RAW.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Peter Linov on May 18, 2013, 06:40:47 AM
Quote from: lourenco on May 18, 2013, 06:18:16 AM
You really do not need to have the cf_acc.mo module. That is just part of the cf card test.
Oh, it's this, my card was quickly shut down when filling bufera?
  before more than 100 frames to write without problems, now after 60  off the record (RAW)
...
I remove the file from the card and it was much better!
  Now I shut easily 370 shots!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Picture and Color on May 18, 2013, 10:21:34 AM
My 1000x Komputerbay 128GB arrived today. Just as I had expected, it is quite slow. Topping out at 70MB/s... so I'm stuck with 1920 x 900 on a good take and most often 1920 x 840 to be safe. This is a slow 1000x card. I'm not sure how others like Cinema5D are writing 1920x1080 to the 128GBs.

(http://img571.imageshack.us/img571/5006/vram0.png)
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 18, 2013, 11:19:39 AM
For slow cards, this seems to help a bit:

https://bitbucket.org/hudson/magic-lantern/commits/2e1bde4e5d7c

at the expense of live preview...
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 18, 2013, 11:49:15 AM
Quote from: Picture and Color on May 18, 2013, 10:21:34 AM
My 1000x Komputerbay 128GB arrived today. Just as I had expected, it is quite slow. Topping out at 70MB/s... so I'm stuck with 1920 x 900 on a good take and most often 1920 x 840 to be safe. This is a slow 1000x card. I'm not sure how others like Cinema5D are writing 1920x1080 to the 128GBs.

(http://img571.imageshack.us/img571/5006/vram0.png)

I get similar speed results as you with my 128.  I have written about this issue here and on Cinema5D.  Bottom line, the KomputerBay 128s seem to be slower then the 64s.  Cinema5D has assured me that their 128s are fast so they just might have been lucky with theirs.  EOSHD was using 64s for his tests.  Most current reports of the 128s are benchmarking around 70MB/s.
Title: Re: uncompressed 14-bit RAW video recording
Post by: CoresNZ on May 18, 2013, 01:36:13 PM
Hey ML team,

Congrats and great work on ML over the past week! I have been following very closely and using the nightly builds since they were first made available. The results have been amazing and i can't believe how much progress has been made in such a short time. It's starting to look like a real usable feature!

There were some earlier mentions on this thread of being able to record raw video while simultaneously recording the standard h.264 stream to the sd or cf card. if this is the case then how would i go about enabling it. Without having to figure out how to turn on the necessarily code switches and compile it myself..

I am generally recording a solid 24 or 25p @ 1920x840 to a delkin x1000 16GB card on a 5d mk 3!

Cheers guys! Keep up the great work!
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 18, 2013, 02:11:58 PM
Quote from: lourenco on May 18, 2013, 06:09:55 AM
I recorded a 12.7GB movie file. The raw to dng converter is not working correctly as 1% noted. I was able to play back all 3,592 frames in the camera by using the raw video menu option for playback. All the frames look good.

I tried a 5.4GB movie file. I was not able to convert it in windows, but I was able to convert it in linux.
Title: Re: uncompressed 14-bit RAW video recording
Post by: eyeland on May 18, 2013, 02:24:47 PM
(i) Would it perhaps be beneficial to compile a list of the different threads that deal with the different aspects of RAW recording? I know we're still in the very early days of the recent RAW developments, but this could easily keep growing to the point where it should have its own Forum category or sub-category?
Edit: (ii) Perhaps we need an additional moderator that is not one of the main developers so the latter can concentrate on their respective fields of expertise rather than forum-moderation?
Perhaps I am just confused and this post just added to the problem it sought to remedy.. If so, just ignore/delete it :)

Title: Re: uncompressed 14-bit RAW video recording
Post by: budafilms on May 18, 2013, 02:34:48 PM
Hi everybody, it's possible record raw more than 4GB? One minute its very few if you when you work with actors, external sound, clapboard and all the things outside cats, dogs, eternal mountines and trees. Thanks a lot!
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 18, 2013, 02:59:34 PM
Tested the lv hack on 600D.. it seems to be helping.
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 18, 2013, 03:16:14 PM
Quote from: budafilms on May 18, 2013, 02:34:48 PM
all the things outside cats, dogs, eternal mountines and trees. Thanks a lot!

That's amazing :D

You can record over 4GB yes but I think people are just struggling to open the files at the moment. Won't be long I'm sure :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 18, 2013, 03:23:33 PM
Linux version was tested and opens the files. Windows and mac... I think not so much... Well good news at least you can open any >4gb files you have.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kuky on May 18, 2013, 04:24:26 PM
Quote from: lourenco on May 18, 2013, 03:24:00 AM
5D3 update files. Replace existing files. 
https://drive.google.com/folderview?id=0B7QlH_BH2m32QjRhRDhUeVBwZmM&usp=sharing
includes cf_acc: module as noted on http://www.magiclantern.fm/forum/index.php?topic=5471.msg36588#msg36588

There is also a raw_rec.o file in MODULES folder. Is this needed? New files you posted don't have this file...
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 18, 2013, 04:57:04 PM
Quote from: Kuky on May 18, 2013, 04:24:26 PM
There is also a raw_rec.o file in MODULES folder. Is this needed? New files you posted don't have this file...

Will these replacements files give more than 4gb recording-time for each clip?
Title: Re: uncompressed 14-bit RAW video recording
Post by: dude on May 18, 2013, 05:02:42 PM
I am a little bit confused about myLexar 1000x 16 GB..
It s writing around 77 MB/s, that s not fast...
Is there a nother benchmark out there with a lexarcard?
Title: Re: uncompressed 14-bit RAW video recording
Post by: JohnBarlow on May 18, 2013, 05:05:05 PM
What is the status of 5D2 aquisition?

If I can get a stable 1280x1080 I will be so happy
Title: Re: uncompressed 14-bit RAW video recording
Post by: jimmy on May 18, 2013, 05:05:55 PM
Heads up to the devs that shooting with the Anamorphic adjustment on the display seems to disrupt my write speed enough to drop frames. Using a Lexar 1000x, runs fine full screen but buffer fills up really fast when I switch to Anamorphic.
Title: Re: uncompressed 14-bit RAW video recording
Post by: AnotherDave on May 18, 2013, 05:09:41 PM
http://delkin.com/i-13072686-best-64gb-cf-1050x-udma-7-cinema-memory-card.html (http://delkin.com/i-13072686-best-64gb-cf-1050x-udma-7-cinema-memory-card.html)

Anyone try the Delkin 1050X 64GB?  I'm wondering if the 120MB/sec write speed is real...

Title: Re: uncompressed 14-bit RAW video recording
Post by: Stedda on May 18, 2013, 05:11:31 PM
Quote from: dude on May 18, 2013, 05:02:42 PM
I am a little bit confused about myLexar 1000x 16 GB..
It s writing around 77 MB/s, that s not fast...
Is there a nother benchmark out there with a lexarcard?

Check out amazon reviews... more than a few people say then can't get over 80 MB/s... the Transcend seems to be the better deal.
Title: Re: uncompressed 14-bit RAW video recording
Post by: dude on May 18, 2013, 05:21:33 PM
is it possible that write speed was faster months ago? i rarely remember i did a benchmark with 93 mb/s.
and i also bought it because amazon and other reviews were good...
i do not trust in transcend any more since they did not hold up ml on  my old 550d.
so the best way to go is a.. sandisk 1000x? i am looking for a new one now..
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 18, 2013, 05:43:28 PM
Quote from: Kuky on May 18, 2013, 04:24:26 PM
There is also a raw_rec.o file in MODULES folder. Is this needed? New files you posted don't have this file...
The raw_rec.o is not needed. only the raw_rec.mo is needed.
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 18, 2013, 05:52:02 PM
Quote from: Danne on May 18, 2013, 04:57:04 PM
Will these replacements files give more than 4gb recording-time for each clip?

It is possible to record past 4GB. This should be the correct one. I modified the version 1% made.  The over 4GB raw movie file will only convert in linux with raw2dng. I cannot manage to make the windows version to work. This will stop recording when card is full or when the user decides to stop recording.
raw_rec file https://docs.google.com/file/d/0B7QlH_BH2m32eDZEbXh5cnZQSTg/edit?usp=sharing

The memory card must be formatted as exfat given fat32 has max file size of about 4GB.

I will post new build and newer modified version of raw_rec later tonight when I have time.
Title: Re: uncompressed 14-bit RAW video recording
Post by: KahL on May 18, 2013, 06:02:29 PM
This is an overprocessed image, but I wanted to test it from Andrew Reid's 600D raw samples.
It's a 960x480, 240dpi resolution image. And when upscaled to 1920 horizontal resolution, it almost totally emulates the clarity of super 16 film!
(http://farm8.staticflickr.com/7394/8749951513_7a2b65371c_h.jpg)
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 18, 2013, 06:03:43 PM
Warning: disabling the 4GB checks will result in corrupted files on FAT32 cards, if recording stops automatically.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 18, 2013, 06:15:16 PM
Yea, nobody should be using fat32 for over 4gb. Glad its just corrupted files... I assumed >4gb would corrupt the whole card.

Title: Re: uncompressed 14-bit RAW video recording
Post by: ashtrai on May 18, 2013, 06:16:10 PM
Quote from: KahL on May 18, 2013, 06:02:29 PM
This is an overprocessed image, but I wanted to test it from Andrew Reid's 600D raw samples.
It's a 960x480, 240dpi resolution image. And when upscaled to 1920 horizontal resolution, it almost totally emulates the clarity of super 16 film!
(http://farm8.staticflickr.com/7394/8749951513_7a2b65371c_h.jpg)

In what way did you process this?
Title: Re: uncompressed 14-bit RAW video recording
Post by: AndreasK on May 18, 2013, 06:26:43 PM
Would it be possible to change the RAW2DNG converter so that it writes cinemaDNG files instead of DNG (foto) files? So that Davinci Resolve can read it?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 18, 2013, 06:30:35 PM
I like the fact that I can basically shoot psuedo-Super 16 but I'm experimenting with upscaling methods for lower res 600d DNGs and I'm finding it's best to batch upscale first in PS and not do it in AE (or your NLE). I'm also looking into algorithms. Bicubic is best for upscaling. Not sure what PS uses but I'm gonna try OnOne's Perfect Resize. It might cause weird artifacts though as it processes each shot independently.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kabuto1138 on May 18, 2013, 06:30:51 PM
Has anybody tried the Toshiba 1066x yet?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 18, 2013, 06:31:58 PM
Not RTFM but RTFF  ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 18, 2013, 06:40:13 PM
Yea big difference when uprezzing in AE vs selecting the bigger file in PS. WTF is ae doing? And it won't let you choose a diff size in ACR like PS does.
Title: Re: uncompressed 14-bit RAW video recording
Post by: KahL on May 18, 2013, 06:55:19 PM
Quote from: ashtrai on May 18, 2013, 06:16:10 PM
In what way did you process this?

Well the POST work is my own private cocktail, and that's not the complete setup either.
But the DNG file was imported into photoshop, changed horizontal resolution to 1920 and then added some sharpening. They probably won't resolve very well beyond 1920, but like super 16, this is a great choice.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Steven on May 18, 2013, 08:15:55 PM
Is 25p recording actually enabled?

It would be interesting to be able to start raw recording to the cf card and h.264 proxy recording to the SD card simultaneously with the dedicated canon start video recording button.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Vegandelight on May 18, 2013, 08:18:35 PM
Quote from: Steven on May 18, 2013, 08:15:55 PM
Is 25p recording actually enabled?

It would be interesting to be able to start raw recording to the cf card and h.264 proxy recording to the SD card simultaneously with the dedicated canon start video recording button.

Just set FPS Override to 25 fps afaik.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Picture and Color on May 18, 2013, 08:22:29 PM
I'm struggling to even complete files in 1920 x 840.

I wonder if I can format the card in a different way... perhaps it's writing blocks fragmented. Is formatting using EOScard or Macbook sufficient?

Quote from: platu on May 18, 2013, 11:49:15 AM
I get similar speed results as you with my 128.  I have written about this issue here and on Cinema5D.  Bottom line, the KomputerBay 128s seem to be slower then the 64s.  Cinema5D has assured me that their 128s are fast so they just might have been lucky with theirs.  EOSHD was using 64s for his tests.  Most current reports of the 128s are benchmarking around 70MB/s.
Title: Re: uncompressed 14-bit RAW video recording
Post by: sergiocamara93 on May 18, 2013, 08:35:27 PM
Quote from: 1% on May 18, 2013, 06:40:13 PM
Yea big difference when uprezzing in AE vs selecting the bigger file in PS. WTF is ae doing? And it won't let you choose a diff size in ACR like PS does.

PS actually reads the 240 ppi of the RAW images but I think After Effects defaults to the 72 ppi of video. It should explain why the images don't upres equally:
After effects is resizing 1920x1080 72 ppi -> 4K (or whatever) 72 ppi
Photoshop should be using 1920x1080 240ppi -> 4K 72 ppi (supposing you are using video resolution)
Title: Re: uncompressed 14-bit RAW video recording
Post by: KahL on May 18, 2013, 09:12:35 PM
Quote from: sergiocamara93 on May 18, 2013, 08:35:27 PM
PS actually reads the 240 ppi of the RAW images but I think After Effects defaults to the 72 ppi of video. It should explain why the images don't upres equally:
After effects is resizing 1920x1080 72 ppi -> 4K (or whatever) 72 ppi
Photoshop should be using 1920x1080 240ppi -> 4K 72 ppi (supposing you are using video resolution)

Pretty much. And a better roundabout way of processing the images would be through Lightroom 4. Then export to TIFF frame sequence, then transcode to DNxHD/Prores, OR export as JPEG frame sequence and edit that natively.

This entire workflow of processing brings things to the days of film on a certain level. Most definitely not a workflow for many people asking for this feature and would be reserved for larger, more well developed production budgets.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Luiz Roberto dos Santos on May 18, 2013, 09:23:27 PM
Quote from: Andy600 on May 18, 2013, 06:30:35 PM
I like the fact that I can basically shoot psuedo-Super 16 but I'm experimenting with upscaling methods for lower res 600d DNGs and I'm finding it's best to batch upscale first in PS and not do it in AE (or your NLE). I'm also looking into algorithms. Bicubic is best for upscaling. Not sure what PS uses but I'm gonna try OnOne's Perfect Resize. It might cause weird artifacts though as it processes each shot independently.

What is After Effects, it's probably using the algorithm "Bilinear". Photoshop works with Bicubic, a little better, but still bad for our application.
The OnOne algorithm works by triangulating areas ... is good, but gives strong halo and is slow.
It seems to me that the best interpolation algorithm at the moment, for our case, is the Lanczos.
Test the "Lanczos4Resize", as Avisynth installed, and opening in VirtualDubmod ... you will see a big difference in this.
After Lancsos, could work with "Spline", the best so far, working with 256 points, but it is slow.
Someone trying Smooth-Spline?
Title: Re: uncompressed 14-bit RAW video recording
Post by: captain_merrrica on May 18, 2013, 09:30:49 PM
Just a quick test, some awesome dynamic range.
May 16th nightly build
http://vimeo.com/66425218
Title: Re: uncompressed 14-bit RAW video recording
Post by: KahL on May 18, 2013, 09:35:19 PM
Quote from: captain_merrrica on May 18, 2013, 09:30:49 PM
Just a quick test, some awesome dynamic range.
May 16th nightly build
http://vimeo.com/66425218

Very nice there :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: AndreasK on May 18, 2013, 09:44:24 PM
Hi guys. I did a quick/dirty nothing scientific sharpness test with my BMCC vs 5D3. https://plus.google.com/u/0/b/111134722005606605004/111134722005606605004/posts/69Be6wvbrMj

I've noticed two strange things (besides the obvios sharpness results).

a) ISO800 on the BMCC is about ISO1250 on the Canon 5D3 when in Raw but ISO800 in H264 is just a tad darker then BMCC
Is Canon pushing the rawfeed +2/3 stops?
b) The Raw not only has the black bar on the left, if you look at the H264 you can also see that there are some more lines thrown away and the image stretched..hmm?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 18, 2013, 09:53:36 PM
Quote from: lourenco on May 18, 2013, 05:52:02 PM
It is possible to record past 4GB. This should be the correct one. I modified the version 1% made.  The over 4GB raw movie file will only convert in linux with raw2dng. I cannot manage to make the windows version to work. This will stop recording when card is full or when the user decides to stop recording.
raw_rec file https://docs.google.com/file/d/0B7QlH_BH2m32eDZEbXh5cnZQSTg/edit?usp=sharing

The memory card must be formatted as exfat given fat32 has max file size of about 4GB.

I will post new build and newer modified version of raw_rec later tonight when I have time.

Thank you very much! I,ll try it out a little later.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Hazer on May 18, 2013, 10:01:44 PM
Hi-

Wanted to throw in a quick Mac workflow tip I've been using.  I haven't seen this mentioned yet;  apologies if it has and I missed it:

As you have probably experienced, the raw .dng files produced by raw2dng don't display in Finder/Quicklook, Preview, FCP/FCPX, Motion et al.  The data is there, but the images show up black/blank.  Ater Effects and Photoshop will take them, but if you'd like to use them in other apps, I've discovered that Adobe's DNG Converter utility fixes the problem *without* needing to convert everything to a tiff sequence and step away from raw.  Here's the download link:

http://www.adobe.com/support/downloads/detail.jsp?ftpID=5568

After installing the app, just select all the .dng files for a given clip right in finder and open them.  This will launch DNG Converter.  Hit "Convert" and you'll get a new set of .dng files in the same folder that will work fine in the above applications.  You now can use Quicklook/spacebar in Finder on these new files as usual.  If you have Motion, you can drag them all right into the layers palette and you've got a raw image sequence.  Color correct and export as ProRes.

Note that these are still raw frames and the original image data *appears* unaltered.  I haven't examined in detail so those more knowledeable in this area should take a closer look.  However I think DNG Converter is just re-wrapping them in a more Mac-friendly way.

Thanks!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 18, 2013, 10:05:51 PM
Quote from: KahL on May 18, 2013, 09:12:35 PM
Pretty much. And a better roundabout way of processing the images would be through Lightroom 4. Then export to TIFF frame sequence, then transcode to DNxHD/Prores, OR export as JPEG frame sequence and edit that natively.

This entire workflow of processing brings things to the days of film on a certain level. Most definitely not a workflow for many people asking for this feature and would be reserved for larger, more well developed production budgets.

I would have preferred to be able to use Lightroom 4 processing raw. Too bad that batch processing show inconsistent results. For instance when getting highligts back whites treats differently on different frames. This is not a problem in adobe camera raw. Personally I believe the raw engine has gotten worse in lightroom 4 compared to version 3. Lightroom 3 is also inconsistent but not as much.
For now I work with fast conversion to ProRes422HQ via AE, using either the PS camera raw engine or directly in AE. It,s only one or two steps away after the files are converted. Made a workflow tutorial about this, easy to follow.

'http://www.youtube.com/watch?v=MSHVi7Jx9r0

//D
Title: Re: uncompressed 14-bit RAW video recording
Post by: pholler on May 18, 2013, 10:10:52 PM
@Hazer:
It looks like the DNGs from raw2dng do not have previews. Adobe DNG Converter adds previews to the filez.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Picture and Color on May 18, 2013, 10:13:40 PM
Ok, my new test clip which aren't of plants or dogs! :)

Every now and then I need to remove the battery and put back in before the camera will start (after re-inserting the card after offload). Using Lorenco's May 17 build. Otherwise, everything is working great.



Title: Re: uncompressed 14-bit RAW video recording
Post by: rudi on May 18, 2013, 10:17:17 PM
i was off some time, but have another, hopefully not too dumb question:
Would it be possible to use the edmac copy routine, to copy only every second line?
So sacrifying vertical resolution for more horizontal Pixels?
It doesn´t not necessary look too bad to interpolate every second line in post, and the possibilites would be enormous, if we could use e.g. 3840 x 820 @24fps, which would result in 125 MB/s, for a Super35mm equivalent Crop Factor of 1.5. In a cinemascope aspect ratio. If  you could it interpolate it to 3840 x 1640 in the post. see what i mean?

greetings

rudi

Title: Re: uncompressed 14-bit RAW video recording
Post by: Hazer on May 18, 2013, 10:17:48 PM
Quote from: pholler on May 18, 2013, 10:10:52 PM
@Hazer:
It looks like the DNGs from raw2dng do not have previews. Adobe DNG Converter adds previews to the filez.

Actually, I have that feature turned off, along with all the other options.  i.e. I have JPEG previews, embedded fast load data, lossy compression, and embedded original all turned off.  I have "preserve pixel count" turned on.  Hit convert.

Again, not sure what changes if any are happening to the original image data, but the resulting image *looks* the same and appears to respond the same to highlight recovery and so forth.  Further investigation is probably necessary.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 18, 2013, 10:21:30 PM
Quote from: Picture and Color on May 18, 2013, 10:13:40 PM
Ok, my new test clip which aren't of plants or dogs! :)

Every now and then I need to remove the battery and put back in before the camera will start (after re-inserting the card after offload). Using Lorenco's May 17 build. Otherwise, everything is working great.



Nice vid! You mention using komputerbay card. Was it 64gb or 128gb you worked with?
//D
Title: Re: uncompressed 14-bit RAW video recording
Post by: 5DanielMIII on May 18, 2013, 10:31:20 PM
Outside bright and sunny!:-)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Mei Lewis on May 18, 2013, 10:31:44 PM
Quote from: Danne on May 18, 2013, 10:05:51 PM
I would have preferred to be able to use Lightroom 4 processing raw. Too bad that batch processing show inconsistent results. For instance when getting highligts back whites treats differently on different frames. This is not a problem in adobe camera raw. Personally I believe the raw engine has gotten worse in lightroom 4 compared to version 3. Lightroom 3 is also inconsistent but not as much.
For now I work with fast conversion to ProRes422HQ via AE, using either the PS camera raw engine or directly in AE. It,s only one or two steps away after the files are converted. Made a workflow tutorial about this, easy to follow.

'http://www.youtube.com/watch?v=MSHVi7Jx9r0

//D

Many people process timelapses in Lightroom, so it must be possible to get consistent frame-frame exposures using it.

This plugin
http://lrtimelapse.com/
is specifically for doing timelapse in Lightroom and it may be some use for processing frames of video too.

If you don't like the current way Lighrtoom deals with RAW you can revert to the old 'process version', it's in the develop module under 'Camera Calibration' (near the bottom of the right hand panel.

Adobe says (and my experience tallies with this) that equivalent versions of Lightroom and Camera Raw use EXACTLY the same conversion engine, so I don't know why you'd have different results with the two, maybe you have slightly different versions installed, as Camera Raw gets updated more often than Lightroom, but Lightroom is sometimes ahead on big releases.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 18, 2013, 10:46:18 PM
Quote from: Mei Lewis on May 18, 2013, 10:31:44 PM
Many people process timelapses in Lightroom, so it must be possible to get consistent frame-frame exposures using it.

This plugin
http://lrtimelapse.com/
is specifically for doing timelapse in Lightroom and it may be some use for processing frames of video too.

If you don't like the current way Lighrtoom deals with RAW you can revert to the old 'process version', it's in the develop module under 'Camera Calibration' (near the bottom of the right hand panel.

Adobe says (and my experience tallies with this) that equivalent versions of Lightroom and Camera Raw use EXACTLY the same conversion engine, so I don't know why you'd have different results with the two, maybe you have slightly different versions installed, as Camera Raw gets updated more often than Lightroom, but Lightroom is sometimes ahead on big releases.

Well, I suggest you do your own testing. I tried lightroom 3 and 4. Also thought It was the exact same camera raw module but I,m getting different results, don,t exactly know why but with some clips there was inconsistencies when retaining highlights and whites in lightroom 4. I have adobe camera raw 6.7, oldie but a goldie.
Still, don,t know if I,d use lightroom since it can,t convert to ProRes without creating a unnecessary extra step converting to either tiff or jpg first. From AE you go straight to ProRes422HQ
Title: Re: uncompressed 14-bit RAW video recording
Post by: adam rubin on May 18, 2013, 10:47:35 PM
another test of mine with 5d iii raw 15 bit vs 5d iii all-i
Title: Re: uncompressed 14-bit RAW video recording
Post by: KMA_WWC on May 18, 2013, 11:03:32 PM
Quote from: Hazer on May 18, 2013, 10:01:44 PM
Hi-

Wanted to throw in a quick Mac workflow tip I've been using.  I haven't seen this mentioned yet;  apologies if it has and I missed it:

As you have probably experienced, the raw .dng files produced by raw2dng don't display in Finder/Quicklook, Preview, FCP/FCPX, Motion et al.  The data is there, but the images show up black/blank.  Ater Effects and Photoshop will take them, but if you'd like to use them in other apps, I've discovered that Adobe's DNG Converter utility fixes the problem *without* needing to convert everything to a tiff sequence and step away from raw.  Here's the download link:

http://www.adobe.com/support/downloads/detail.jsp?ftpID=5568

After installing the app, just select all the .dng files for a given clip right in finder and open them.  This will launch DNG Converter.  Hit "Convert" and you'll get a new set of .dng files in the same folder that will work fine in the above applications.  You now can use Quicklook/spacebar in Finder on these new files as usual.  If you have Motion, you can drag them all right into the layers palette and you've got a raw image sequence.  Color correct and export as ProRes.

Note that these are still raw frames and the original image data *appears* unaltered.  I haven't examined in detail so those more knowledeable in this area should take a closer look.  However I think DNG Converter is just re-wrapping them in a more Mac-friendly way.

Thanks!

Thanks for sharing!!! Definitely didn't see that before. I will try to test it out later today and see if its better than current post workflow.
Title: Re: uncompressed 14-bit RAW video recording
Post by: nahua on May 18, 2013, 11:06:58 PM
I'd like to thank the developers, all this work is just awesome!  However I have problems with the battery consumption.  Even after my 5dmkIII is off, the LED still blinks from time to time and my camera stays warm.  Also, battery power goes down real fast.  I have to remove the battery if I'm leaving it alone for awhile.  Any ideas? 
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 18, 2013, 11:09:19 PM
Quote from: a1ex on May 18, 2013, 06:03:43 PM
Warning: disabling the 4GB checks will result in corrupted files on FAT32 cards, if recording stops automatically.

Is File spanning in FAT32 not possible?
Title: Re: uncompressed 14-bit RAW video recording
Post by: eoshq on May 18, 2013, 11:20:51 PM
Some info about Adobe Camera Raw (ACR) and Lightroom:

ACR and Lightroom have the exact same raw processing engine and results, as long as they are of the same generation.

That is to say:
Lightroom 5 = ACR 8 (CC and CS6 with June 2013 update)
Lightroom 4 = ACR 7 (CS6)
Lightroom 3 = ACR 6 (CS5)

Of course different generations of ACR, like Lightroom 3 vs Lightroom 4, are going to give different results.

Starting with Lightroom 4 and ACR 7, Adobe introduced process version 2012, which automatically controls highlights from frame to frame without user interaction. This is very likely the cause of some flicker from frame to frame. You can switch to process 2010 in the camera calibration area to turn this feature off, but also loose the advances made in highlight and shadow control made in process 2012.

Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 18, 2013, 11:26:13 PM
A proposal to this wonderful Open Source community!

To help the DEVS and everybody it would be a good idea if specific RAW video issues are discussed in a separate thread based on the camera under discussion.

5D3  Uncompressed 14-bit RAW video testing - 5D Mark III
HERE:  http://www.magiclantern.fm/forum/index.php?topic=5473.0

5D2  Raw video on 5DMK2
HERE:  http://www.magiclantern.fm/forum/index.php?topic=5419.0

600D Uncompressed 600D Raw Video 
HERE: http://www.magiclantern.fm/forum/index.php?topic=5494.0

650D I dont see a specific RAW video thread.  Should there be one??  Anyone care to propose so I can update this message?

6D I dont see a specific RAW video thread.  Should there be one??  Anyone care to propose so I can update this message?
Any other ones??  Anyone care to propose so I can update this message?

--------------------

Also there are other specific threads for  stated by by g3g00 and Alex:

All questions regarding RAW VIDEO POST PROCESSING on any OS
HERE: http://www.magiclantern.fm/forum/index.php?topic=5404.0

All questions regarding RAW_REC MODULE COMPILATION/INSTALLATION/USAGE
HERE: http://www.magiclantern.fm/forum/index.php?topic=5405.0

Card benchmark - what's the optimal write buffer size?
HERE: http://www.magiclantern.fm/forum/index.php?topic=5471.0

Supported 1000x CF cards for continuous RAW recording
HERE: http://www.magiclantern.fm/forum/index.php?topic=5391.0

--------------------

Also there are three How to threads under Creative Tutorials that I have posted:

How to: -- RAW VIDEO ON 5D3 --
HERE: http://www.magiclantern.fm/forum/index.php?topic=5451.0

How to: -- RAW Digital Cropped Video on 5D3--
HERE: http://www.magiclantern.fm/forum/index.php?board=15.0

How to: -- 60fps RAW Video for 5D3 --
HERE: http://www.magiclantern.fm/forum/index.php?topic=5472.0

Maybe all of this should be added to first post as a sort of directory that can be updated where required.
uncompressed 14-bit RAW video recording
HERE: http://www.magiclantern.fm/forum/index.php?topic=5247.0

--------------------

Installation Stuff:

How to install Magic Lantern on 5D Mark III
HERE: http://www.magiclantern.fm/forum/index.php?topic=2602.0

Where to get Canon Firmware 1.1.3 for sure:
HERE: http://pel.hu/eoscard/

Where to get Compiled Nightly Builds?
HERE: http://www.magiclantern.fm/forum/index.php?topic=5413.0
Title: Re: uncompressed 14-bit RAW video recording
Post by: Mickeyboo on May 18, 2013, 11:33:53 PM
Upon pressing the trash button to get into ML Alpha3 for 5D Mark 3 I get the message " Script DIR Missing"  when I go to load modules
the raw video module loads "successfully" however when I proceed to the Movie Camera Icon and scroll down to start after making size 1920x1080  and start regardless of the card I'm using 2 different 1000x 128 gig Komputer Bay Cards and 1 Lexar Professional 300X (which I know is too slow) It stops with the message "Buffer usage Frames 32".  The other day I had more success than this...I've reformatted and
reloaded Alpha 3 twice but I still get what I've described. :'(

I am hoping to use this in Iceland soon for a project on glaciers and global warming any assistance or suggestions would be greatly appreciated.
In the last few days I had been fairly successful with 300x cards with of course some dropped frames but the Komputer Bay Cards and now the Lexar 300x are all pooping out at just 32 frames...shutter speed is 50.  Thanks for any information people can provide feel free to call me a moron if you wish for forgetting something basic but I think I've covered the bases....perhaps problem lies in "Script DIR Missing" message or in something I've got set on the Mark 3 which the Alpha software doesn't like. VIVE Magic Lantern!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 18, 2013, 11:50:26 PM
@RenatoPhoto - I'm in favor of more organized raw topics as you suggest. There are so many people wanting to know the raw capabilities and specifics of their camera body or just basic info and it's making the main topic messy.

A lot of people are so eager to find out info regarding raw video that I think there needs to be a post on the website's front page with the current state of play for each camera.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mnememedia on May 18, 2013, 11:50:47 PM
Danne,

I used lightroom 4 to process my raw DNGs for my test shoot and I didn't see any inconsistent highlight treatment. I used exposure, highlight/shadow recovery in nearly every shot. I did not use the tone curve for any of my shots though, so I'm not sure if that is consistent.



Quote from: Danne on May 18, 2013, 10:05:51 PM
I would have preferred to be able to use Lightroom 4 processing raw. Too bad that batch processing show inconsistent results. For instance when getting highligts back whites treats differently on different frames. This is not a problem in adobe camera raw. Personally I believe the raw engine has gotten worse in lightroom 4 compared to version 3. Lightroom 3 is also inconsistent but not as much.
For now I work with fast conversion to ProRes422HQ via AE, using either the PS camera raw engine or directly in AE. It,s only one or two steps away after the files are converted. Made a workflow tutorial about this, easy to follow.

'http://www.youtube.com/watch?v=MSHVi7Jx9r0

//D
Title: Re: uncompressed 14-bit RAW video recording
Post by: matt2491 on May 19, 2013, 12:00:42 AM
Quick test clip. Garbage Youtube compression. 1080p at least.

http://www.youtube.com/watch?v=BlEys3RB-Ck
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 19, 2013, 12:09:21 AM
Quote from: mnememedia on May 18, 2013, 11:50:47 PM
Danne,

I used lightroom 4 to process my raw DNGs for my test shoot and I didn't see any inconsistent highlight treatment. I used exposure, highlight/shadow recovery in nearly every shot. I did not use the tone curve for any of my shots though, so I'm not sure if that is consistent.



Hi! Looks nice! I might try the workflow with lightroom further on to see if i,ve done something wrong. I believe there was at least one more person in here that experienced the same problems as me. Still, if anyone has AE I don,t really see the benefits going through the hassle exporting from dng to tiff and then to for instance ProRes i lightroom. Dng staight to ProRes seems faster.
Thanks
//D
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 19, 2013, 12:49:05 AM
Quote from: Andy600 on May 18, 2013, 11:50:26 PM
@RenatoPhoto - I'm in favor of more organized raw topics as you suggest. There are so many people wanting to know the raw capabilities and specifics of their camera body or just basic info and it's making the main topic messy.

A lot of people are so eager to find out info regarding raw video that I think there needs to be a post on the website's front page with the current state of play for each camera.

I can only lead by example! 
Title: Re: uncompressed 14-bit RAW video recording
Post by: CoresNZ on May 19, 2013, 01:07:40 AM
Is it currently possible to record h.264 .movs simultaneously to sd for use as proxies in editing?

I'd love to try using the RAW feature for a short film shoot next weekend but the quick turnaround won't really allow for processing the dngs before editing..
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 19, 2013, 02:18:59 AM
Today I finally got my KomputerBay Professional 32GB 1000x CF UDMA7 so I tested it ...  ;D

http://www.youtube.com/watch?feature=player_detailpage&v=Pf9iib286Vs (http://www.youtube.com/watch?feature=player_detailpage&v=Pf9iib286Vs)
Title: Re: uncompressed 14-bit RAW video recording
Post by: bass1012dash on May 19, 2013, 05:26:45 AM
Quote from: a1ex on May 17, 2013, 07:43:45 PM
I've also tried to remove the 4GB check, but I've got files without footer (which can't be converted). So it stays there until we have a better solution.

any update on fixing the 4 gig limit?
at 1880 x 1080 we don't even have a minute of record time.

Title: Re: uncompressed 14-bit RAW video recording
Post by: optik on May 19, 2013, 06:11:08 AM

I don't have a Vimeo plus account so I am not able to upload in 1080 P however the YouTube clip is 1080 P.
Again big ups to the whole magic lantern team.
If the original video files are wanted please PM

KomputerBay Professional 64GB 1000x CF

TIF STILL - http://www.mediafire.com/view/?4lgpkresk1v52va
DNG STILL- http://www.mediafire.com/?br1697j90vftafs
http://vimeo.com/66484186  only 720 P :(
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 19, 2013, 06:26:21 AM
Quote from: mageye on May 19, 2013, 02:18:59 AM
Today I finally got my KomputerBay Professional 32GB 1000x CF UDMA7 so I tested it ...  ;D

http://www.youtube.com/watch?feature=player_detailpage&v=Pf9iib286Vs (http://www.youtube.com/watch?feature=player_detailpage&v=Pf9iib286Vs)
How long are you able to record for in 1080?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Stedda on May 19, 2013, 07:11:52 AM
Quote from: bass1012dash on May 19, 2013, 05:26:45 AM
any update on fixing the 4 gig limit?
at 1880 x 1080 we don't even have a minute of record time.

Let them develope. Alex already answered that question less than 24 hours ago.  ::)
Title: Re: uncompressed 14-bit RAW video recording
Post by: bass1012dash on May 19, 2013, 08:34:53 AM
Quote from: Stedda on May 19, 2013, 07:11:52 AM
Let them develope. Alex already answered that question less than 24 hours ago.  ::)

LoL, don't mean to bug.

Things are moving fast, just wanted to stay updated.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Picture and Color on May 19, 2013, 09:56:46 AM
Need a hand. I tried to go back to an earlier build and loaded the older files on to the CF card.

When I booted up the camera with this older build, ML did not run. Strange. So I tried to update firmware again. Partway through the process it appeared frozen at the 'boot flag enabled' message. I had to take the battery out.

Upon restart, ML does not run at all, no matter which build I have on my CF card. Also, when I try to update the firmware, it says no firmware update found and I have to pop out the battery.

Edit: Went and loaded the Mk3 alpha... then went back to the development May 17 build and it's back to normal. Leaving this one here in case someone else encounters this.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 19, 2013, 10:32:14 AM
Quote from: Picture and Color on May 19, 2013, 09:56:46 AM
Need a hand. I tried to go back to an earlier build and loaded the older files on to the CF card.

When I booted up the camera with this older build, ML did not run. Strange. So I tried to update firmware again. Partway through the process it appeared frozen at the 'boot flag enabled' message. I had to take the battery out.

Upon restart, ML does not run at all, no matter which build I have on my CF card. Also, when I try to update the firmware, it says no firmware update found and I have to pop out the battery.

Edit: Went and loaded the Mk3 alpha... then went back to the development May 17 build and it's back to normal. Leaving this one here in case someone else encounters this.

Did you try formatting the card on the computer before putting in another ML version?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 5DanielMIII on May 19, 2013, 10:49:39 AM


Really pushing the colors!:-)
Raw vs h.264 video

Thanks to MLheroes;-) all over the world for donating their time doing this!!!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 19, 2013, 11:18:14 AM
Courtesy of James Miller. This is lovely!

Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 19, 2013, 11:33:01 AM
Quote from: Andy600 on May 19, 2013, 11:18:14 AM
Courtesy of James Miller. This is lovely!



I need to go and shoot today
Title: Re: uncompressed 14-bit RAW video recording
Post by: budafilms on May 19, 2013, 11:50:17 AM
Quote from: Picture and Color on May 19, 2013, 09:56:46 AM
Need a hand. I tried to go back to an earlier build and loaded the older files on to the CF card.

When I booted up the camera with this older build, ML did not run. Strange. So I tried to update firmware again. Partway through the process it appeared frozen at the 'boot flag enabled' message. I had to take the battery out.

Upon restart, ML does not run at all, no matter which build I have on my CF card. Also, when I try to update the firmware, it says no firmware update found and I have to pop out the battery.

Edit: Went and loaded the Mk3 alpha... then went back to the development May 17 build and it's back to normal. Leaving this one here in case someone else encounters this.

Same problem. Err 80. Any ideas?
Title: Re: uncompressed 14-bit RAW video recording
Post by: jakdaniel1975 on May 19, 2013, 12:47:37 PM
Hello everyone, yesterday analyzing the dng file after conversion with raw2dng and comparing them with raw files taken with the same settings and lighting, I realized that working them in lightroom appear compression artifacts .. I do not know if developers ml if they are noticed .... I am attaching a 'picture .. or lowered the value of clarity .... https://dl.dropboxusercontent.com/u/71964196/CR2vsDNG.jpg

high  cr2

down  dng
Title: Re: uncompressed 14-bit RAW video recording
Post by: bronson on May 19, 2013, 01:08:45 PM
New video test shots.
Canon 5d Mark 2
Tokina 16-28mm, Sigma 70-200mm, Canon 24-105mm
Magic Lantern Build 16th may

Severail shots @1080p @960p @900p @840p
CF : Transcend 32Gb 1000x
Write 80-85mb/s and read 120-130mb/s but only 65mb/s in LV for now.



DNG are developped with Canon ACR and edited with After Effects (upscaled from 2 to 10 %)
Title: Re: uncompressed 14-bit RAW video recording
Post by: eyeland on May 19, 2013, 01:17:41 PM
Quote from: RenatoPhoto on May 18, 2013, 11:26:13 PM
A proposal to this wonderful Open Source community!

To help the DEVS and everybody it would be a good idea if specific RAW video issues are discussed in a separate thread based on the camera under discussion.

5D3  Uncompressed 14-bit RAW video testing - 5D Mark III
HERE:  http://www.magiclantern.fm/forum/index.php?topic=5473.0

5D2  Raw video on 5DMK2
HERE:  http://www.magiclantern.fm/forum/index.php?topic=5419.0

600D Uncompressed 600D Raw Video 
HERE: http://www.magiclantern.fm/forum/index.php?topic=5494.0

650D I dont see a specific RAW video thread.  Should there be one??  Anyone care to propose so I can update this message?

6D I dont see a specific RAW video thread.  Should there be one??  Anyone care to propose so I can update this message?
Any other ones??  Anyone care to propose so I can update this message?

--------------------

Also there are other specific threads for  stated by by g3g00 and Alex:

All questions regarding RAW VIDEO POST PROCESSING on any OS
HERE: http://www.magiclantern.fm/forum/index.php?topic=5404.0

All questions regarding RAW_REC MODULE COMPILATION/INSTALLATION/USAGE
HERE: http://www.magiclantern.fm/forum/index.php?topic=5405.0

Card benchmark - what's the optimal write buffer size?
HERE: http://www.magiclantern.fm/forum/index.php?topic=5471.0

Supported 1000x CF cards for continuous RAW recording
HERE: http://www.magiclantern.fm/forum/index.php?topic=5391.0

--------------------

Also there are three How to threads under Creative Tutorials that I have posted:

How to: -- RAW VIDEO ON 5D3 --
HERE: http://www.magiclantern.fm/forum/index.php?topic=5451.0

How to: -- RAW Digital Cropped Video on 5D3--
HERE: http://www.magiclantern.fm/forum/index.php?board=15.0

How to: -- 60fps RAW Video for 5D3 --
HERE: http://www.magiclantern.fm/forum/index.php?topic=5472.0

Maybe all of this should be added to first post as a sort of directory that can be updated where required.
uncompressed 14-bit RAW video recording
HERE: http://www.magiclantern.fm/forum/index.php?topic=5247.0

--------------------

Installation Stuff:

How to install Magic Lantern on 5D Mark III
HERE: http://www.magiclantern.fm/forum/index.php?topic=2602.0

Where to get Canon Firmware 1.1.3 for sure:
HERE: http://pel.hu/eoscard/

Where to get Compiled Nightly Builds?
HERE: http://www.magiclantern.fm/forum/index.php?topic=5413.0
+1
been waiting/asking for this patiently :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Mayo on May 19, 2013, 01:26:17 PM
Hey guys,

First, congratulations on the breakthrough: it's mindblowing, I feel like Darius Khondji  8)

However I do have the same battery draining (when camera off) problem as nahua below, on 6D. This is fairly worrying... anyone else having this problem? Devs?

Quote from: nahua on May 18, 2013, 11:06:58 PM
I'd like to thank the developers, all this work is just awesome!  However I have problems with the battery consumption.  Even after my 5dmkIII is off, the LED still blinks from time to time and my camera stays warm.  Also, battery power goes down real fast.  I have to remove the battery if I'm leaving it alone for awhile.  Any ideas?

Cheers, and thanks again for the great work.
Title: Re: uncompressed 14-bit RAW video recording
Post by: eyeland on May 19, 2013, 01:27:16 PM
Quote from: KahL on May 18, 2013, 06:02:29 PM
This is an overprocessed image, but I wanted to test it from Andrew Reid's 600D raw samples.
It's a 960x480, 240dpi resolution image. And when upscaled to 1920 horizontal resolution, it almost totally emulates the clarity of super 16 film!
(http://farm8.staticflickr.com/7394/8749951513_7a2b65371c_h.jpg)
Nice! When mentioned earlier, for some reason I thought that the 240dpi was too good to be true. Would love to see a comparison between 5D3 raw upscaled to 4K and 4K from BMC and 1D C respectively
Title: Re: uncompressed 14-bit RAW video recording
Post by: OliverRH on May 19, 2013, 01:29:45 PM
Quote from: jakdaniel1975 on May 19, 2013, 12:47:37 PM
Hello everyone, yesterday analyzing the dng file after conversion with raw2dng and comparing them with raw files taken with the same settings and lighting, I realized that working them in lightroom appear compression artifacts .. I do not know if developers ml if they are noticed .... I am attaching a 'picture .. or lowered the value of clarity .... https://dl.dropboxusercontent.com/u/71964196/CR2vsDNG.jpg

high  cr2

down  dng
Yeh that look weird..
Maybe a raw2cr2 is possible?
I don't have that much experience with Lighteroom, but do you have to set how many bit you're working with?
Title: Re: uncompressed 14-bit RAW video recording
Post by: AndreasK on May 19, 2013, 01:45:48 PM
Be sure to shoot 1928 width to get the same FOV as when shooting stills. Just did a comparsion. With 1928 + cropleft 7 and cropright 1 you get the identical piture as with a CR2 still. H264 is cropped and scaled  :o

https://plus.google.com/u/0/b/111134722005606605004/111134722005606605004/posts/TE4uJBWZU5m
Title: Re: uncompressed 14-bit RAW video recording
Post by: vicnaum on May 19, 2013, 01:54:58 PM
I wonder, if we can implement some lossless bayer compression algorithm. Take a look at this paper:
http://www4.comp.polyu.edu.hk/~cslzhang/paper/RTI_05_Aug.pdf

It's pretty old (2005), and maybe something newer has been done from that time, but... 4-5 bits for pixel! From a 8bit source - that's 2x compression. And maybe 14bit will compress better, dunno.

REDCODE, Cineform, etc - also compress RAW bayer data (although lossy, but...)

Anyways - uncompressed RAW data is too much overbandwidth, so sooner or later some compression that can be handled by our little camera processors - should be done. And I will be patiently waiting for it.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Redrocks on May 19, 2013, 02:02:47 PM
Quote from: RenatoPhoto on May 18, 2013, 11:26:13 PM
A proposal to this wonderful Open Source community!

To help the DEVS and everybody it would be a good idea if specific RAW video issues are discussed in a separate thread based on the camera under discussion.




+1 and maybe change the title of this thread to direct everyone to the first post for FAQ.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mvejerslev on May 19, 2013, 02:08:23 PM
>When mentioned earlier, for some reason I thought that the 240dpi was too good to be true

There is some confusion regarding dpi (ppi) in here. Resolution information does not matter at all when media is displayed on a monitor. On a monitor, the media will always display at the monitors resolution (ppi). The only thing that matters for size on a monitor is the medias dimensions (xy pixels) together with the ppi of the display unit.

Embedded resolution info is only used in print, to determine how many of those xy pixels should fit in one inch on paper.
Title: Re: uncompressed 14-bit RAW video recording
Post by: vipmask on May 19, 2013, 02:56:26 PM

Why? As it can be solved?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 19, 2013, 05:11:16 PM
I was messing around with the audio + video recording and had trouble syncing it up. After a quick search, I found that this was a general issue with ML's audio recorder ever since it was introduced, and that 1% investigated it for a bit. I wonder what's going on here. Even if I adjust the sample rate to some arbitrary value to sync it up perfectly, I don't think it'll guarantee sync at other parts of the clip. Testing the video frame rate with a stopwatch showed that it stays smooth and steady. Is there some kind of syncing process that happens in real time for normal recording? That might be why recording with audio on cost such a huge amount of overhead versus recording it separately lol.
Title: Re: uncompressed 14-bit RAW video recording
Post by: menoc on May 19, 2013, 06:01:15 PM
Quote from: Andy600 on May 19, 2013, 11:18:14 AM
Courtesy of James Miller. This is lovely!



The 5D3 Miller is using has the OLPF removed. Which gives the image an added boost in sharpness. The image is stunning!
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 19, 2013, 06:11:29 PM
QuoteIs there some kind of syncing process that happens in real time for normal recording? That might be why recording with audio on cost such a huge amount of overhead versus recording it separately lol.


Canon does something... seems to have been fixed with dma_memcpy which was taken out on 5dIII. It doesn't stretch anymore so much. Still a little shorter than video so I think you have to sync from one of the ends.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Vegandelight on May 19, 2013, 06:31:29 PM
I got pulled the last build. Doesnt compile anymore. Crap.


Edit: Redid it all, now its working again. May19th.

The Hack3D mode gave me the ability to shoot 1920x1280. But the cards seems to be so inconsistent, or do you change buffer size every time the shooting start? A1ex?

It goes from like not being able to shoot 1920x1080 for more than 1 second, to shooting 4gb worth of 1920x1280.
Title: Re: uncompressed 14-bit RAW video recording
Post by: rs555 on May 19, 2013, 09:08:05 PM
On my 5d2: when I run it at 1280x1080 it seems that the recording area (white rectangle) is not at the center position when an external monitor is attached via HDMI. Might just be a display problem...
Title: Re: uncompressed 14-bit RAW video recording
Post by: payne on May 19, 2013, 11:08:19 PM
Quote from: RenatoPhoto on May 18, 2013, 11:26:13 PM
A proposal to this wonderful Open Source community!

To help the DEVS and everybody it would be a good idea if specific RAW video issues are discussed in a separate thread based on the camera under discussion.

5D3  Uncompressed 14-bit RAW video testing - 5D Mark III
HERE:  http://www.magiclantern.fm/forum/index.php?topic=5473.0

5D2  Raw video on 5DMK2
HERE:  http://www.magiclantern.fm/forum/index.php?topic=5419.0

600D Uncompressed 600D Raw Video 
HERE: http://www.magiclantern.fm/forum/index.php?topic=5494.0

650D I dont see a specific RAW video thread.  Should there be one??  Anyone care to propose so I can update this message?

6D I dont see a specific RAW video thread.  Should there be one??  Anyone care to propose so I can update this message?
Any other ones??  Anyone care to propose so I can update this message?

--------------------

Also there are other specific threads for  stated by by g3g00 and Alex:

All questions regarding RAW VIDEO POST PROCESSING on any OS
HERE: http://www.magiclantern.fm/forum/index.php?topic=5404.0

All questions regarding RAW_REC MODULE COMPILATION/INSTALLATION/USAGE
HERE: http://www.magiclantern.fm/forum/index.php?topic=5405.0

Card benchmark - what's the optimal write buffer size?
HERE: http://www.magiclantern.fm/forum/index.php?topic=5471.0

Supported 1000x CF cards for continuous RAW recording
HERE: http://www.magiclantern.fm/forum/index.php?topic=5391.0

--------------------

Also there are three How to threads under Creative Tutorials that I have posted:

How to: -- RAW VIDEO ON 5D3 --
HERE: http://www.magiclantern.fm/forum/index.php?topic=5451.0

How to: -- RAW Digital Cropped Video on 5D3--
HERE: http://www.magiclantern.fm/forum/index.php?board=15.0

How to: -- 60fps RAW Video for 5D3 --
HERE: http://www.magiclantern.fm/forum/index.php?topic=5472.0

Maybe all of this should be added to first post as a sort of directory that can be updated where required.
uncompressed 14-bit RAW video recording
HERE: http://www.magiclantern.fm/forum/index.php?topic=5247.0

--------------------

Installation Stuff:

How to install Magic Lantern on 5D Mark III
HERE: http://www.magiclantern.fm/forum/index.php?topic=2602.0

Where to get Canon Firmware 1.1.3 for sure:
HERE: http://pel.hu/eoscard/

Where to get Compiled Nightly Builds?
HERE: http://www.magiclantern.fm/forum/index.php?topic=5413.0

We need to make sure and have a 60D thread when it's time. I am under the impression that the 60D will have the same power and bottlenecks as the 600D. Isn't this correct? Many people bought the 60D for the video at the time as it had the nicer build quality than the 550D and the articulating screen.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 19, 2013, 11:34:44 PM
Quote from: aaphotog on May 19, 2013, 06:26:21 AM
How long are you able to record for in 1080?

I wasn't able to record in 1080 (I wish! maybe one day  ;D)

1880x840

I rendered out the video 1:1 but in a 1920x1080 frame.

Even though I would obviously prefer 1920x1080 I like what I have at the moment the RAW makes so much difference. I love it!

Thanks again ML!
Title: Re: uncompressed 14-bit RAW video recording
Post by: zachnfine on May 20, 2013, 12:41:01 AM
Is anyone else finding that their 5D3 goes to sleep in the middle of recording? I've formatted my 32Gb Lexar 1000x CF card as exFAT and enabled >4Gb recording, and the longest recording I've managed (at 1928x1080 raw) before the screen shuts down and the mirror locks back into place is 5,159,790,912 bytes (5.16Gb). Hitting the start/stop button wakes the camera back up, but it's as if it's doing a full boot from shutdown as far as Magic Lantern is concerned (I see the card speed test happen, and then I have to go re-enable the modules and configure raw and audio before recording).

I'm not sure what's going on. Maybe somehow ML is taking over to such an extent that the Camera OS doesn't know there's activity going on, so it goes to sleep? It happens every time.

It's amazing how quickly this hack is developing into a useful feature, btw.
Title: Re: uncompressed 14-bit RAW video recording
Post by: sergiocamara93 on May 20, 2013, 12:44:34 AM
Quote from: zachnfine on May 20, 2013, 12:41:01 AM
I'm not sure what's going on. Maybe somehow ML is taking over to such an extent that the Camera OS doesn't know there's activity going on, so it goes to sleep? It happens every time.

Look if "Power saving" is set to "Disable" on Canon menu. If it is, that shouldn't happen.
Title: Re: uncompressed 14-bit RAW video recording
Post by: silvertonesx24 on May 20, 2013, 12:46:58 AM
Quote from: rs555 on May 19, 2013, 09:08:05 PM
On my 5d2: when I run it at 1280x1080 it seems that the recording area (white rectangle) is not at the center position when an external monitor is attached via HDMI. Might just be a display problem...

Same thing here
Title: Re: uncompressed 14-bit RAW video recording
Post by: moomilk on May 20, 2013, 12:56:18 AM
Quote from: payne on May 19, 2013, 11:08:19 PM
We need to make sure and have a 60D thread when it's time. I am under the impression that the 60D will have the same power and bottlenecks as the 600D. Isn't this correct? Many people bought the 60D for the video at the time as it had the nicer build quality than the 550D and the articulating screen.

It seems like 60D as well as the 550D\600D will be maxed out near ~1280x400, while 650D is already reported to work fine at 1280x540.
I really hope the great ML developer team will make 50D the best entry level highres RAW video DSLR soon.))
Title: Re: uncompressed 14-bit RAW video recording
Post by: swinxx on May 20, 2013, 01:18:52 AM
Canon 5d Mk3 RAW Video shot with Iscorama 36 Lens


Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 20, 2013, 01:55:00 AM
Quote from: Yoshiyuki Blade on May 19, 2013, 05:11:16 PM
I was messing around with the audio + video recording and had trouble syncing it up. After a quick search, I found that this was a general issue with ML's audio recorder ever since it was introduced, and that 1% investigated it for a bit. I wonder what's going on here. Even if I adjust the sample rate to some arbitrary value to sync it up perfectly, I don't think it'll guarantee sync at other parts of the clip. Testing the video frame rate with a stopwatch showed that it stays smooth and steady. Is there some kind of syncing process that happens in real time for normal recording? That might be why recording with audio on cost such a huge amount of overhead versus recording it separately lol.

Ted was testing it and said it's consistently off by 4 frames.
Title: Re: uncompressed 14-bit RAW video recording
Post by: zachnfine on May 20, 2013, 02:00:02 AM
Quote from: sergiocamara93 on May 20, 2013, 12:44:34 AM
Look if "Power saving" is set to "Disable" on Canon menu. If it is, that shouldn't happen.

I'll check that setting when I next try. If it fixes it, I'll add that to the list of settings I have to double-check or change before recording.

For what it's worth, raw2dng.exe is unable to convert my 5.16Gb raw file to dng, comes up with "this ain't a lv_rec"... error. So I'd guess maybe no footer was recorded to the file due to the sleep/shutdown.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Kabuto1138 on May 20, 2013, 03:29:58 AM
Hey Guys,

Have any of you experienced color shifts halfway through a clip with the 5d2?
Title: Re: uncompressed 14-bit RAW video recording
Post by: haysuess on May 20, 2013, 03:42:18 AM
I tried the May 19 build on my 5D Mark II, and I can only get 155-185 frames at 1280x720 @ 24fps on a Transcend 400x 32GB card, even with Hack3d mode.

I know the card isn't the best, but it should only need 36.9MB/s for these settings, and my card benchmarks around 45-50MB/s. I feel like I'm getting way less than I should, anyone have any other ideas?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 5DanielMIII on May 20, 2013, 04:48:19 AM
.
Title: Re: uncompressed 14-bit RAW video recording
Post by: nahua on May 20, 2013, 09:07:19 AM
Quote from: sergiocamara93 on May 20, 2013, 12:44:34 AM
Look if "Power saving" is set to "Disable" on Canon menu. If it is, that shouldn't happen.

Yes I was having this problem too.  During recording I thought it was the hack crashing my camera.  But it happened during playback too which is strange.  I looked at power saving and it was set at 1 minute.  I changed to disable and it's fine, I can record long clips.  Just remember that it will stay in live view mode continuously, so don't point your camera at the sun otherwise it'll burn the sensor out.  LOL
Title: Re: uncompressed 14-bit RAW video recording
Post by: nahua on May 20, 2013, 09:10:43 AM
Quote from: Mayo on May 19, 2013, 01:26:17 PM
Hey guys,

First, congratulations on the breakthrough: it's mindblowing, I feel like Darius Khondji  8)

However I do have the same battery draining (when camera off) problem as nahua below, on 6D. This is fairly worrying... anyone else having this problem? Devs?

Cheers, and thanks again for the great work.
Yeah my 5D mkIII battery drains real fast.  I have the battery grip with 2 batteries.  After an hour my batteries are around 40%.  They aren't that old, and I don't remember them draining so fast with the H.264 recording.  I do think this has to do with live view on all the time and also constant recording to the CF cards.  In any case, I think I need to get more batteries. Another thing, my camera is still "working" - LED blinking - even if it's off.  So I pull the battery every time I put my camera away just in case.
Title: Re: uncompressed 14-bit RAW video recording
Post by: budafilms on May 20, 2013, 09:22:27 AM
Quote from: nahua on May 20, 2013, 09:10:43 AM
Yeah my 5D mkIII battery drains real fast.  I have the battery grip with 2 batteries.  After an hour my batteries are around 40%.  They aren't that old, and I don't remember them draining so fast with the H.264 recording.  I do think this has to do with live view on all the time and also constant recording to the CF cards.  In any case, I think I need to get more batteries. Another thing, my camera is still "working" - LED blinking - even if it's off.  So I pull the battery every time I put my camera away just in case.

Yes, the hack result the same problem for me. I am looking for a more stable realease. The problem is in the compiled May 19. Do you know an stable one?
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 20, 2013, 09:28:53 AM
I doubt the battery drain is higher than with Canon firmware. See http://www.magiclantern.fm/forum/index.php?topic=1573.0

Measurements please, not speculation.
Title: Re: uncompressed 14-bit RAW video recording
Post by: PressureFM on May 20, 2013, 11:37:51 AM
Quote from: haysuess on May 20, 2013, 03:42:18 AM
I tried the May 19 build on my 5D Mark II, and I can only get 155-185 frames at 1280x720 @ 24fps on a Transcend 400x 32GB card, even with Hack3d mode.

I know the card isn't the best, but it should only need 36.9MB/s for these settings, and my card benchmarks around 45-50MB/s. I feel like I'm getting way less than I should, anyone have any other ideas?

I can easily do 1200-1300 frames on my Transcend 300x 16 GB card at 1280x720@24p, so try a different build.
Title: Re: uncompressed 14-bit RAW video recording
Post by: denmarkdavis on May 20, 2013, 11:52:41 AM
Quote from: platu on May 18, 2013, 11:49:15 AM
I get similar speed results as you with my 128.  I have written about this issue here and on Cinema5D.  Bottom line, the KomputerBay 128s seem to be slower then the 64s.  Cinema5D has assured me that their 128s are fast so they just might have been lucky with theirs.  EOSHD was using 64s for his tests.  Most current reports of the 128s are benchmarking around 70MB/s.

To add: My new Komputerbay 128 1000x is writing at around 67mb/s... Trying to send back to try a 64gb.
Title: Re: uncompressed 14-bit RAW video recording
Post by: weigertj on May 20, 2013, 12:25:56 PM
I am thinking, whether it is possible with the 5D MK III using both a CF and SD card at the same time to record every third or fourth frame only on the SD card? The filenames would be the same, thus uncompressing could easily go to the same folder. This way we could reach a 120-130 MB write speed with the fastest cards at the moment (80-90MB/s CF + 40 MB/sec SD).

Anyway, kudos to the Magic Lantern Team! 
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 20, 2013, 03:43:31 PM
So with memory chunks I'm thinking this:

Are the chunks even on 5dIII? 6D lost speed by allocating all memory and I think I have a theory as to why.
The chunks are all uneven now and the smallest chunks are getting used first. When it was ordered the other way small chunks were at the end. Now when buffer 1 or 2 is blinking and that chunk is lets say 27MB the card either needs 2 chunks to write at 31MB or is just writing at 27MB and being inefficient.

Thoughts?

Title: Re: uncompressed 14-bit RAW video recording
Post by: KMA_WWC on May 20, 2013, 04:17:47 PM
Here is my take on ML 14bit RAW hack. Used 5dIII with Hoodman Steel 1000x 64gb card. Please let me know what you think. #NonCatDogFlowerWindowVideo

http://vimeo.com/66555717
Title: Re: uncompressed 14-bit RAW video recording
Post by: deknk12 on May 20, 2013, 04:21:49 PM
With a SanDisk 16GB Extreme 90mb/s card, I hit the 4gb file size limit at 1880x880 and 1880x720 with global draw off.

Now, if I could only get the DNG out of that file...

Quick sample:

https://vimeo.com/66530384
Title: Re: uncompressed 14-bit RAW video recording
Post by: haysuess on May 20, 2013, 04:56:38 PM
Quote from: PressureFM on May 20, 2013, 11:37:51 AM
I can easily do 1200-1300 frames on my Transcend 300x 16 GB card at 1280x720@24p, so try a different build.

I've tried every build so far. The amount of frames I can get is always in the 185 range. With no sound, global draw, and with Hack3d mode on.

I think I'll have to get my hands on a new card and see if the one I have is just bad for some reason.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 20, 2013, 05:22:05 PM
I do apologise. More flowers :(

RAW Test - KomputerBay Professional 32GB 1000x CF UDMA7 - Part II

http://www.youtube.com/watch?feature=player_detailpage&v=RBt9nvER--o (http://www.youtube.com/watch?feature=player_detailpage&v=RBt9nvER--o)
Title: Re: uncompressed 14-bit RAW video recording
Post by: DjJuvan on May 20, 2013, 08:36:52 PM
Maybe a stupid question, which could be answered already somewhere but I didn't notice.... Why can't I record more than 1920x1288? Even if I can't on 24fps, I could lower framerate and make timelapses at bigger resolution?

Thank you ^^
Title: Re: uncompressed 14-bit RAW video recording
Post by: Vegandelight on May 20, 2013, 10:47:28 PM
Quote from: DjJuvan on May 20, 2013, 08:36:52 PM
Maybe a stupid question, which could be answered already somewhere but I didn't notice.... Why can't I record more than 1920x1288? Even if I can't on 24fps, I could lower framerate and make timelapses at bigger resolution?

Thank you ^^

Use either 5x or 10x zoom mode, then you can choose higher resolutions. But that mode crops the sensor output.
Title: Footage from Salinas Valley State Prison on Saturday - Canon 600D 960x500 raw
Post by: mattmvpmedia on May 20, 2013, 11:01:49 PM


Here are a few shots from a prison ministry outreach event held this last Saturday, May 18 at Salinas Valley State Prison in California. Prison Fellowship brought in a team of more than 30 volunteers to minister to inmates in the level 4 maximum security area of Salinas Valley prison. It was a great day.
I shot just a few shots that day in raw as a test. Shot in 960x500 resolution raw on a Canon t3i (600D) using Magic Lantern firmware. I did get some frames corrupted with magenta blocking and tearing, but not too bad. I edited out those frames for this video. Lens: Tokina 11-16mm F2.8. ISO: 200 Aperture: F3.2. Genus Variable ND filter. Sorry the sound doesn't match up - I used sound recorded earlier in the day on a Zoom H4n and used it with these clips.
Workflow - I used dng2raw from Magic Lantern to convert the raw files to dng files on my MacBookPro. I opened the dng files in Adobe Photoshop CS 5.5, applied basic settings, and saved as TIFF files. Opened the image sequence in Quicktime 7. Exported as ProRes HQ movie. Edited the ProResHQ movies in Premiere Pro CS 5.5, exported as a ProRes HQ movie, uploaded to Vimeo.
Title: Re: uncompressed 14-bit RAW video recording
Post by: zachnfine on May 21, 2013, 01:09:38 AM
Quote from: zachnfine on May 20, 2013, 02:00:02 AM
For what it's worth, raw2dng.exe is unable to convert my 5.16Gb raw file to dng, comes up with "this ain't a lv_rec"... error. So I'd guess maybe no footer was recorded to the file due to the sleep/shutdown.

I looked at the 5.16Gb raw file in a hex editor (hexfiend), and it definitely has a footer. I also tried overwriting its footer with that of a known working raw file, but raw2dng.exe still gave me the "ain't" error. If anyone would like to take a look at the original raw file to see if they can figure out why it can't be converted to dng, I've shared it from my dropbox here:
https://www.dropbox.com/s/aaxl01lptoqbywp/plants-exfat-nolvrec.RAW

Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 21, 2013, 01:34:20 AM
I know they have footers.. maybe only ones that stopped with the card might be missing them. Use the linux binary to convert them, its not working in windows/mac.
Title: Re: uncompressed 14-bit RAW video recording
Post by: rodobot on May 21, 2013, 07:48:31 AM
First of all, thanks for great tools!

I am not a programmer, so this is probably not a correct way to do (In fact, I had a problem even with simple type casting ending up hardcoding..), but I hope this will help you iron out the issue with raw2dng.

First, I guessed that fseek does not support files larger than 4 gb on OSX, and after little googling, I decided to replace "fseek" with "fseeko".

I compiled and tried... failed...

Next, I guessed the second parameter of the first fseeko (i.e. -sizeof(lv_rec_file_footer_t)) is not casted correctly. After inserting printf's to see actual values, I found out that lv_rec_file_footer_t is 192.

So I replaced the parameter -sizeof(lv_rec_file_footer_t) with -192 and compiled (I don't know a correct way to cast sizeof to off_t (or long long?)).

Now, with these changes, raw2dng seems to be processing raw files larger than 4gb on OSX.

I used the file provided by zachnfine (thanks a lot!) and was able to extract 1416 files (to 001415.dng).

I have checked that 001415.dng opens normally using PhotoShop.

Again, thanks a lot for great tools and I hope this is useful.
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 21, 2013, 07:53:38 AM
Great, can you check this changeset?

http://bitbucket.org/hudson/magic-lantern/commits/9c05522bc3ad

If it works, I'm going to update the exe on too.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mjneubrander on May 21, 2013, 08:01:38 AM
After installing build 006d890 May 19th Nightly for 5D mark III I have been infected with the 145 frame limit (at 720p) 5x zoom issue. After starting the build I enabled hack3d to test it. Next I tried the 5x zoom which stopped at 145 frames. I tested 4 previous builds (e91e2bc, f686b8c, and builds from may 15, 13) and the 145 frame zoom issue persists. I also changed sd cards and used eoscard to rebuild. Before using build 006d890, I can confirm working (centered only) 5x zoom on builds e91e2bc, f686b8c. This all lead me to believe that it changes a setting on the camera. Did I miss a fix for this? Side note digital dolly is great wouldn't mind being able to set the movement speed from a menu.
Title: Re: uncompressed 14-bit RAW video recording
Post by: rodobot on May 21, 2013, 08:03:23 AM
I also changed fseek on Line 39 to fsseko.

But I must say that I don't know if this works on windows.

*********
I tested updated rwa2dng.c in repository, and it worked.
So I guess you don't need to replace 2nd one on Line 39 (Guess why I am not a programmer lol).
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 21, 2013, 08:10:49 AM
Quote from: a1ex on May 21, 2013, 07:53:38 AM
Great, can you check this changeset?

http://bitbucket.org/hudson/magic-lantern/commits/9c05522bc3ad

If it works, I'm going to update the exe on too.

I,d love to test this on my mac. But how  :)?
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 21, 2013, 08:16:56 AM
You can set the movement speed by pressing the arrow key once, twice etc. It starts at the minimal one possible right now (8 pixels at a time...)

It can be slowed down, but only by changing the converter (it needs metadata for every single frame). Not easy.

e91e2bc works and 006d890 doesn't? What exact settings?

The only backend difference is a 32MB write fix, which only makes a difference if the frame size (w * h * 14/8) is a divisor of 32MB.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mjneubrander on May 21, 2013, 08:45:14 AM
After installing 006d890 no previous build works in zoom not sure if there are newer builds to test :(.

Should I try pull the backup battery or is that pointless?

Same settings Global draw, zebras, spotmeter, histogram all off. Sound set as sync beep video set 1280x720 using a x400 card with 36Mb write (tested in camera). When I start a raw record in zoom it only shows (...) for ram instead of (....) when in non zoom.

Found the arrow key multiple press thanks ! Now if I just had smoother hands :).

Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 21, 2013, 08:49:45 AM
There was some slowdown in 5x after fixing the tearing (this required some more memory). Just pushed a fix that should restore the old speed (almost).
Title: Re: uncompressed 14-bit RAW video recording
Post by: scrax on May 21, 2013, 09:08:18 AM
Quote from: a1ex on May 21, 2013, 07:53:38 AM
Great, can you check this changeset?

http://bitbucket.org/hudson/magic-lantern/commits/9c05522bc3ad

If it works, I'm going to update the exe on too.
working here too!
Title: Re: uncompressed 14-bit RAW video recording
Post by: mjneubrander on May 21, 2013, 09:24:12 AM
I was also able clear the 145 frame issue using the files from below. And starting a zoom raw rec from any C1, C2 ,C3 modes but not the M mode.
Quote from: lourenco on May 18, 2013, 03:24:00 AM
5D3 update files. Replace existing files. 
https://drive.google.com/folderview?id=0B7QlH_BH2m32QjRhRDhUeVBwZmM&usp=sharing
includes cf_acc: module as noted on http://www.magiclantern.fm/forum/index.php?topic=5471.msg36588#msg36588
Then as any good tester I was able to repeat the issue from the 006d890 build. And repeat the fix!
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 21, 2013, 09:27:49 AM
How much free memory do you have in C modes? (in Debug menu).

I have 168MB, sometimes 88.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mjneubrander on May 21, 2013, 10:12:01 AM
Showing 168 M in both Manual and C modes for both 006d890 and e91e2bc.

Trying to pin down how this error is made can't seem to get it yet 60+ tests and configurations. It was repeatable from the memory card with 006d890 but then I rebuilt the ML on that card and it works...
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 21, 2013, 10:56:41 AM
Quote from: rodobot on May 21, 2013, 07:48:31 AM
First of all, thanks for great tools!

I am not a programmer, so this is probably not a correct way to do (In fact, I had a problem even with simple type casting ending up hardcoding..), but I hope this will help you iron out the issue with raw2dng.

First, I guessed that fseek does not support files larger than 4 gb on OSX, and after little googling, I decided to replace "fseek" with "fseeko".

I compiled and tried... failed...

Next, I guessed the second parameter of the first fseeko (i.e. -sizeof(lv_rec_file_footer_t)) is not casted correctly. After inserting printf's to see actual values, I found out that lv_rec_file_footer_t is 192.

So I replaced the parameter -sizeof(lv_rec_file_footer_t) with -192 and compiled (I don't know a correct way to cast sizeof to off_t (or long long?)).

Now, with these changes, raw2dng seems to be processing raw files larger than 4gb on OSX.

I used the file provided by zachnfine (thanks a lot!) and was able to extract 1416 files (to 001415.dng).

I have checked that 001415.dng opens normally using PhotoShop.

Again, thanks a lot for great tools and I hope this is useful.

Nice one man! Would be nice to see to see this ported to the Windows .exe :)

Edit:
Quote from: a1ex on May 21, 2013, 07:53:38 AM
If it works, I'm going to update the exe on too.

*slaps forhead* reading thread properly FAIL. Sorry ;)[/i]
Title: Re: uncompressed 14-bit RAW video recording
Post by: lourenco on May 21, 2013, 01:09:51 PM
Quote from: noisyboy on May 21, 2013, 10:56:41 AM
Nice one man! Would be nice to see to see this ported to the Windows .exe :)

Edit:
*slaps forhead* reading thread properly FAIL. Sorry ;)

There is a new one for windows too now. It works fine over 4GB now. It was able to convert my 8GB raw movie file.
https://bitbucket.org/hudson/magic-lantern/downloads/RAw2dng.exe
Title: Re: uncompressed 14-bit RAW video recording
Post by: tihon on May 21, 2013, 01:19:30 PM
Hi! Do you think that is possible to have  a raw zebra?   As i know blackmagic have it.

Thanks! I understand that all pros uses an histogram, but...
Title: Re: uncompressed 14-bit RAW video recording
Post by: marcb on May 21, 2013, 01:20:07 PM
Is it possible to simultaneously record H.264 proxies to an SD card while recording RAW to the CF on the 5DIII? The proxies would be very useful for offline editing and have the added advantage that sound is embedded in the file.
Title: Re: uncompressed 14-bit RAW video recording
Post by: djoulbdx on May 21, 2013, 02:03:27 PM
Hi,

I am aware of being boring ... I wanted to know if it will be possible on the 7D. You've explained to me the problem of slow bus between the two chips, but I can not resign myself. But I try;-)

g3gg0, Alex and the other ones : Thx.
Title: Re: uncompressed 14-bit RAW video recording
Post by: dude on May 21, 2013, 03:19:49 PM
The fact that you are not able to READ (especially the first and most important page)

(- bloody beginners and non-geeks should not touch the whole thing. wait until it is "beginner-proof". we will tell you on the website.
- you know that you are a bloody beginner, when you read the whole thread and you still cannot get it to work.
- DON'T be disappointed if it doesn't work or we figure out the whole thing is unstable and/or unusable
- NO, there is no manual yet
- NO, there is no all-in-one tool that fits every use case
- NO, we don't have tutorials how to use it
- NO, not all models are supported yet ;)
- we are just at the moment testing how good it works and what we have missed and what to improve
- you are welcome to post comparisons, experiences (both good and bad), or even deep analysis or just cool videos
- if you are a programmer and you see potential for improvements, grab the source and support :) )
doesn t make your begging better...
Title: Re: uncompressed 14-bit RAW video recording
Post by: djoulbdx on May 21, 2013, 03:34:45 PM
Quote from: dude on May 21, 2013, 03:19:49 PM
The fact that you are not able to READ (especially the first and most important page)
- NO, not all models are supported yet ;)

"Yet" : That's why I prefer asking for the 7D. I want to be the first  ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: dngrhm on May 21, 2013, 04:09:36 PM
What if the footer was changed into a header?  Write a default header to the file.  Write the sequence of frames. After manual stop or stop on card-full, go back and overwrite the header at the beginning of the file.

This would eliminate the issue of not having space to write the footer if the card is full.  This would also put the info needed to read the file at the beginning instead of having to fseek to the end.  It might also make it easier to fix the header if needed.

This would kill compatibility with anything anyone has already done. :(
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 21, 2013, 04:29:45 PM
I think its at the end because it provides a frame count.
Title: Re: uncompressed 14-bit RAW video recording
Post by: dngrhm on May 21, 2013, 04:56:55 PM
Quote from: 1% on May 21, 2013, 04:29:45 PM
I think its at the end because it provides a frame count.

Yes and frame skips.  This is what makes the overwrite of the header after the recording is done necessary.  I don't see any reason in the code and how ML writes files why the file can't be closed then reopen and pass that file pointer to lv_rec_save_footer.  (I could be wrong on that) Otherwise you have to get into some tap dancing to make sure there is enough space on the card to write the footer. 
Title: Re: uncompressed 14-bit RAW video recording
Post by: AndreasK on May 21, 2013, 05:04:25 PM
hmm just received my 64gb transcend and I can not record 1080p25 witout skipped frames :(
when i do the benchmark i can get >100mb/sec with a different buffer. is there a way to adjust the buffer the raw encoding uses?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 21, 2013, 05:17:01 PM
Quote from: scrax on May 21, 2013, 09:08:18 AM
working here too!

Scrax - think you can get it working for us mac users?



*edit scrax already got it goin on - http://www.magiclantern.fm/forum/index.php?topic=5508.0
Title: Re: uncompressed 14-bit RAW video recording
Post by: stevefal on May 21, 2013, 05:26:36 PM
5X doesn't work for me - "Memory error MB"

running 006d890 on 5D3 + Lexar 64GB 1000X

[posted on test thread: http://www.magiclantern.fm/forum/index.php?topic=5473.msg38331#msg38331]
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 21, 2013, 06:46:46 PM
on all of the latest compiled nightly builds I'm getting a "scripts dir missing" message upon loading.
Do I NEED the script folder on the card? Where can I find the folder so that I may add it to my card?
Also, if I were to just download the 5d3 alpha 1.1.3, would I be able to just use the scripts folder from that, without issue?
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 21, 2013, 06:50:47 PM
Quote from: aaphotog on May 21, 2013, 06:46:46 PM
on all of the latest compiled nightly builds I'm getting a "scripts dir missing" message upon loading.
Do I NEED the script folder on the card? Where can I find the folder so that I may add it to my card?
Also, if I were to just download the 5d3 alpha 1.1.3, would I be able to just use the scripts folder from that, without issue?
Dont needed.  That is special function PicoC that is currently disabled.  Maybe you can download any of the nightly builds, get the scripts dir that is missing and put in inside ML.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 5DanielMIII on May 21, 2013, 07:42:59 PM
Quote from: RenatoPhoto on May 21, 2013, 06:50:47 PM
Dont needed.  That is special function PicoC that is currently disabled.  Maybe you can download any of the nightly builds, get the scripts dir that is missing and put in inside ML.

That is what I did, and script "warning" was gone!! ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 21, 2013, 09:23:25 PM
Quote from: 5DanielMIII on May 21, 2013, 07:42:59 PM
That is what I did, and script "warning" was gone!! ;)

That is also what I did.

Also, I just purchased a 16GB Sandisk CF card. It says up to 90mb/s 600x udma7
I did a benchmark and it only gets about 80mb/s. but when recording 1080p video(about 82-84mb/s), it works perfect!
I've recorded up to 3500 mb and the buffer never went past 1*

I wanted to buy the 64gb komputerbay for only 30 dollars more, but I'm impatient and don't like waiting on things that I can't personally go into a store and purchase. So, for those with 75 bucks, feel free to pick one of these cards up! Speed is of no issue!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Northernlight on May 21, 2013, 09:30:32 PM
RAW video sure is exciting, partcularly for me making my living in the darkness! I have installed it on my 5D3, and it records really well @24fps using my 128GB Lexar Pro 1000x UDMA7 card.

BUT, the vertical banding on the stills captured with my camera worries me. In my opinion it is so bad, that it renders the whole feature useless!

The banding is so bad that it is not something you just get rid of using noise reduction. In a way I really hope it is something wrong with my camera, but I doubt it. Will post pictures later.

What on earth could be causing this banding?

Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 22, 2013, 12:08:42 AM
Quote from: Northernlight on May 21, 2013, 09:30:32 PM
RAW video sure is exciting, partcularly for me making my living in the darkness! I have installed it on my 5D3, and it records really well @24fps using my 128GB Lexar Pro 1000x UDMA7 card.

BUT, the vertical banding on the stills captured with my camera worries me. In my opinion it is so bad, that it renders the whole feature useless!

The banding is so bad that it is not something you just get rid of using noise reduction. In a way I really hope it is something wrong with my camera, but I doubt it. Will post pictures later.

What on earth could be causing this banding?



Overexpose a little. (Expose to the right: google it) further reading into the forum will explain it (fixed pattern noise)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Lars Steenhoff on May 22, 2013, 12:13:16 AM
Neat video seems to get rid of this pattern noise when I tried it on a shot.

http://www.neatvideo.com
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 22, 2013, 12:28:06 AM
Quote from: AndreasK on May 21, 2013, 05:04:25 PM
hmm just received my 64gb transcend and I can not record 1080p25 witout skipped frames :(
when i do the benchmark i can get >100mb/sec with a different buffer. is there a way to adjust the buffer the raw encoding uses?

HI! Is it a 1000x card and have you tried it on 24p at 1080p? I,m about to buy one but it doesn,t seem to work that good?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 22, 2013, 12:32:39 AM
Quote from: Northernlight on May 21, 2013, 09:30:32 PM
RAW video sure is exciting, partcularly for me making my living in the darkness! I have installed it on my 5D3, and it records really well @24fps using my 128GB Lexar Pro 1000x UDMA7 card.

BUT, the vertical banding on the stills captured with my camera worries me. In my opinion it is so bad, that it renders the whole feature useless!

The banding is so bad that it is not something you just get rid of using noise reduction. In a way I really hope it is something wrong with my camera, but I doubt it. Will post pictures later.

What on earth could be causing this banding?

As stated before it helps a lot to overexpose your shots. That,s the workaround with canon sensors right now. This is not the case with the sony sensors which can read the shadows much better. Still, when overexposing and retaining the highlights in post you can achieve really good results
Title: Re: uncompressed 14-bit RAW video recording
Post by: sergiocamara93 on May 22, 2013, 12:59:23 AM
Quote from: Danne on May 22, 2013, 12:28:06 AM
HI! Is it a 1000x card and have you tried it on 24p at 1080p? I,m about to buy one but it doesn,t seem to work that good?
I'm receiving two of those in a couple days, I'll post the results when I test them. I've seen a couple videos done with those and people said they worked well. In the other hand, I asked James Miller and he said it buffered fast in 1080p (unsure if 24p or 25p). Anyway, I'll let you know what happens.
Title: Re: uncompressed 14-bit RAW video recording
Post by: thiagokunz on May 22, 2013, 01:06:37 AM
People, is it possible to create a raw video without line skipping process? One of the things that is a down side of 5D filming is its shallow depth of field, achieved because os its large sensor size. If Magic Lanter could select only the central part of sensor (the one that receives better image quality from lenses) to create its images, this could be prevented when more depth of field is needed. It could also be used to change the focal length multiplier, transforming a wide lens into a more normal or tele lens.

Hi People... I'm so excited about the 5D raw that I signed in Magic Lantern Forum, even without knowing any code at all. You guys, in my humble opinion, are changing the history of cinema. (I'm sorry... is this the right place to talk about this topic?).
Title: Re: uncompressed 14-bit RAW video recording
Post by: Rush on May 22, 2013, 01:10:28 AM
Quote from: thiagokunz on May 22, 2013, 01:06:37 AM
People, is it possible to create a raw video without line skipping process.
it was available right from first implementations of RAW recording... in 5x/10x focus assist mode it will record w/o lineskipping: https://vimeo.com/66604210
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 22, 2013, 01:19:24 AM
Quote from: thiagokunz on May 22, 2013, 01:06:37 AM
If Magic Lanter could select only the central part of sensor (the one that receives better image quality from lenses) to create its images, this could be prevented when more depth of field is needed.

Unfortunately the Devs have not figure out how to sample the sensor at 1:1 in the center.  The default Canon data read via LV comes from the center Top Left Side.  We all hope that someday they will uncover how to get this info from the center of the sensor.

You can get more detailed info oh how to get 1:1 video here:

How to record: -- RAW Digital Cropped Video on 5D3 --
http://www.magiclantern.fm/forum/index.php?topic=5441.0
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 22, 2013, 01:22:18 AM
Quote from: Northernlight on May 21, 2013, 09:30:32 PM
The banding is so bad that it is not something you just get rid of using noise reduction. In a way I really hope it is something wrong with my camera, but I doubt it. Will post pictures later.

What on earth could be causing this banding?

http://en.wikipedia.org/wiki/Fixed-pattern_noise
This is how your sensor looks +3 stops @ 160 ISO. It's got nothing to do with ML except for the fact it's no longer denoised and compressed in camera.
(http://i.imgur.com/KlZc1Er.jpg)
Title: Re: uncompressed 14-bit RAW video recording
Post by: ajay on May 22, 2013, 02:08:49 AM
Quote from: Danne on May 22, 2013, 12:28:06 AM
HI! Is it a 1000x card and have you tried it on 24p at 1080p? I,m about to buy one but it doesn,t seem to work that good?

I have been using (2) 1000x 32GB Transcend cards and it will work solid 1080p 24fps. That's with everything disabled (no global draw). Also, if I go into crop mode I have to scale down to 1920 x 840 in order for it not to skip frames. Does anyone know if the Lexar can do full 1080p @ 3x?

Thanks,

AJ
Title: Re: uncompressed 14-bit RAW video recording
Post by: sergiocamara93 on May 22, 2013, 02:13:26 AM
Quote from: ajay on May 22, 2013, 02:08:49 AM
I have been using (2) 1000x 32GB Transcend cards and it will work solid 1080p 24fps. That's with everything disabled (no global draw). Also, if I go into crop mode I have to scale down to 1920 x 840 in order for it not to skip frames. Does anyone know if the Lexar can do full 1080p @ 3x?

Thanks,

AJ
If the card is able to record the bitrate in normal mode, the crop mode issue should be software related. I've seen a fix in the code related to the zoom and double buffering, I guess the next build will help with that.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Dignan21 on May 22, 2013, 02:23:17 AM
I apologize for the lame question. Will this module be available eventually and compatible with the new 1.2.1 Canon firmware. I, unfortunately upgraded and can't find the 1.1.3 firmware.
Title: Re: uncompressed 14-bit RAW video recording
Post by: hirethestache on May 22, 2013, 02:50:49 AM
Quote from: Dignan21 on May 22, 2013, 02:23:17 AM
I apologize for the lame question. Will this module be available eventually and compatible with the new 1.2.1 Canon firmware. I, unfortunately upgraded and can't find the 1.1.3 firmware.

I have the firmware and can upload it to my DB when I get a chance. But I am also in this situation, and have two 5D3's that are both on 1.2.1...any chance on a port?
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 22, 2013, 03:00:32 AM
The priority is to get a stable 1.1.3 release out first.
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 22, 2013, 03:22:33 AM
Quote from: Dignan21 on May 22, 2013, 02:23:17 AM
I, unfortunately upgraded and can't find the 1.1.3 firmware.

See below for the link to 1.13:


OTHER HELPFUL LINKS:

To help the DEVS and everybody it would be a good idea if specific RAW video issues are discussed in a separate thread based on the camera under discussion.

5D3  Uncompressed 14-bit RAW video testing - 5D Mark III
HERE:  http://www.magiclantern.fm/forum/index.php?topic=5473.0

5D2  Raw video on 5DMK2
HERE:  http://www.magiclantern.fm/forum/index.php?topic=5419.0

600D Uncompressed 600D Raw Video
HERE: http://www.magiclantern.fm/forum/index.php?topic=5494.0

6D 6D + Raw thread (with instillation guide)
HERE: http://www.magiclantern.fm/forum/index.php?topic=5530.msg37766#msg37766

650D I dont see a specific RAW video thread.  Should there be one??  Anyone care to propose so I can update this message?


--------------------
How to install Magic Lantern on 5D Mark III
HERE: http://www.magiclantern.fm/forum/index.php?topic=2602.0

Where to get Canon Firmware 1.1.3 for sure:
HERE:http://pel.hu/eoscard/

Where to get Compiled Nightly Builds?
HERE: http://www.magiclantern.fm/forum/index.php?topic=5413.0

All questions regarding RAW VIDEO POST PROCESSING on any OS
HERE: http://www.magiclantern.fm/forum/index.php?topic=5404.0

All questions regarding RAW_REC MODULE COMPILATION/INSTALLATION/USAGE
HERE: http://www.magiclantern.fm/forum/index.php?topic=5405.0

Card benchmark - what's the optimal write buffer size?
HERE: http://www.magiclantern.fm/forum/index.php?topic=5471.0

Supported 1000x CF cards for continuous RAW recording
HERE: http://www.magiclantern.fm/forum/index.php?topic=5391.0
Title: Re: uncompressed 14-bit RAW video recording
Post by: Colemar on May 22, 2013, 03:31:25 AM
Quote from: squig on May 22, 2013, 01:22:18 AM
http://en.wikipedia.org/wiki/Fixed-pattern_noise
This is how your sensor looks +3 stops @ 160 ISO. It's got nothing to do with ML except for the fact it's no longer denoised and compressed in camera.
(http://i.imgur.com/KlZc1Er.jpg)

Dan Hudgins http://www.dancad3d.com/ (http://www.dancad3d.com/) is currently working on eliminating this.  He's making great headway and, as with ML, it's only a matter of time.
Title: uncompressed 14-bit RAW video recording
Post by: platu on May 22, 2013, 03:56:30 AM
I've been testing the raw recording with my 5D3 via HDMI and my SmallHD DP4 LCD monitor.   The main issue I am experiencing (at least when using the SmallHD DP4) is the outline overlay which represents the selected resolution is being displayed at the wrong location. It appears too far to the left and also is placed too high.  Here are the screen captures when I enable Raw Recording...

This one is for 1920 x 1080
(http://i.imgur.com/Iuqohqo.png)

This is an example of 1920 x 840.  I enabled crop marks (cinemascope) just to show where the outline should be.
(http://i.imgur.com/zOpJWff.png)

If it is not possible to correct this, may I suggest adding the ability to disable the resolution overlay.  Since this is probably one the more popular monitors out there, I thought I would just let you know this issue exists.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 22, 2013, 03:59:48 AM
same also with my SWIT monitor. the guides are way off to the left. same as shown by Platu.
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 22, 2013, 04:10:13 AM
Quote from: ted ramasola on May 22, 2013, 03:59:48 AM
same also with my SWIT monitor. the guides are way off to the left.

I wasn't sure if this was a monitor specific issue... I guess not.   Can anyone else chime in if they also experience this when using an external LCD monitor via hdmi?
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 22, 2013, 04:25:28 AM
also, attempting to playback raw video with lcd monitor attached via hdmi hangs up the camera.
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 22, 2013, 04:31:40 AM
Quote from: ted ramasola on May 22, 2013, 04:25:28 AM
also, attempting to playback raw video with lcd monitor attached via hdmi hangs up the camera.

Yes... raw playback mode seems to always lock up the camera... end up having to removing the battery. 

Other than the misalignment of the resolution overlays and playback mode locking up the camera, everything else seems to work well when using an external monitor. 
Title: Re: uncompressed 14-bit RAW video recording
Post by: hirethestache on May 22, 2013, 05:13:42 AM
Quote from: platu on May 22, 2013, 03:56:30 AM
I've been testing the raw recording with my 5D3 via HDMI and my SmallHD DP4 LCD monitor.   The main issue I am experiencing (at least when using the SmallHD DP4) is the outline overlay which represents the selected resolution is being displayed at the wrong location. It appears too far to the left and also is placed too high.  Here are the screen captures when I enable Raw Recording...

This one is for 1920 x 1080
(http://i.imgur.com/Iuqohqo.png)

This is an example of 1920 x 840.  I enabled crop marks (cinemascope) just to show where the outline should be.
(http://i.imgur.com/zOpJWff.png)

If it is not possible to correct this, may I suggest adding the ability to disable the resolution overlay.  Since this is probably one the more popular monitors out there, I thought I would just let you know this issue exists.

I wasnt able to find the post anywhere, but I remember reading something about getting live preview of the RAW output in the LCD. With this idea, would it be possible to also get a live FEED from the HDMI, utilizing said raw preview? I'm probably misunderstanding this whole thing, regardless...But theres still a slight hope of utilizing the HDMI output to record RAW via xNAMEBRANDx field recorder (such as a Ninja).
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 22, 2013, 05:19:14 AM
Another RAW video  ;)

http://www.youtube.com/watch?feature=player_detailpage&v=rPKnehmFdh0 (http://www.youtube.com/watch?feature=player_detailpage&v=rPKnehmFdh0)
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 22, 2013, 05:27:37 AM
Quote from: hirethestache on May 22, 2013, 05:13:42 AM
I wasnt able to find the post anywhere, but I remember reading something about getting live preview of the RAW output in the LCD. With this idea, would it be possible to also get a live FEED from the HDMI, utilizing said raw preview? I'm probably misunderstanding this whole thing, regardless...But theres still a slight hope of utilizing the HDMI output to record RAW via xNAMEBRANDx field recorder (such as a Ninja).

This has been suggested many times.  Apparently, utilizing HDMI is difficult to do and the focus at the moment is to come up with a solution via the CF slot to external HDD or SSD.
Title: Re: uncompressed 14-bit RAW video recording
Post by: hirethestache on May 22, 2013, 05:44:18 AM
Quote from: platu on May 22, 2013, 05:27:37 AM
This has been suggested many times.  Apparently, utilizing HDMI is difficult to do and the focus at the moment is to come up with a solution via the CF slot to external HDD or SSD.

I concur that getting a stable release should definitely be the main focus, just curious.
Title: Re: uncompressed 14-bit RAW video recording
Post by: AndreasK on May 22, 2013, 08:15:21 AM
Unfortunately I can't get 24p working in full 1920x1080 mode on my 1000x Transcend 64gb. I posted some benchmarks in the other thread. If I read the already posted chart correctly the 32gb transcend seems to be faster. In 1920x1080 I have about 1 frame skipped every 1-2 seconds :( Just a tad too slow :(
Title: Re: uncompressed 14-bit RAW video recording
Post by: Audionut on May 22, 2013, 08:43:27 AM
Quote from: Colemar on May 22, 2013, 03:31:25 AM
Dan Hudgins http://www.dancad3d.com/ (http://www.dancad3d.com/) is currently working on eliminating this.  He's making great headway and, as with ML, it's only a matter of time.

You're talking about software denoising right?  That pattern noise is the noise floor of the camera (a hardware problem).
Title: Re: uncompressed 14-bit RAW video recording
Post by: aviel740 on May 22, 2013, 09:05:31 AM
Quote from: AndreasK on May 22, 2013, 08:15:21 AM
Unfortunately I can't get 24p working in full 1920x1080 mode on my 1000x Transcend 64gb. I posted some benchmarks in the other thread. If I read the already posted chart correctly the 32gb transcend seems to be faster. In 1920x1080 I have about 1 frame skipped every 1-2 seconds :( Just a tad too slow :(

Have you tried turning on Hak3d in the raw menu?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Nachelsoul on May 22, 2013, 09:18:22 AM
A 5DmkII Raw boring test with a 100 years old lens

Thanks to A1ex and developers for this incredible feature
Title: Re: uncompressed 14-bit RAW video recording
Post by: deleted.account on May 22, 2013, 09:27:32 AM
Quote from: Audionut on May 22, 2013, 08:43:27 AM
You're talking about software denoising right?  That pattern noise is the noise floor of the camera (a hardware problem).

Pattern noise removal would be dark frame subtraction?

Any noise reduction for raw video would be best done temporal and possibly with motion compensation.  So basic spatial and or dark frame in raw then temporal noise reduction, luma sharpening etc either in image sequence stage or during video edit / grading stage if chosen raw wrangling apps don't support temporal noise reduction.
Title: Re: uncompressed 14-bit RAW video recording
Post by: AndreasK on May 22, 2013, 09:44:12 AM
Quote from: aviel740 on May 22, 2013, 09:05:31 AM
Have you tried turning on Hak3d in the raw menu?

Hak3D?

--> when running benchmark from photomode i get depending on buffer ~100
--> in videomode i get 85-95mb/s depending on buffer, 95 would be enough but I don't know what kind of buffer is used
Title: Re: uncompressed 14-bit RAW video recording
Post by: 3pointedit on May 22, 2013, 09:45:35 AM
y3llow, why would you need to do temporal NR on a RAW sequence? There is no inter-frame referal of data?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Mei Lewis on May 22, 2013, 09:46:59 AM
Quote from: squig on May 22, 2013, 01:22:18 AM
http://en.wikipedia.org/wiki/Fixed-pattern_noise
This is how your sensor looks +3 stops @ 160 ISO. It's got nothing to do with ML except for the fact it's no longer denoised and compressed in camera.
(http://i.imgur.com/KlZc1Er.jpg)

I don't think that's a major problem, why would you ever want to push a nearly black image by 3 stops???
Would be good to get rid of it anyway though!



I see you're shooting at ISO 160, why?

The idea of shooting in multiples of 160 is only relevant for standard compressed video shooting, due to the way intermediate ISOs are processed in camera. Eg 160 is a 1/3 stop pull from ISO 200 sensor data before writing to h264.

The 'real' ISOs of the mk3, as evident from analysis of RAW stills , are:
100, 200, 400, 800, 1600, 3200 and 6400. Maybe also 12800.

Going forward for maximum RAW quality these are the multiples we should be shooting in a think
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 22, 2013, 09:53:40 AM
My brain hasn't made the raw adjustment yet.... LAG!

I pushed it 3 stops just so people can see it. I do the same with every other cameras for comparison.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Audionut on May 22, 2013, 10:03:18 AM
Quote from: y3llow on May 22, 2013, 09:27:32 AM
Pattern noise removal would be dark frame subtraction?

If it's fixed pattern noise, yes.

I was incorrect before when talking about pattern noise.  I was talking about the read noise (amplifier noise).

Quote from: squig on May 22, 2013, 09:53:40 AM
I pushed it 3 stops just so people can see it. I do the same with every other cameras for comparison.

It's good to see what the camera is doing to the shadows, but ISO 160 is not the best test as it's been reduced 1/3rd of a stop in camera.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 22, 2013, 10:15:34 AM
Quote from: Audionut on May 22, 2013, 10:03:18 AM

It's good to see what the camera is doing to the shadows, but ISO 160 is not the best test as it's been reduced 1/3rd of a stop in camera.

Yeah I'll do some 400 and 1600 ISO tests shortly.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 22, 2013, 10:52:33 AM
Komputerbay 64GB 1000x card was working fine earlier...( 5D MARK III) tried hack 3d to see what i could push it to. Then went back to 1080p it wouldnt even run that (normally does easily). then took it out of the camera and put it into the computer - no response - put it back in the camera - camera couldnt read it or format it..

what do i do next?

card was formatted to exfat but i didnt record over 4gb files on it yet. any ideas? thank
Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 22, 2013, 11:05:00 AM
Quote from: Habitat on May 22, 2013, 10:52:33 AM
Komputerbay 64GB 1000x card was working fine earlier...( 5D MARK III) tried hack 3d to see what i could push it to. Then went back to 1080p it wouldnt even run that (normally does easily). then took it out of the camera and put it into the computer - no response - put it back in the camera - camera couldnt read it or format it..

what do i do next?

card was formatted to exfat but i didnt record over 4gb files on it yet. any ideas? thank

Took it downstairs to the imac used a different card reader the Imac read it. Disk Utility for some reason thought it was 613GB but Finder read it properly I deleted the raw file manually put it back in the cam - it ran perfectly with no issues at 1080p...anomaly i guess.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Colemar on May 22, 2013, 11:05:24 AM
Quote from: Audionut on May 22, 2013, 08:43:27 AM
You're talking about software denoising right?  That pattern noise is the noise floor of the camera (a hardware problem).

that's correct, software noise removal specific to the 5DmkIII's sensor and any others that can be extrapolated from that data
Title: Re: uncompressed 14-bit RAW video recording
Post by: AndreasK on May 22, 2013, 11:28:15 AM
Ah found the hack3d.

With that enabled in 1920x1080 it skipps a frame about every 2 seconds
Title: Re: uncompressed 14-bit RAW video recording
Post by: Lukasz on May 22, 2013, 12:23:01 PM
Hi, wanted to add my Lexar 16gb 400x RAW test put out off 134 frame clips in 1880x720 which I am able to do. Continuosly 1880x540, only on first builds.

Title: Re: uncompressed 14-bit RAW video recording
Post by: Griffleiste on May 22, 2013, 02:47:42 PM
've been Magic Lantern user since the beginning on the 5D II.
Battery drain with the 20.05.13 version is huge, compared to normal video recording.
Having a battery grip connected and fully loaded, I experienced a drop from 50% charging level to 11% charging level while only leaving the camera on for about 15 minutes (mainly to check out menues etc.), while about 6 minutes of the 15 minutes were raw recording.
Powering the camera off at 11% and leaving it alone for the night, I experienced on the next morning that even with the power switch's pointing to off the two batteries in the grip have been discharged completely.

I don't want to be repetitive, I just wanted to give you guys the feedback you need for maxing out ML's possibilities.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 22, 2013, 02:59:56 PM
Dunno whats happening with 5DII... 600D (similar generation) doesn't do this... I have 2 batteries in the grip and they keep going and going.

Heh... since aewb left on I had AWB change on me during a shot.. yellow taxi cab drove by and suddenly everything is blue... maybe I should try to just shoot with UNIWB since everything is raw now.
Title: Re: uncompressed 14-bit RAW video recording
Post by: coutts on May 22, 2013, 03:02:00 PM
Found required edmac stubs in 40d:
https://bitbucket.org/hudson/magic-lantern/commits/cf080443d8ffcbf3cfc21c06aa3bb95d2e341ca9

I don't think TCC or Modules work yet with the 40d so someone will have to port raw_rec.c into the main code, but looks possible (alex says it has lv_save_raw too). Amazing that even a 6 year old VxWorks camera possesses the ability to shoot raw video.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 22, 2013, 03:07:26 PM
@coutts - That's amazing. To my eyes the 40d beats the 50d for photos even though it's 10mp as opposed to 15mp. It will be interesting to see what it can do with raw video if it's possible and they are amazingly cheap now >200 euros

I wonder what the LV frame size is?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 22, 2013, 03:27:51 PM
Lower sensor density = better I guess.

UNiWB is working perfectly and write seemed to speed up. No more need of push WB or autoWB,etc. With the auto ETTR just need M mode. There should be some iso limit tho... I had aperture turned down from shooting outside and it pushed me up to 256K.
Title: uncompressed 14-bit RAW video recording
Post by: swinxx on May 22, 2013, 04:54:50 PM
Strange.
I tried a 64gb komputerbay card with my 5d mk3. Had 97mb/s in the benchmark with a buffer 27mb. Great result. Then i recorded with 1980x1152 - great! No skipped frames.
Then i played a little bit with the different menus.
Suddenly write speed was only 65mb/s???
So i recopied the 22may build on the card. But no change.. I have disabled nearly everything except raw..
What can i do now..? I can only record 17 frames.??
Please help.. Thx
Title: Re: uncompressed 14-bit RAW video recording
Post by: haysuess on May 22, 2013, 05:11:27 PM
Quote from: swinxx on May 22, 2013, 04:54:50 PM
Strange.
I tried a 64gb komputerbay card with my 5d mk3. Had 97mb/s in the benchmark with a buffer 27mb. Great result. Then i recorded with 1980x1152 - great! No skipped frames.
Then i played a little bit with the different menus.
Suddenly write speed was only 65mb/s???
So i recopied the 22may build on the card. But no change.. I have disabled nearly everything except raw..
What can i do now..? I can only record 17 frames.??
Please help.. Thx

This sounds similar to my situation. My benchmarks used to be 45-55MB/s write speed, all of a sudden, they are now ~30MB/s no matter what build I use. I'm using a 400x Transcend 32GB on a 5D Mark II. Global draw off. I can't even do 1280x720 at 24fps for more than 150 frames.
Title: Re: uncompressed 14-bit RAW video recording
Post by: weigertj on May 22, 2013, 05:17:47 PM
I created a 5D MK III raw video group on Vimeo. There are already 120 videos in the list and always updating. Please feel free to join:

https://vimeo.com/groups/192344 (https://vimeo.com/groups/192344)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Samir on May 22, 2013, 06:58:43 PM
Hi Team. First off thanks to the Magic Lantern team for breathing new life into our cameras.

Now a question. I've noticed a 2 pixel black border to the left of all of my RAW frames. Is there an option around this. The max my CF card will record is without sound 1920x1080p. I tried the 1928x1080p with the intention of cutting it out in post but that option only gives me a few frames before it stops recording. Am I doing something wrong or is that border there until this is out of Alpha?
Title: Re: uncompressed 14-bit RAW video recording
Post by: mjneubrander on May 22, 2013, 07:00:59 PM
haysuess I was having the same issue. Take a look back at
Quote from: mjneubrander on May 21, 2013, 09:24:12 AM
I was also able clear the 145 frame issue using the files from below. And starting a zoom raw rec from any C1, C2 ,C3 modes but not the M mode.
No clue what causes this, but this voodoo process cleared it for me.
Made a ML card using the files in that post.
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 22, 2013, 07:03:00 PM
Quote from: haysuess on May 22, 2013, 05:11:27 PM
This sounds similar to my situation. My benchmarks used to be 45-55MB/s write speed, all of a sudden, they are now ~30MB/s no matter what build I use. I'm using a 400x Transcend 32GB on a 5D Mark II. Global draw off. I can't even do 1280x720 at 24fps for more than 150 frames.

I have noticed a slight decrease in card performance in 5D3 with May 22nd builds.  On all cards but not as significant as reported above.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 5DanielMIII on May 22, 2013, 07:11:27 PM
I just tested the Sandisk Extreme Pro 32 gb rated at 90m/s

*A quick CrystalDiskMark test gave me a number of 92 Read and 85 write speed
*In ML 1920 x 1080 Raw record it won't go higer than 82-83.

Then I tried the Lexar 64gb 1000 x rated at "up to 150" read and "up to 145" write.


*Same CrystalDiskMark test gave 120 read and 103 write.
*In ML 1920 x 1080 Raw record it is the same at 82-83.
But when trying to do 1920 x 1280 I still only get 82-83..

Sugestions?
Title: Re: uncompressed 14-bit RAW video recording
Post by: deleted.account on May 22, 2013, 07:13:01 PM
Quote from: 3pointedit on May 22, 2013, 09:45:35 AM
y3llow, why would you need to do temporal NR on a RAW sequence? There is no inter-frame referal of data?

By "no inter-frame referral of data' you mean because it's not "video"? Every frame refers to each other as it's in motion, we view it in motion? :-)

By temporal NR I mean that the NR tool does motion analysis, creates vectors and the noise reduction algo's check frames backwards and forwards, maybe just a couple each way, maybe 10 or 50 each way depending on processing power.

The algos attempt to establish what over time it assumes to be detail, noise, high frequency jitter and shimmer and sets about reducing it based on users set parameters and the movement or lack of movement in the noise.

Noise reduction in singular image processing is a subjective and visual process, we adjust the sliders, we decide on the parameters for a single image, until we're happy with it, at the point that a balance is struck between noise and detail.

But at 24fps we're going to need something a bit more than profiling the camera and creating a dark frame.

I think noise reduction is best done in a number of small increments targeting specific problems whether it be fixed noise, shimmer, flicker whatever. And then the adding of controlled levels of noise back in to avoid banding.

Really don't see how anything other than a temporal approach would be beneficial.
Title: Re: uncompressed 14-bit RAW video recording
Post by: IliasG on May 22, 2013, 07:52:19 PM
Quote from: y3llow on May 22, 2013, 09:27:32 AM
Pattern noise removal would be dark frame subtraction?

Any noise reduction for raw video would be best done temporal and possibly with motion compensation.  So basic spatial and or dark frame in raw then temporal noise reduction, luma sharpening etc either in image sequence stage or during video edit / grading stage if chosen raw wrangling apps don't support temporal noise reduction.

I think the first that can help to fight any "Channel imbalance" (which is the the main reason for this banding or pattern noise) is measuring the Black Levels per Channel and write them at the related exif tag.  Then use this info to normalize ...
Title: Re: uncompressed 14-bit RAW video recording
Post by: sergiocamara93 on May 22, 2013, 07:53:43 PM
Neat Video works great with 5D3 RAW (also with H.264), it provides temporal NR and keeps detail pretty well. Though I would only use it if you have a lot of fixed pattern or visible color noise, the regular RAW grain can be taken care of with ACR's luminance noise slider.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Limiter on May 22, 2013, 08:09:24 PM
Quote from: weigertj on May 22, 2013, 05:17:47 PM
I created a 5D MK III raw video group on Vimeo. There are already 120 videos in the list and always updating. Please feel free to join:

https://vimeo.com/groups/192344 (https://vimeo.com/groups/192344)

Hey weigertj,

thx for creating this group, just browsed though some of the videos... Unbelieveable!!

Magic Lantern has done a BIG thing here.. OMG  :o

Time to sell my 5D 2 and get a M3 instead..  :P (+ a fast CF card)

I'm hoping for a ML release where 1080p25 will be possible stable and with user friendly menu etc..

I have also big hopes for the CF->SDD project, keep it going guys!  8)

Just had to get this off my chest  ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: haysuess on May 22, 2013, 10:19:31 PM
Quote from: mjneubrander on May 22, 2013, 07:00:59 PM
haysuess I was having the same issue. Take a look back at No clue what causes this, but this voodoo process cleared it for me.
Made a ML card using the files in that post.

Looks like that post is for 5D Mark III, I have a Mark II. I will try other modes besides M though and see if it makes a difference.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Dunc101 on May 22, 2013, 10:58:21 PM
Really nice comparison video for RAW and H264: https://vimeo.com/groups/192344/videos/66738132

Shows you why you need Raw!!!
Title: Re: uncompressed 14-bit RAW video recording
Post by: bronson on May 23, 2013, 12:20:44 AM
5D Mark II / Raw Video : Test 1000mm with x5 Liveview



Canon 5d Mark 2
Sigma 150-500mm
Canon Extender EF X2

Magic lantern (Build May 22th)
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 23, 2013, 02:06:44 AM
Fixed pattern noise thread http://www.magiclantern.fm/forum/index.php?topic=5490.0
Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 23, 2013, 02:21:17 AM
Will we ever be able to consolidate the audio with the clips (maybe within the raw file) or is that impossible?
Title: Re: uncompressed 14-bit RAW video recording
Post by: carmstrong on May 23, 2013, 02:26:54 AM
I have the may 19 alpha build on my 5d3 - works great. The only thing I am trying to figure out is why it will not let me shoot larger than 1928 w, 1288 h -

If I try and make the width larger, ex: 3592 - the dialogue says "can't do 3592"

Anyone know what Im missing?!
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 23, 2013, 02:48:16 AM
it's a safety measure to prevent your camera from exploding  :D
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 23, 2013, 03:03:26 AM
Try zoom mode or call canon and tell them to make a 2k+ image in regular mode.
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 23, 2013, 03:42:36 AM
Quote from: carmstrong on May 23, 2013, 02:26:54 AM
I have the may 19 alpha build on my 5d3 - works great. The only thing I am trying to figure out is why it will not let me shoot larger than 1928 w, 1288 h -

If I try and make the width larger, ex: 3592 - the dialogue says "can't do 3592"

Anyone know what Im missing?!

You can get access to those resolutions in cropped mode. 
Some instructions here:  http://www.magiclantern.fm/forum/index.php?topic=5441.0
Title: Re: uncompressed 14-bit RAW video recording
Post by: vstrglv on May 23, 2013, 07:04:45 AM
Quote from: bronson on May 23, 2013, 12:20:44 AM
5D Mark II / Raw Video : Test 1000mm with x5 Liveview

Canon 5d Mark 2
Sigma 150-500mm
Canon Extender EF X2

Magic lantern (Build May 22th)
Do you get 1920x1078@25 for full and 1880x840@25 for 5x zoom on 5D2 without drop frames?
Title: Re: uncompressed 14-bit RAW video recording
Post by: MrEdmondDantes on May 23, 2013, 08:58:43 AM
Long time since my last post, but I've been experimenting with all this RAW stuff and it's obviously incredible.  I shot about 30 gigs of stuff today, so I'm gonna spend the next couple hours seeing how beautiful it is tonight.  I've been pretty lucky in the 4 or 5 builds I've installed.  No real problems - other than a few hiccups people have mentioned.   I have the May 22nd build on there now, and here are a few things I've noticed (that don't bother me, but might be useful to know):

1.  When hitting the "Magnify" button, the screen turns pink and temporarily freezes 95% of the time.  Only once or twice did it not turn pink and noisy for a few seconds.  It almost always clears up after about 1.5 - 2 seconds.  This does not happen when going from 5x --> 10x, or from 10x --> 1X.   Only from the initial first zoom. 

2.  Occasionally, after the pink "jam" screen vanishes, there will be vertical green/pink lines left over that take about 3 or 4 seconds to slowly fade away - always near the center/right of the frame.   It's never frozen my camera or caused a problem.  It does slightly worry me, but I'm sure it's fine.

3.  As others have mentioned, the sound is disabled when you turn off raw mode and try to record h.264 video.  I think this might be a feature though, and I apologize for not researching more.  I've not searched the forums for a fix because it doesn't affect me at all.  I wasn't shooting audio today so I didn't care. 

Besides that, everything has been going great.  This is really a game changer.  I have a SanDisk 60MB/sec, a G-Monster 50MB/sec, and a 64GB 1000x Komputer Bay that I lent to a friend who wanted to install this before me that I'm getting back Friday.  I'll be doing a lot of tests with that bad boy over the weekend - I'll post back here when I finish. 
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 23, 2013, 09:19:59 AM
Quote from: MrEdmondDantes on May 23, 2013, 08:58:43 AM
3.  As others have mentioned, the sound is disabled when you turn off raw mode and try to record h.264 video.

I just noticed this as well.  Was working in previous builds.
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 23, 2013, 09:38:31 AM
Sound may be disabled by FPS override, check that.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Northernlight on May 23, 2013, 09:46:29 AM
Quote from: Northernlight on May 21, 2013, 09:30:32 PM
RAW video sure is exciting, partcularly for me making my living in the darkness! I have installed it on my 5D3, and it records really well @24fps using my 128GB Lexar Pro 1000x UDMA7 card.

BUT, the vertical banding on the stills captured with my camera worries me. In my opinion it is so bad, that it renders the whole feature useless!

The banding is so bad that it is not something you just get rid of using noise reduction. In a way I really hope it is something wrong with my camera, but I doubt it. Will post pictures later.

What on earth could be causing this banding?



Here are a few screenshots.  Overexposing does not help at all. I have lifted shadows and decreased highlights (which is the whole point of having bigger lattitude in DR), and I have added +20 clarity.

50% resolution full frame:
(http://www2.arcticlightphoto.no/band_540.jpg)

100% partial crop:

(http://www2.arcticlightphoto.no/band_crop.jpg)

This is REALLY REALLY bad !! :(
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 23, 2013, 09:49:05 AM
@Northernlight:
please post that in the thread for this problem:
Quote from: squig on May 23, 2013, 02:06:44 AM
Fixed pattern noise thread http://www.magiclantern.fm/forum/index.php?topic=5490.0
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 23, 2013, 10:17:04 AM
Quote from: a1ex on May 23, 2013, 09:38:31 AM
Sound may be disabled by FPS override, check that.

I had already checked that.  There are 2 repeatable scenarios which cause audio issues in H.264 video recording...

FPS overide is NOT enabled in either scenario...

Scenario 1
- Enable raw video mode with "Sync beep" option.
- Record raw video
- Disable raw video option
- Record H264 video... audio does not get recorded

Scenario 2
- Enable raw video mode with "Separate WAV" option.
- Record raw video
- Disable raw video option
- Record H264 video... audio gets recorded but is distorted (sped up like chimpmunk)
- Record H264 video again... audio gets recorded normally from now on

I hope this helps.
Title: Re: uncompressed 14-bit RAW video recording
Post by: dude on May 23, 2013, 11:05:26 AM
I noticed following (19th built):
Pressing 5x causes magenta stripes through the picture(on the display), 10x does not. But this only shows up like every 2nd time i use it.
And when i am recording raw, and disable it, the white (framing) square still is on the screen in liveview.
For raw processing, it s great if you already did photo raw processing with acr, pretty much the same and absolutely awesome!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 23, 2013, 04:28:43 PM
Quote from: platu on May 23, 2013, 10:17:04 AM
I had already checked that.  There are 2 repeatable scenarios which cause audio issues in H.264 video recording...

FPS overide is NOT enabled in either scenario...

Scenario 1
- Enable raw video mode with "Sync beep" option.
- Record raw video
- Disable raw video option
- Record H264 video... audio does not get recorded

Scenario 2
- Enable raw video mode with "Separate WAV" option.
- Record raw video
- Disable raw video option
- Record H264 video... audio gets recorded but is distorted (sped up like chimpmunk)
- Record H264 video again... audio gets recorded normally from now on

I hope this helps.

I've just been removing the CF with ML on it and shooting H264 on my SD card for now.  When I don't want to do amazing raw video, that is.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 23, 2013, 04:36:13 PM
QuoteRecord H264 video... audio gets recorded but is distorted (sped up like chimpmunk)

On 5d3? 5d3 is using a couple of direct asif continue functions... on 6D audio using those was sped up too. Shouldn't be affecting H264 audio tho.
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 23, 2013, 04:54:28 PM
Quote from: platu on May 23, 2013, 10:17:04 AM
- Record H264 video... audio gets recorded but is distorted (sped up like chimpmunk)

Sounds like a sample rate issue. Is the sample rate of your project the same as the sample rate recorded at?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 23, 2013, 04:56:19 PM
Remember that 1 build where playback was all fast? its like that.. if the recording continue is direct too then its going to be sped up and unfix-able.
Title: Re: uncompressed 14-bit RAW video recording
Post by: haysuess on May 23, 2013, 05:23:44 PM
This was posted in the "Raw video on 5DMK2" thread, but I thought it was well worth mentioning in case it hasn't been seen by all.

If you have RAW selected as your photo type and record, say you get 100 frames at any given resolution and fps.

If you then change your photo settings to disable RAW and shoot only Small JPG, you'll get MANY more frames of RAW video.

It seems like there was 2-3 more *'s worth of buffer when I did this.

Benchmarks are the same, so it appears it just opens up some more buffer room.

In my tests, I went from 150-170 frames to 290 just like that! :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 23, 2013, 06:19:33 PM
Quote from: haysuess on May 23, 2013, 05:23:44 PM
This was posted in the "Raw video on 5DMK2" thread, but I thought it was well worth mentioning in case it hasn't been seen by all.

If you have RAW selected as your photo type and record, say you get 100 frames at any given resolution and fps.

If you then change your photo settings to disable RAW and shoot only Small JPG, you'll get MANY more frames of RAW video.

It seems like there was 2-3 more *'s worth of buffer when I did this.

Benchmarks are the same, so it appears it just opens up some more buffer room.

In my tests, I went from 150-170 frames to 290 just like that! :)

I have a Sandisk 16GB 600x card and leave my photo setting with RAW only. When I record raw video, my buffer doesn't go beyond 1 star. Mind you, this is with only a 600x, so I don't think disabling raw images would make it any better.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 23, 2013, 06:22:31 PM
Just look at shoot_malloc size to see if its doing anything. Sraw gives more memory on 600D, Ljpeg works on EOSM. Haven't found any way to get more memory on 6D.. only less.

Quotemy buffer doesn't go beyond 1 star.

This would not affect write speed.
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 23, 2013, 06:55:30 PM
Quote from: 1% on May 23, 2013, 06:22:31 PM
Just look at shoot_malloc size to see if its doing anything. Sraw gives more memory on 600D, Ljpeg works on EOSM. Haven't found any way to get more memory on 6D.. only less.

This would not affect write speed.
I thought the buffer gave an indication of how well your card was keeping up with the raw recording???

When you say sraw gives more recording, do you mean it allows more frames per space, or it allows the same amount of frames(and quality) but with less strain on the card?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Picture and Color on May 23, 2013, 06:59:33 PM
My latest! Comparing H264 Technicolor Cinestyle to ML Raw. Highlight recovery, curves tweaked slightly in Adobe Camera Raw. No other correction applied, and no post sharpening applied. Didn't touch cinestyle H264 footage at all because frankly, it's no longer pleasurable to tweak compressed footage. Camera settings were identical for each set of shots.

Shot on May 22nd build, on 128GB Komputerbay 1000x.

My write speeds have gotten slower with the May 22nd. I can no longer hit 70MB peak at all and struggle to maintain 64MB/sec needed for 1920 x 840. Most clips have at least one dropped frame. I use CF workaround which seems to help. Global draw fully off.

Vimeo is set to 1080p, so please give it a second to load and then hit fullscreen. The parrot shots and the clocktower shots are my favorites in terms of showing off the power of the extended DR and color corrective capabilities.

Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 23, 2013, 07:00:15 PM
Your card writes at X speed depending on frame size and other unknowns. The memory you have for the buffer changes from this so you can store up lets say 100 pics vs 60 pics, etc.

Ultimately disabling photo mode when in video mode will gain back more and then you'll be able to store extra frames before it stops/skips/dies, etc.

Lol, my write speeds are the fastest ever.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Picture and Color on May 23, 2013, 07:14:19 PM
I just tested this on the Mark III, and it seems to make a difference in my May 22nd build. I've never really past 70MB/sec peak on my 128GB Komputerbay (in actual recording) but now I'm seeing 72.5 MB/sec sustained in a test clip. CF card workaround on, global draw off, ex-Fat formatted card.


Quote from: haysuess on May 23, 2013, 05:23:44 PM
This was posted in the "Raw video on 5DMK2" thread, but I thought it was well worth mentioning in case it hasn't been seen by all.

If you have RAW selected as your photo type and record, say you get 100 frames at any given resolution and fps.

If you then change your photo settings to disable RAW and shoot only Small JPG, you'll get MANY more frames of RAW video.

It seems like there was 2-3 more *'s worth of buffer when I did this.

Benchmarks are the same, so it appears it just opens up some more buffer room.

In my tests, I went from 150-170 frames to 290 just like that! :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 23, 2013, 07:58:14 PM
Ok, I have the Sandisk 600x 16gb CF. haven't had any issues recording. I can record until I hit stop without any dropped frames. I tried putting my image setting to SRaw and got a dropped frame in the first second of recording. I guess it has the opposite effect on me.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 23, 2013, 08:00:08 PM
Yep... look in the memory menu... on 6D sraw = less.
Title: Re: uncompressed 14-bit RAW video recording
Post by: hini on May 23, 2013, 08:17:20 PM
Quote from: Northernlight on May 23, 2013, 09:46:29 AM
Here are a few screenshots.  Overexposing does not help at all.

100% partial crop:

(http://www2.arcticlightphoto.no/band_crop.jpg)

This is REALLY REALLY bad !! :(

In my opinion this has nothing to do with ML.
And it is not the kind of noise you could fix with noise reduction.
I think your sensor needs a mechanical calibration.
Canon service can do that for you.
If you don´t have money for service you could try adding some noise by shooting with higher iso.
hini
Title: Re: uncompressed 14-bit RAW video recording
Post by: stevefal on May 23, 2013, 08:28:12 PM
Quote from: Picture and Color on May 23, 2013, 06:59:33 PM
My latest! Comparing H264 Technicolor Cinestyle to ML Raw. Highlight recovery, curves tweaked slightly in Adobe Camera Raw. No other correction applied, and no post sharpening applied. Didn't touch cinestyle H264 footage at all because frankly, it's no longer pleasurable to tweak compressed footage. Camera settings were identical for each set of shots.

Shot on May 22nd build, on 128GB Komputerbay 1000x.

My write speeds have gotten slower with the May 22nd. I can no longer hit 70MB peak at all and struggle to maintain 64MB/sec needed for 1920 x 840. Most clips have at least one dropped frame. I use CF workaround which seems to help. Global draw fully off.

Vimeo is set to 1080p, so please give it a second to load and then hit fullscreen. The parrot shots and the clocktower shots are my favorites in terms of showing off the power of the extended DR and color corrective capabilities.



Notice hot pixels flaring on her zipper below the pendant at ~1:03.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 23, 2013, 08:46:07 PM
Quote from: haysuess on May 23, 2013, 05:23:44 PM
This was posted in the "Raw video on 5DMK2" thread, but I thought it was well worth mentioning in case it hasn't been seen by all.

If you have RAW selected as your photo type and record, say you get 100 frames at any given resolution and fps.

If you then change your photo settings to disable RAW and shoot only Small JPG, you'll get MANY more frames of RAW video.

It seems like there was 2-3 more *'s worth of buffer when I did this.

Benchmarks are the same, so it appears it just opens up some more buffer room.

In my tests, I went from 150-170 frames to 290 just like that! :)

So the resultant dng files are the same size too?  Have you tried jpg small?  I'm about to.
Title: Re: uncompressed 14-bit RAW video recording
Post by: haysuess on May 23, 2013, 08:50:39 PM
Quote from: Shield on May 23, 2013, 08:46:07 PM
So the resultant dng files are the same size too?  Have you tried jpg small?  I'm about to.

Yes, it has 0 effect on the RAW recording, other than I believe freeing up more buffer memory, which gives you more frames. Granted, I don't think this helps you if you are already recording without running out of buffer room, but for those of us with slower cards, it can give another few seconds at least, which is a huge difference when I was only get 150 frames to start with.

Now for 10/12-bit to be implemented and my Komputerbay 64GB card to arrive >: )
Title: Re: uncompressed 14-bit RAW video recording
Post by: nahua on May 23, 2013, 10:13:32 PM
Music video for the teen group EMKE shot over the weekend.  Using May 17 build on a Canon 5D mkIII.  No sound yet, so I did all syncing by hand.  That plus the current RAW workflow meant I spent a TON of time doing this.  But the image is great, there's no doubt about it.  I look forward to shooting more with the new builds.  Thank you again to the Magic Lantern Team!  Please check out the video and support your local teen music groups!

https://vimeo.com/66681135
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 23, 2013, 11:55:07 PM
Quote from: nahua on May 23, 2013, 10:13:32 PM
Music video for the teen group EMKE shot over the weekend.  Using May 17 build on a Canon 5D mkIII.  No sound yet, so I did all syncing by hand.  That plus the current RAW workflow meant I spent a TON of time doing this.  But the image is great, there's no doubt about it.  I look forward to shooting more with the new builds.  Thank you again to the Magic Lantern Team!  Please check out the video and support your local teen music groups!

https://vimeo.com/66681135

This doesn't go here dude. Please post under Share Your Video's (http://www.magiclantern.fm/forum/index.php?board=19.0).
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 24, 2013, 02:01:59 AM
Audio synchronisation with RAW? :'(

First of all I am so so sorry if this is posted in the wrong place. I am not sure where I should post this exactly? I just know that there are good people here that may possibly point me in the right direction. Again I apologise in advance.

I have now had several experiments trying to synchronise audio with the RAW video in post.

I am used to using the traditional method of recording audio on a external sound recorder (ZoomH2) and using a clapper board to get an audio/visual sync.

I have used this technique to get greatly improved sound when compared to the 5DMKII's inbuilt mic.

I have found it to be pretty reliable and with a visual cue (the clapper board) its pretty easy to align them in most decent NLE's

However I have tried aligning the separate audio with the RAW video and there is something that's not quite right.

A few days ago I just recorded some video with separate audio using clapper board and aligned them but in the video there was nothing else for me to see if the sync was staying in sync. I just made the assumption that everything was fine. Although I could never be sure exactly how good the sync was it just looked ok because all the sounds were happening off screen.

Anyway today I decide to test a little more thoroughly. I just set up a simple shot where I would use the clapper and then I would tap a little beat on a surface and obviously keep my fingers in the frame (so I could see if it was all in sync). I recorded a few takes of the same kind of thing with both RAW video being recorded and my ZoomH2 recording the sound.

I got them on the computer processed the RAW files and import them into After Effects. I then dropped the audio into After Effects and went in to align them. I aligned the sound of the clapper at the frame where the clapper shuts. This is something I have done numerous times before (with the old H264 video) and would usually be confident at this stage, that when I hit play everything will be perfectly (or very nearly perfectly) in sync.

Well this is not so. I am not sure whats happening. Where I have aligned the clap it's perfect but it seems that it must be drifting.

My only conclusion is that there is some mismatch somewhere in the timing of the two. The problem is where would that be?

One thing I would really like to know is that should the 23.976fps be processed as 24fps? (I have tried both and they both drift somehow).

In fact this one has really got me because I can't seem to get the sync whatever I do?

I am wondering maybe if I will have to do a slight pitch shift on the audio or maybe some kind of re-conform of the video. I don't want the additional processing involved but if I could somehow get some workflow that 'worked' then maybe that would help?

I must say at this stage I am lost with this one.

I was wondering if anyone else had any experience of audio synchronisation with the RAW footage?

Any help would be greatly appreciated.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 24, 2013, 02:14:50 AM
I'll do an audio sync test on the MK3 over the weekend. What build are you running on the MK2? Do you have another app like FCP that you can drop the files into just to rule out whether it's an AE problem. I'm having some issues with exposure flickering in AE and I just don't trust it.

I found this-

"The Zoom H2 seems to record audio at a very slightly different speed than most video recorders. The real problem is that non-professional audio recorders record at a sample rate of 44.1kHz. The audio quality is just fine -- CDs run at 44.1kHz -- but professional video cameras record audio at 48kHz (broadcast standard). That slight difference is enough to cause about a one- to two-frame (at NTSC video's 29.97fps frame rate) offset every minute or so. However, if you run these recorders at 48kHz, that will sync up with video from most cameras -- provided they are also running at 48kHz."
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 24, 2013, 02:46:00 AM
In camera audio is 48k
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 24, 2013, 03:16:15 AM
Quote from: squig on May 24, 2013, 02:14:50 AM
I'll do an audio sync test on the MK3 over the weekend. What build are you running on the MK2? Do you have another app like FCP that you can drop the files into just to rule out whether it's an AE problem. I'm having some issues with exposure flickering in AE and I just don't trust it.

I found this-

"The Zoom H2 seems to record audio at a very slightly different speed than most video recorders. The real problem is that non-professional audio recorders record at a sample rate of 44.1kHz. The audio quality is just fine -- CDs run at 44.1kHz -- but professional video cameras record audio at 48kHz (broadcast standard). That slight difference is enough to cause about a one- to two-frame (at NTSC video's 29.97fps frame rate) offset every minute or so. However, if you run these recorders at 48kHz, that will sync up with video from most cameras -- provided they are also running at 48kHz."

Thanks for the response

I am thinking that this kind of problem will become quite common unless there is a distinct strategy to deal with it. I have been into audio production for some years now and am reasonably aware of sampling frequencies and what is used for what. I also understand about the mismatch (and assosciated problems) that could happen if the speed of one is not consistent with another.

Anyway since I got the ZoomH2 I have always used the frequency of 48khz 16bit. I chose this setting for two good reasons. One was for quality and the other was for compatibility. As I already said I had no problems synchronising the H264 video with audio from the H2. It's an enigma. I am not trying to fault the RAW video (to me its a dream come true) but I want to be able to work around possible problems. I want to develop a workflow that works and I am sure plenty more people will also want that too.

I think I may try using other programs (such as Premiere and FCP) and see if they produce different results. I am open on this one and if I find a way that works then I will happily tell the world.

Anyway thanks again to have responded so soon and presented some ideas. :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 24, 2013, 03:17:19 AM
Build is May22nd BTW  :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 24, 2013, 03:29:05 AM
I was a record producer before I became a filmmaker, when it comes to audio sync I think in milliseconds. What did you have the FPS override set to? I'm assuming the video is perfectly smooth, no dropped frames or cadence issues. How much is the sync off by (time/frames)?
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 24, 2013, 04:27:14 AM
Yes the video is perfectly smooth. No problems with that.

I have the fps override turned OFF. If I go into the fps override and turn it on it just reads 2.502 and will record at that rate. I don't seem to be able to change the fps override (is there some way of changing this that I have maybe missed?).

My belief is that it's recording at 29.976 (that's what it says at the bottom when the RAW video option is selected)

Also I haven't been able to measure properly how many frames its offset yet. I think I need to record something more precise than my fingers tapping! something more definite (maybe a machine or something?). I was thinking about doing the clapper several times within a take that will probably be the most reliable so far for me to measure.

I like the line of enquiry. I have always wondered why I can't seem to change the fps override???

Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 24, 2013, 04:50:04 AM
Click on the "Q" button to go into "FPS override" then the "set" button and turn the set dial until you get 24 then scroll down to "optimize for" and select "exact FPS", it should show "actual FPS: 24.001". Also set H.264 video recording to 24p in the Canon menu, I'm not sure if that makes a difference but just to be on the safe side.

I just recorded 8 minutes of cookie monster with my Sony PCM D50, I'm transcoding the file to prores in AE and I'll try to sync it up in FCPX. Oh boy oh boy oh boy, mmm cookies!
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 24, 2013, 04:55:17 AM
Quote from: mageye on May 24, 2013, 04:27:14 AM
Yes the video is perfectly smooth. No problems with that.

I have the fps override turned OFF. If I go into the fps override and turn it on it just reads 2.502 and will record at that rate. I don't seem to be able to change the fps override (is there some way of changing this that I have maybe missed?).

My belief is that it's recording at 29.976 (that's what it says at the bottom when the RAW video option is selected)

Also I haven't been able to measure properly how many frames its offset yet. I think I need to record something more precise than my fingers tapping! something more definite (maybe a machine or something?). I was thinking about doing the clapper several times within a take that will probably be the most reliable so far for me to measure.

I like the line of enquiry. I have always wondered why I can't seem to change the fps override???

I too was boggled how I could change the framerate since what appears in the info below the raw  options is 29.97, even when fps was changed, then I realized I had to go to the CANON MENU and change the movie rate there from 30p to 24p, now the info in the raw option reflected 23.976.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 24, 2013, 05:05:17 AM
Ahh right, I never shoot 30p so I hadn't noticed that. I just synced up a minute of cookie monster, no drift. Trying for 10min now.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 24, 2013, 05:14:22 AM
OMG I feel like such a moron!

I have been wondering how to access those 'Q' menus for some time :o ;) :-[. I really should read more manuals!

I have now managed to fps override to 24pfs. I don't know if this solves my problem yet as I am yet to test it with the sync setup (clapper board and stuff).

I do feel however, that I this will go a long way into solving my little problem if not completely resolve it. Thanks so much for your help.

I will do some tests in the day (today) and hopefully I will triumph. I will indeed post my results and conclusions.

Good luck with the Cookie mmmmm!!!
Title: Re: uncompressed 14-bit RAW video recording
Post by: mvejerslev on May 24, 2013, 06:02:54 AM
Since we're lobbing everything in here, I thought I'd re-iterate the discovery of someone else here: Setting photo quality to small JPG in-camera instead of Raw, frees up buffer, at least on the 5DII. It enabled me to shoot twice the amount of frames at any given size before buffer overrun. It has no effect on the frame size in bytes on the DNGs. Perhaps the devs could look into exploiting this memory allocation?
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 24, 2013, 06:08:44 AM
Quote from: mvejerslev on May 24, 2013, 06:02:54 AM
Since we're lobbing everything in here, I thought I'd re-iterate the discovery of someone else here: Setting photo quality to small JPG in-camera instead of Raw, frees up buffer, at least on the 5DII. It enabled me to shoot twice the amount of frames at any given size before buffer overrun. It has no effect on the frame size in bytes on the DNGs. Perhaps the devs could look into exploiting this memory allocation?

The devs have already noted this since the very first builds of ML raw, and that was recommended to testers of the nightly builds.
I think once every feature is stable , that this will be mentioned in setup.
I should suggest that if performance take so much hit when raw stills is enabled, that ML team should might as well automatically disable raw stills once raw video is engaged.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Picture and Color on May 24, 2013, 06:31:39 AM
Quote from: stevefal on May 23, 2013, 08:28:12 PM
Notice hot pixels flaring on her zipper below the pendant at ~1:03.

You're right... I'm seeing them too now. Not just in that one clip, but others too. Is this happening with anyone else's footage?
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 24, 2013, 06:38:14 AM
Quote from: Picture and Color on May 24, 2013, 06:31:39 AM
You're right... I'm seeing them too now. Not just in that one clip, but others too. Is this happening with anyone else's footage?

http://www.magiclantern.fm/forum/index.php?topic=5614.msg39655;topicseen#msg39655
Title: Re: uncompressed 14-bit RAW video recording
Post by: aaphotog on May 24, 2013, 07:16:40 AM
Quote from: ted ramasola on May 24, 2013, 06:08:44 AM
The devs have already noted this since the very first builds of ML raw, and that was recommended to testers of the nightly builds.
I think once every feature is stable , that this will be mentioned in setup.
I should suggest that if performance take so much hit when raw stills is enabled, that ML team should might as well automatically disable raw stills once raw video is engaged.

I do not like the idea of automatically setting this. I  am working with a 5D3 and I've tried setting my image quality to sRaw. Normally(with raw being selected) my buffer doesn't go beyond 1* and I can record until I manually stop recording(1080p raw). When I selected sraw though, I only got one second of recording before it automatically stopped on me.
Title: Re: uncompressed 14-bit RAW video recording
Post by: kbru on May 24, 2013, 08:04:13 AM
Recording raw via HDMI to Atomos Ninja w/SSD possible???
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 24, 2013, 08:07:43 AM
Quote from: kbru on May 24, 2013, 08:04:13 AM
Recording raw via HDMI to Atomos Ninja w/SSD possible???

No, but a CF card slot to SSD drive option is being explored.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 24, 2013, 09:09:40 AM
Ok I've tested external audio sync on a 12min 24p file in FCPX, it's perfect!
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 24, 2013, 09:49:33 AM
Now for those curious if rolling shutter in Raw video is different as in h264.

Here is a rolling shutter test on 2 cameras, in raw vs h264.

You can draw you own conclusions on the test. :)

Title: Re: uncompressed 14-bit RAW video recording
Post by: tihon on May 24, 2013, 09:51:41 AM
Quote from: ted ramasola on May 24, 2013, 09:49:33 AM
Now for those curious if rolling shutter in Raw video is different as in h264.

Here is a rolling shutter test on 2 cameras, in raw vs h264.


it seems to mark2 in raw provide more jello:)
Title: Re: uncompressed 14-bit RAW video recording
Post by: budafilms on May 24, 2013, 10:06:23 AM
Do you know when recording draw if the preset have any influences over the footage? And the WB? Thanks!
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 24, 2013, 10:09:15 AM
Quote from: budafilms on May 24, 2013, 10:06:23 AM
Do you know when recording draw if the preset have any influences over the footage? And the WB? Thanks!

You mean picture profiles? like neutral, faithful, cinestyle etc? I think Not when shooting dngs. Also with WB as you can change all that in post.
Title: Re: uncompressed 14-bit RAW video recording
Post by: dhilung on May 24, 2013, 11:02:10 AM
Quote from: ted ramasola on May 24, 2013, 09:49:33 AM
Now for those curious if rolling shutter in Raw video is different as in h264.

I think the actual image acquisition pipeline of DIGiC is the same for both cases until the RAW frames are copied over using EDMAC (raw_rec) or encoded in H.264 (normal). This means the rolling shutter effect should be the same.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Lars Steenhoff on May 24, 2013, 12:34:13 PM
The rolling shutter you are comparing is between the 5d and the 7d, that's not a valid test, full frame vs crop camera, even when if they had the same sensor.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bumkicho on May 24, 2013, 12:40:50 PM
Quote from: squig on May 24, 2013, 09:09:40 AM
Ok I've tested external audio sync on a 12min 24p file in FCPX, it's perfect!

Great to hear. I am assuming you set up your external audio to 48hz and set fps override to 24fps for this test, right?
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 24, 2013, 02:59:23 PM
Quote from: Lars Steenhoff on May 24, 2013, 12:34:13 PM
The rolling shutter you are comparing is between the 5d and the 7d, that's not a valid test, full frame vs crop camera, even when if they had the same sensor.

Yes, its not the same cameras that's why I tested them both first with the same codecs so you can use as reference each of their angle of tilt then use those individual results as basis for the next one which uses raw on the other with the h264 on the other if there was a difference in their individual angles. Its result will tell you if there is a difference in RS angle between raw and h264 for the 5D alone. Not if there is a similarity between the 2 cameras. :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: JackDaniel412 on May 24, 2013, 05:54:33 PM
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 24, 2013, 06:46:07 PM
Quote from: bumkicho on May 24, 2013, 12:40:50 PM
Great to hear. I am assuming you set up your external audio to 48hz and set fps override to 24fps for this test, right?

Affirmative.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 24, 2013, 09:50:28 PM
How can I get my 24fps RAW sequences to import into After Effects at 24fps and not 30fps?

My example is a sequence of 150 frames at 24fps should be a sequence of 6.25 seconds in length

What is happening is After Effects seems to be recognising the RAW(DNG) sequence as 30fps whatever it was recorded at

This means that my 150 frames translates to 5 seconds in length.

HELP!
Title: Re: uncompressed 14-bit RAW video recording
Post by: NedB on May 24, 2013, 09:55:13 PM
mageye: Right-click on imported sequence in After Effects, select Interpret Footage>Main. Set Frame Rate to 24.

To make this the default: Edit>Preferences>Import, set Sequence Footage to 24 (or whatever you want the default to be).

Cheers!
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 24, 2013, 10:02:15 PM
Quote from: NedB on May 24, 2013, 09:55:13 PM
mageye: Right-click on imported sequence in After Effects, select Interpret Footage>Main. Set Frame Rate to 24.

To make this the default: Edit>Preferences>Import, set Sequence Footage to 24 (or whatever you want the default to be).

Cheers!

YAY! You are a star. I have been messing around for something like two days and it took me that long to analyse what exactly was going on.

It also means that all of the visual sequences I have put together so far have been a little shorter than they should be and running a little overcranked!

Its amazing. That little nugget of information will enable me to synchronise my footage with audio and that is what I have been trying to achieve.

Again thanks. Lots! Also thanks lots to squig for the invaluable information about 'q' so I could actually set fps override! ;D :D :P
Title: Re: uncompressed 14-bit RAW video recording
Post by: Lars Steenhoff on May 24, 2013, 10:39:55 PM
Quote from: ted ramasola on May 24, 2013, 02:59:23 PM
Yes, its not the same cameras that's why I tested them both first with the same codecs so you can use as reference each of their angle of tilt then use those individual results as basis for the next one which uses raw on the other with the h264 on the other if there was a difference in their individual angles. Its result will tell you if there is a difference in RS angle between raw and h264 for the 5D alone. Not if there is a similarity between the 2 cameras. :)

Thanks for pointing that to me,  I understand what you tested now, seems the raw has a bit more jello, I just don't get how this appears.
the sensor is read out before both the raw and the h264 right ?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 24, 2013, 10:50:39 PM
H264 uses motion manager... I think this might be its purpose.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 25, 2013, 12:15:49 AM
@Lars,

In my observation, there is "visually" no difference in the behavior of rolling shutter in the 5D mkII between Raw record and H264.
If there is, it is very small.
Title: Re: uncompressed 14-bit RAW video recording
Post by: noisyboy on May 25, 2013, 12:45:31 AM
Mageye - you can ask questions regarding post processing here: http://www.magiclantern.fm/forum/index.php?topic=5404.0

JackDaniel412 - please post your work under share your videos (http://www.magiclantern.fm/forum/index.php?board=19.0) board - not under the dev threads unless it's relavant. And no more of this double posting bizniz you cheeky devil ;)

Here endeth the moderation  8)
Title: Re: uncompressed 14-bit RAW video recording
Post by: KMikhail on May 25, 2013, 10:40:48 AM
Hi,

Not sure if i should post it here, but... would you please add 800/400 to vertical resolutions and 1728 to horizontal? 1920*800, 960*400 and 1728*720 have a perefect 2.40:1 ratio. Besides, all of them are perfect multiples of 16.

Your work is much appreciated!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Levinson on May 25, 2013, 11:01:55 AM
Hello.

Thanks again for the awesome mods ML and other developers!!

Maybe my request is not possible because the RAW mod is still a module (but could be achieved if compiling myself - at which I'm no expert), but would it be possible to have ML remember the last used H & V resolution settings upon boot?

Thank you :)

Lev
Title: Re: uncompressed 14-bit RAW video recording
Post by: vipmask on May 25, 2013, 01:44:44 PM
Shooting from a helicopter  "aerocrane" 8)

8620 frames
1920х1024 25fps! - scaling to 1920х1080
DNG http://yadi.sk/d/4qPQiKvf59S20
ML 18-05 "rw_over_test"
Title: Re: uncompressed 14-bit RAW video recording
Post by: Stedda on May 25, 2013, 02:04:55 PM
Quote from: vipmask on May 25, 2013, 01:44:44 PM
Shooting from a helicopter  "aerocrane" 8)

8620 frames
1920х1024 25fps! - scaling to 1920х1080
DNG http://yadi.sk/d/4qPQiKvf59S20
ML 18-05 "rw_over_test"

GD you're brave... flying over water with a $3k camera on a RC Helo. I can't even hover mine! LOL
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 25, 2013, 04:46:57 PM
Plus flying over the water like a boss.
Title: Re: uncompressed 14-bit RAW video recording
Post by: eyeland on May 25, 2013, 05:22:26 PM
Quote from: 1% on May 25, 2013, 04:46:57 PM
Plus flying over the water like a boss.
+1
Title: Re: uncompressed 14-bit RAW video recording
Post by: vipmask on May 25, 2013, 09:29:43 PM
Quote from: Stedda on May 25, 2013, 02:04:55 PM
GD you're brave... flying over water with a $3k camera on a RC Helo. I can't even hover mine! LOL
It's not terrible if the technique is reliable :-) We have 7 years of flying over water and nothing :-) Now you can take on the RAW mark 3 and do not hang expensive Red Epic ;-)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 25, 2013, 09:49:01 PM
Well, it's been about 2 weeks since I started fooling around with RAW and I'm still constantly amazed by the quality. My brain keeps telling me "this shady region should be clipped to pitch blackness so don't expect any details" but I still see everything and get surprised lol.

RAW is not just about the sheer quality and resolution increase either. I've been fooling around with other technical stuff, like IT8 targeting for increased color accuracy which is more flexible with RAW data. I still have a reflective target used for flatbed scanners. so it's not optimal and a hassle to avoid reflections. However, the results have been pretty darn good I think the colors are fairly representative with what i see with my eyes. I've also learned about color-managed video. All my frames are converted to Rec.709 (using this profile: http://www.color.org/rec709.xalter) instead of sRGB and encoded with the Rec.709 flags on in x264. Playback in MPC-HC with color management on makes everything look exactly as it should (at least, I think it does). Fun times with the new opportunities RAW gives us!
Title: Re: uncompressed 14-bit RAW video recording
Post by: bashi on May 26, 2013, 12:19:55 AM
My first Post, so thanks to all devs and others for your work.

Recording Raw runs fine on 5d III, since i disabled Raw Photo and set Jpeg to S3 ;-) (no idea if the S3 does anything) I can even enable Overlay with Zebra, Focus, Histogram, Waveform and it still records fine in 1928/960. It however, in some cases, is not able to save Footer, usually if ran out of Buffer, but as well on ordinary video stop occasionally. Now, if this happens and i continue to record it usually is not able to save Footer for the following recordings. It happens as well then frequently that the Filesystem on the CF Card gets messed up - not read/writable by 5d III anymore - so renders card unusable in field! Have to reformat in mac then. This is bad, since i've got only one fast Card yet...

Another issue i have is Audio. I try since several hours to get the Recorded Wav from the 5D III in Sync with Video in ffmpeg. No matter what i set on camera 23.976/24.001 Frames or in ffmpeg vsync, async, audio offset, 23.976, 24 frames - the Audio always goes out of sync pretty badly even after 7-8 sec. It appears the sound is faster than the Video.
So my questions (to start with):

Has anyone in any software being able to Sync Audio (recorded in Cam to Wav) with Raw footage?
If so, where and what and how, you know?

is 5D III Wav recorded in 48000 Rate? Any way to set this in Camera?

I've noticed the Audio Length is 00:00:14.76 while the Video length is 00:00:14.50 (according to ffmpeg)...

And another one - Bitrate setting for 5d III H.264 is not working? Nothing happens with h264.ini (or how it's called..)

thanks
bashi

EDIT: I just see: ffmpeg to avi is in sync
ffmpeg -i %6d.tiff -i M0000000.WAV -acodec copy test.avi
but prores_kostya isn't...

EDIT 2: Forget the Audio thing, i think i solved it. Probably messed up some flags. This is working:
ffmpeg -i %6d.tiff -r 24 -i M0000000.WAV -async 1 -vsync 0 -vcodec prores_kostya -profile:v 3 -qscale:v 6 -vendor ap10 -pix_fmt yuv422p10le -r 24 -threads 4 -strict -2 -y Audio.mov
Title: Re: uncompressed 14-bit RAW video recording
Post by: dhilung on May 26, 2013, 01:24:36 AM
Quote from: vipmask on May 25, 2013, 01:44:44 PM
Shooting from a helicopter  "aerocrane" 8)

8620 frames
1920х1024 25fps! - scaling to 1920х1080
DNG http://yadi.sk/d/4qPQiKvf59S20
ML 18-05 "rw_over_test"

Awesome!
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 26, 2013, 03:13:38 AM
Quote from: bashi on May 26, 2013, 12:19:55 AM
My first Post, so thanks to all devs and others for your work.

Recording Raw runs fine on 5d III, since i disabled Raw Photo and set Jpeg to S3 ;-) (no idea if the S3 does anything) I can even enable Overlay with Zebra, Focus, Histogram, Waveform and it still records fine in 1928/960. It however, in some cases, is not able to save Footer, usually if ran out of Buffer, but as well on ordinary video stop occasionally. Now, if this happens and i continue to record it usually is not able to save Footer for the following recordings. It happens as well then frequently that the Filesystem on the CF Card gets messed up - not read/writable by 5d III anymore - so renders card unusable in field! Have to reformat in mac then. This is bad, since i've got only one fast Card yet...

I did an external audio sync test in FCPX on a 12min file and it synced perfectly. Make sure FPS override is set to 24/25fps and the frame rate in the Canon menu is set the same.

Changing the photo size frees up some ram for ML.

I'm just shooting with global draw off (don't need it) and recording 1920x840 (can do 1080) with no hiccups on a Toshiba 1066x card.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 26, 2013, 03:40:37 AM
Found where LV "work" memory is I think:

ROM:FF0EEEA0 ShootMalloc384M DCD 0x4D637352, 0x632E7267, 0 ; DATA XREF: smemTerm+14o

Everything checks there for free memory? So if you resize this pool it should respect it? Unlike rscmgr at initialization?

There is also smemSetCraw and memdump... I guess this writes to a file what is using memory?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 26, 2013, 06:15:53 AM
Quote from: bashi on May 26, 2013, 12:19:55 AM
Another issue i have is Audio. I try since several hours to get the Recorded Wav from the 5D III in Sync with Video in ffmpeg. No matter what i set on camera 23.976/24.001 Frames or in ffmpeg vsync, async, audio offset, 23.976, 24 frames - the Audio always goes out of sync pretty badly even after 7-8 sec. It appears the sound is faster than the Video.
So my questions (to start with):

Has anyone in any software being able to Sync Audio (recorded in Cam to Wav) with Raw footage?
If so, where and what and how, you know?

is 5D III Wav recorded in 48000 Rate? Any way to set this in Camera?

I've noticed the Audio Length is 00:00:14.76 while the Video length is 00:00:14.50 (according to ffmpeg)...

And another one - Bitrate setting for 5d III H.264 is not working? Nothing happens with h264.ini (or how it's called..)

thanks
bashi

EDIT: I just see: ffmpeg to avi is in sync
ffmpeg -i %6d.tiff -i M0000000.WAV -acodec copy test.avi
but prores_kostya isn't...

EDIT 2: Forget the Audio thing, i think i solved it. Probably messed up some flags. This is working:
ffmpeg -i %6d.tiff -r 24 -i M0000000.WAV -async 1 -vsync 0 -vcodec prores_kostya -profile:v 3 -qscale:v 6 -vendor ap10 -pix_fmt yuv422p10le -r 24 -threads 4 -strict -2 -y Audio.mov

I too am having trouble with the onboard recorded wav file synching up.  I realize someone replied to this talking about a 12 minute external recorder, but I'm talking about the internal audio that I need for reference, or when I don't shoot with an external recorder.
Will dig into this as well, and I'll try your ffmpeg settings.
According to Mediainfo, the recorded onboard mono .wav file is:
Audio
Format                                   : PCM
Format settings, Endianness              : Little
Bit rate mode                            : Constant
Bit rate                                 : 768 Kbps
Channel(s)                               : 1 channel
Sampling rate                            : 48.0 KHz
Bit depth                                : 16 bits
Stream size                              : 4.39 MiB (100%)

For reference, the audio recorded with a non-hacked 5d3 is:
Audio
ID                                       : 2
Format                                   : PCM
Format settings, Endianness              : Little
Bit rate mode                            : Constant
Bit rate                                 : 1 536 Kbps
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 KHz
Bit depth                                : 16 bits
Delay relative to video                  : 42ms


Don't know if the "delay relative to video" means anything or not in the ML world, and I'm sure it's something they're aware of.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 26, 2013, 06:28:28 AM
Ted said the internal audio is off by up to 4 frames if I recall correctly, that's on the MK2 but it's probably the same with the MK3.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 26, 2013, 06:28:40 AM
Quote from: bashi on May 26, 2013, 12:19:55 AM
EDIT: I just see: ffmpeg to avi is in sync
ffmpeg -i %6d.tiff -i M0000000.WAV -acodec copy test.avi
but prores_kostya isn't...

EDIT 2: Forget the Audio thing, i think i solved it. Probably messed up some flags. This is working:
ffmpeg -i %6d.tiff -r 24 -i M0000000.WAV -async 1 -vsync 0 -vcodec prores_kostya -profile:v 3 -qscale:v 6 -vendor ap10 -pix_fmt yuv422p10le -r 24 -threads 4 -strict -2 -y Audio.mov

I don't understand either of these commands - what is %dt.tiff, and why output it to a .mov file?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 26, 2013, 06:29:18 AM
Quote from: squig on May 26, 2013, 06:28:28 AM
Ted said the internal audio is off by up to 4 frames if I recall correctly, that's on the MK2 but it's probably the same with the MK3.

We're not talking 4 frames though on the 5d3 - I'm talking like 1 full second.
Title: Re: uncompressed 14-bit RAW video recording
Post by: pacman829 on May 26, 2013, 06:31:40 AM
I'm curious to know what you were using to convert the color space to rec 709 , what program adds that icc?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 26, 2013, 11:42:56 AM
Well, I've basically been up all night trying to get the audio sync working for the 5d3.  After much trial and error, I've found that about a 103% "time stretch" does a really great job.  It's between 103-104%; probably 103.2 would be damn near perfect.  Make sure you whack the end of the audio file off first before you do the stretch.
I tested this on a 4 minute clip and it's off by maybe a nanosecond all the way at the end.
Keep in mind this is with FPS override ENABLED, and set to 24p.  If I disable this, the video gets waaaaay ahead of the audio and then "catches up" and I lose even more sleep.  :)  Hope this helps someone who shoots without an external recorder.  Workflow = Adobe after effects CS6.

You can trim the furthest point of the wav file in your composition by hitting the "alt" and ] keys (will remove everything to the right).
Shawn
Title: Re: uncompressed 14-bit RAW video recording
Post by: swinxx on May 26, 2013, 01:24:42 PM
i have tried to record a clip with audio enabled, the wav was then recorded onto the sd card, the filename was like the raw filename, e.g. m00001.wav
so, i have tied to open the wav with several programs, none worked?
can someone please explain how the audio recording works when record with raw?
thx.
Title: Re: uncompressed 14-bit RAW video recording
Post by: g3gg0 on May 26, 2013, 01:57:12 PM
"so, i have tied to open the wav with several programs, none worked? "

what does mean "didnt work" ? more details please.
you recorded in movie mode?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 26, 2013, 01:58:33 PM
Quote from: swinxx on May 26, 2013, 01:24:42 PM
i have tried to record a clip with audio enabled, the wav was then recorded onto the sd card, the filename was like the raw filename, e.g. m00001.wav
so, i have tied to open the wav with several programs, none worked?
can someone please explain how the audio recording works when record with raw?
thx.

You have to put it on the timeline of your editor and remove the extra hours of data; shrink it down to be the same size as your video clip.  Or run it through an audio editor like Soundforge, Adobe Audition, or the free command line ffmpeg.  But it should at least have the same filename as your raw file.  Synching is a pain in the rear though.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Andy600 on May 26, 2013, 02:11:49 PM
It would be nice if the camera could generate SMTPE timecode. Can a pico script maybe do this in parallel to recording raw video?
Title: Re: uncompressed 14-bit RAW video recording
Post by: N/A on May 26, 2013, 02:37:48 PM
Reaper is a great little audio program, powerful, fast, and lots of great tools. Or fcpx's re-time.

Ha, just looked at Reaper's site and noticed they added audio stretch markers.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bashi on May 26, 2013, 02:47:51 PM
@Shield
this is the command to convert tiff sequence and wav to ProRes 422 with ffmpeg. Will do some more tests now if the Audio is in Sync.
Title: Re: uncompressed 14-bit RAW video recording
Post by: dude on May 26, 2013, 03:11:55 PM
with the audi problem, you should give plural eyes a try. sure, it s not a perfect solution, but works pretty well
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 26, 2013, 03:12:51 PM
Quote from: bashi on May 26, 2013, 02:47:51 PM
@Shield
this is the command to convert tiff sequence and wav to ProRes 422 with ffmpeg. Will do some more tests now if the Audio is in Sync.

Sounds like you have a separate step in your workflow - I import the dng sequence directly into AE.  Let me know if your time is synched up.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 26, 2013, 03:15:07 PM
Quote from: dude on May 26, 2013, 03:11:55 PM
with the audi problem, you should give plural eyes a try. sure, it s not a perfect solution, but works pretty well

If I had a good reference track I'd already be using pluraleyes to sync it.  The problem is when you're not using an external audio device and just the onboard audio.  It's either off in the beginning, the middle, or the end.  I've had longer clips where the audio was "ahead" in the beginning, ok in the middle, and behind towards the end.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bashi on May 26, 2013, 03:20:55 PM
No ae here, all open source  ;D  My workflow is Raw -> dng -> Tiff -> ProRes 422. Using DCRaw and ffmpeg. My goal is to setup a automatic system, where i put in the CF Card and hit one Button and end up with ProRes including Audio. Still having the Tiff if i need them.

I still get corrupted CF Card though, after Footer Save Error. Preety preety Bad ;-)

Title: Re: uncompressed 14-bit RAW video recording
Post by: dude on May 26, 2013, 03:24:59 PM
Quote from: Shield on May 26, 2013, 03:15:07 PM
If I had a good reference track I'd already be using pluraleyes to sync it.  The problem is when you're not using an external audio device and just the onboard audio.  It's either off in the beginning, the middle, or the end.  I've had longer clips where the audio was "ahead" in the beginning, ok in the middle, and behind towards the end.

Oh ok, i understand. Did not know the whole track ist like unstable.

Would be cool to manage this inside the cam, since a external recorder (and even every equipment attached to the dslr) makes the run and shoot cams more like a lame elephant
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 26, 2013, 03:31:09 PM
Quote from: NedB on May 24, 2013, 09:55:13 PM
mageye: Right-click on imported sequence in After Effects, select Interpret Footage>Main. Set Frame Rate to 24.

To make this the default: Edit>Preferences>Import, set Sequence Footage to 24 (or whatever you want the default to be).

Cheers!

Personally I think that the audio issues that people are having deserves a complete topic of its own. I think when people are getting sync mismatch the above advice served me. After lots of investigation seeing that my audio was 'drifting' out of sync it was purely because I was not bringing in footage at the correct frame rate. It's already been covered a couple of pages back.

Make sure in the camera that you force 24fps (or whatever you want to work in) and that After Effects is consistent.

Yes this is a post subject and really should be in a post topic of its own. I am sorry to be continue posting on this particular topic but it seems people need to be able to address this.
Title: Re: uncompressed 14-bit RAW video recording
Post by: swinxx on May 26, 2013, 04:36:33 PM
so i managed to record a wav file which is indeed working after conversion to prores422hq.
and it is - as described - out of sync.
so, i found 2 possibilities in the menu - RAW Video and FPS Override - where it is possible to enable and disable seperate wav recordings.
i have recorded in 23,976fps and used that setting in post.

my settings are:
in the canon menu - photo: jpeg smallest possibility, 1920x108024p,
in magic lantern - raw Video enabled, 1920x1080, 23.976fps, global draw off, turned record seperate wav on in both menus.

how is it possible to manage the audio in sync recording?
has anyone recorded audio in sync yet?
thx
Title: Re: uncompressed 14-bit RAW video recording
Post by: Griffleiste on May 26, 2013, 05:15:59 PM
I kinda get lost in all these new daily/nightly stuff. Is there anywhere some sort of changelog where you can see which interesting new features have arrived?
Title: Re: uncompressed 14-bit RAW video recording
Post by: bashi on May 26, 2013, 06:25:39 PM
Audio Sync Wav (internal) with Raw Video is not working. I think there's something wrong with the Wav. VLC gives me length of over 1 hour for a some Minutes Recording. And it's not just offset but stretched too. Is it really 48000Hz?
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 26, 2013, 06:31:37 PM
Re-save with audacity to get the right length.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bashi on May 26, 2013, 06:36:51 PM
Isn't this something that could and should be done in camera? imo clearly yes!
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 26, 2013, 06:39:26 PM
Its from the way it makes files.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mageye on May 26, 2013, 06:42:45 PM
I recorded audio internally (in camera) and externally (on my ZoomH2) simultaneously

I then recorded some RAW whilst both audio recorders were running

What I noticed is there was some mismatch in the two files. I will note that the pitch did NOT change. (so its not a sampling frequency issue)

When I aligned the two files (audio) in After effects it seems that there were added spaces of time and/or gaps in the recorded audio.

So this would make it impossible to synchronise the recorded wav file because sync would be in in some places but with just some audio cut out (or time added) would loose any sync you have.

The recording I made on the ZoomH2 did not do this.

I think there maybe something interrupting the recording process (maybe the buffering of so much RAW data!) and making it both unstable and more importantly unreliable.
Title: Re: uncompressed 14-bit RAW video recording
Post by: bashi on May 26, 2013, 06:43:49 PM
And this is not changeable at all? If not, we have to live with it, if yes it makes sense to make it work.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 26, 2013, 07:17:19 PM
dma memcpy somewhat fixed this issue on 600D. It could on 5d3 I guess.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 26, 2013, 11:40:33 PM
Quote from: mageye on May 26, 2013, 06:42:45 PM
When I aligned the two files (audio) in After effects it seems that there were added spaces of time and/or gaps in the recorded audio.


I noticed this as well.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 26, 2013, 11:46:34 PM
Quote from: bashi on May 26, 2013, 06:43:49 PM
And this is not changeable at all? If not, we have to live with it, if yes it makes sense to make it work.

For now I'm leaving the FPS override option on and doing a 103% time stretch in post.  It's at least close to good enough.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yomommassis on May 27, 2013, 04:53:26 AM
I posted this in the 5d2 RAW thread but I felt this also applied to any of the cameras

after hearing about the new raw feature I was trying to find a cool practical use for it, and the big thing that came to mind was time lapses without adding shutter actuations



This test took 1 hour out behind my house, pretty dark aside from the full moon blasting the scene
5D Mark 2 May 20th build
640 ISO
T/1.5
FPS Override 0.178
RAW Video 1880x1250

I had to really adjust the color in post because the RAW kicked the magenta up a lot
Also had artifacts at the bottom of the frame

http://i.imgur.com/G1N6ya8.jpg

and there was this weird problem where every other frame was the first frame...
so instead of capturing 323 I actually captured 162

162 frame time lapse without adding any actuations
Title: Re: uncompressed 14-bit RAW video recording
Post by: Habitat on May 27, 2013, 09:03:20 PM

I really like this idea though. Battery life better or worse compared to a conventional time lapse?
Title: Re: uncompressed 14-bit RAW video recording
Post by: platu on May 27, 2013, 09:33:33 PM
If you want to save shutter actuations, wouldn't it be better to just turn on the silent pictures option combined with the intervalometer?  That should essentially give you the same hight quality DNGs as the raw video option.  I recently did a timelapse that way for the same reason of wanting to save my shutter.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 28, 2013, 02:47:28 AM
Peeps if you're gonna quote someone don't include the video links in the quote.

I finally got a workflow going on the mac that actually works. I shot this just before I got it working, see notes on vimeo page about how to setup ACR.

Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 28, 2013, 03:03:49 AM
lol @ the pissing ape :D

Great video. It seems to be one of the few I've seen that haven't been mangled *after* being captured in post. Lots of high frequency detail preserved, particularly noticeable on the animal textures (fur/skin) and the girl's striped cap (your daughter I presume?) It really shows the difference between the 5D3 and 5D2 too, even though my jaw still constantly drops by how good the 5D2 looks in RAW.
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 28, 2013, 04:20:28 AM
I presume she's mine. It's a little mangled thx to ACR but I've got that sorted now. I shot mostly ISO multiples of 100 which are cleaner, I didn't denoise at all in ACR, just added some film convert grain to mask any FPN. Next time I'll expose a bit more to the left to control the highlights better. There's tonnes of detail, it looks too sharp with the default 25 sharpness on my 32" Samsung.
Title: Re: uncompressed 14-bit RAW video recording
Post by: mvejerslev on May 28, 2013, 04:45:57 AM
Very nice video.

But I hate to say, using 4 year old raw software with my brand new 5D Mark Free is not going to be satisfying. Better to work with the CR team to nail the flicker issues :-)
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 28, 2013, 05:15:37 AM
Thx. The new ACR build is a bit better but with a bit of work I was able to get an almost identical result, but sure Adobe can fix it.
Title: Delete feature build May 27 5D2 very risky
Post by: ted ramasola on May 28, 2013, 06:57:17 AM
I'm testing build may 27 for 5d2 posted by a.d. -bfd891fe53ab.zip  and here is my observation so far.

The delete feature is a nice feature but currently it is very risky. Use with caution.

scenario:
Shoot several clips raw video, say clip 1 2, 3.
Go to ML menu>raw video menu>video file (this shows NO VIDEO FILE as default)
Click SET to show first clip, M0000000.RAW,
Say you want to delete 3rd clip so you turn the main scroll wheel to go to M0000003.RAW
then you ROLL down to select DELETE then press SET to delete video size but file remains but 0kb in size. This is ok for now, also note WAV file remains unchanged and not deleted.

But here is the risk,
While selection in still in DELETE if you happen to roll the TOP SCROLL WHEEL, the clips be DELETED 1 by 1. ! THIS IS RISKY as you think you are selecting the clips but in fact they are already deleted without warning.
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 28, 2013, 08:17:22 AM
... that's why I didn't include it in the official repo.

When you use unofficial forks, please mention so. Or, better, post in the thread dedicated to that fork.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 28, 2013, 08:25:10 AM
edited my post to indicate who posted and the build file. Which is by a.d. may 27 : bfd891fe53ab.zip

I also posted already in the thread started by a.d.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 28, 2013, 09:02:21 AM
There,s a delete feature now, that is amazing, gotta test it out. Alex, genius

*I didn,t understand this feature was a sidedish on some other firmware-version. I live and learn :).
Thanks again for your efforts
//D
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on May 28, 2013, 09:06:06 AM
Don't use it, and don't blame me for the bugs introduced by others.
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 28, 2013, 09:14:12 AM
Quote from: Danne on May 28, 2013, 09:02:21 AM
There,s a delete feature now, that is amazing, gotta test it out. Alex, genius

Read my post above post #1584 regarding this feature.
Title: Re: uncompressed 14-bit RAW video recording
Post by: 5DanielMIII on May 28, 2013, 09:20:42 AM
Quote from: a1ex on May 28, 2013, 09:06:06 AM
Don't use it, and don't blame me for the bugs introduced by others.

Thank you for donating your time, Alex.
It is very generous of you! Don't waste your energy on all the "what about" people, and the growing crowd of people demanding their bug free ML software NOW!! They are impatience and unfortunatly have forgotten the skills of manners! (and some even the art of reading)

Thanks again, to all of you devs;-)
Title: Re: uncompressed 14-bit RAW video recording
Post by: DjJuvan on May 28, 2013, 10:18:59 AM
I am dissapointed over CF Toshiba 1066x cards... they don't go over 100MB/s speed :(

(http://shrani.si/f/3x/wA/2kOGCCmY/img5005.jpg)

You can record 1920x1080 without any problem, but it doesn't go over this, and you can't record at 50fps with such resolution.

I guess their cards aren't worth the risk :\

Or maybe the new ML builds handle this better?
Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 28, 2013, 10:22:38 AM
What did you expect? The Toshiba is about on par with the Lexar, there are no faster cards.  ::)
Title: Re: uncompressed 14-bit RAW video recording
Post by: KMA_WWC on May 28, 2013, 10:36:59 AM
Quote from: squig on May 28, 2013, 10:22:38 AM
What did you expect? The Toshiba is about on par with the Lexar, there are no faster cards.  ::)
Well...I'd like to argue about "there are no faster cards." :) Hoodman Steel 1000X 64gb pulls of solidly up to 100mb/s and most of the time bounces between 108-116mb/s. Just saying ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Audionut on May 28, 2013, 10:38:54 AM
Quote from: KMA_WWC on May 28, 2013, 10:36:59 AM
Well...I'd like to argue about "there are no faster cards." :)

There's nothing to argue.  Go complain to Toshiba about it.

Lets keep the discussion relevant, there's plenty of other threads about card speeds.
Title: Re: uncompressed 14-bit RAW video recording
Post by: skotopes on May 28, 2013, 03:19:27 PM
Transcend 1000x 32 Gb card on my 5DmkII
(http://i.imgur.com/gkpol2o.png)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Mei Lewis on May 28, 2013, 04:34:23 PM
Just had another though  if anyone is struggling with inconsistencies in Lightroom.

As Lightroom and ACR use the same RAW conversion engine (provided they are on the same versions of Camera Raw) you could get around inconsisten Lightroom highlight recovery by:


- Grade one DNG in lightroom,
- save the settings to xmp sidecar data (Ctrl-S is the keyboard shortcut on Windows, but it's in the menus too)
- close lightroom and open that dng in ACR lightroom settings should already be applied to it (brought across via xmp)
- synch this dng to all the others in ACR
Title: Re: uncompressed 14-bit RAW video recording
Post by: 1% on May 28, 2013, 04:37:55 PM
Quotethen you ROLL down to select DELETE then press SET to delete video size but file remains but 0kb in size. This is ok for now, also note WAV file remains unchanged and not deleted.

I found this about his bin when I was porting it... had it working and then naming changed so now it only deletes last file. He is overwriting the file instead of deleting and then trying to delete all the 3.7gb chunks his build makes. File browser can get past files anyways.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 28, 2013, 05:50:38 PM
Quote from: Mei Lewis on May 28, 2013, 04:34:23 PM
Just had another though  if anyone is struggling with inconsistencies in Lightroom.

As Lightroom and ACR use the same RAW conversion engine (provided they are on the same versions of Camera Raw) you could get around inconsisten Lightroom highlight recovery by:


- Grade one DNG in lightroom,
- save the settings to xmp sidecar data (Ctrl-S is the keyboard shortcut on Windows, but it's in the menus too)
- close lightroom and open that dng in ACR lightroom settings should already be applied to it (brought across via xmp)
- synch this dng to all the others in ACR

Interesting, but Lightroom 4 doesn't do sidecar files as far as I can tell - but you can export the dng back to a dng and it'll save the metadata along with it.  Are you suggest we sync the series in Photoshop then?
Title: Re: uncompressed 14-bit RAW video recording
Post by: Mei Lewis on May 28, 2013, 05:57:21 PM
Sync the series in Camera Raw - which is a separate program from Photoshop but comes bundled with it.

I've been using sidecar files with .CR2, not tried with DNG yet. Reading Adobe's site it works slightly differently for DNG. http://help.adobe.com/en_US/lightroom/using/WS638E3AC9-A04C-4445-A0D3-F7D8BA5CDE37.html (http://help.adobe.com/en_US/lightroom/using/WS638E3AC9-A04C-4445-A0D3-F7D8BA5CDE37.html) seems to say it should work the same, but with the metadata written into the DNG files rather than as sidecar files.
Title: Re: uncompressed 14-bit RAW video recording
Post by: moomilk on May 28, 2013, 06:06:29 PM
Quote from: skotopes on May 28, 2013, 03:19:27 PM
Transcend 1000x 32 Gb card on my 5DmkII
(http://i.imgur.com/gkpol2o.png)

Can this be right? Transcend should be capable of around 100Mb/s? Has the card been through low level format using the Transcend format programm? I heard it may rise the write speed.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 28, 2013, 06:12:21 PM
Quote from: Mei Lewis on May 28, 2013, 05:57:21 PM
Sync the series in Camera Raw - which is a separate program from Photoshop but comes bundled with it.

I've been using sidecar files with .CR2, not tried with DNG yet. Reading Adobe's site it works slightly differently for DNG. http://help.adobe.com/en_US/lightroom/using/WS638E3AC9-A04C-4445-A0D3-F7D8BA5CDE37.html (http://help.adobe.com/en_US/lightroom/using/WS638E3AC9-A04C-4445-A0D3-F7D8BA5CDE37.html) seems to say it should work the same, but with the metadata written into the DNG files rather than as sidecar files.
I think it's a problem with the Adobe raw 2012 camera process itself, not Lightroom per se.  See this thread here:
http://www.magiclantern.fm/forum/index.php?topic=5710.0

I mean one should be able to grade the first file in Lightroom and then "sync" it without any changes from file to file, right?

Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 28, 2013, 06:31:17 PM
Quote from: moomilk on May 28, 2013, 06:06:29 PM
Can this be right? Transcend should be capable of around 100Mb/s? Has the card been through low level format using the Transcend format programm? I heard it may rise the write speed.

Though the 5D2 can't reach those numbers, this benchmark is still lower than what it's capable of. The highest values I've seen so far were in the high 60s, almost 70 MB/s. Heck, with this new build, my 600x Sandisk card can record at a stable 59.5+ MB/s. Even though it's just below the speed required, I can finally do 1880x800 continuously at least until my 16GB card fills up (love the huge buffer we have :) )
Title: Re: uncompressed 14-bit RAW video recording
Post by: moomilk on May 28, 2013, 08:30:01 PM
Quote from: Yoshiyuki Blade on May 28, 2013, 06:31:17 PM
Though the 5D2 can't reach those numbers, this benchmark is still lower than what it's capable of. The highest values I've seen so far were in the high 60s, almost 70 MB/s. Heck, with this new build, my 600x Sandisk card can record at a stable 59.5+ MB/s. Even though it's just below the speed required, I can finally do 1880x800 continuously at least until my 16GB card fills up (love the huge buffer we have :) )

Hm, I was not entirely following the 5D markII thread, what exactly is causing the write speed drop while using LV?

Just thought if LV mode is limiting the write speeds, maybe it will be also worth making a RAW video mode not involving the LV at all, for videos with static camera position that could mean a higher res possibilities.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yoshiyuki Blade on May 28, 2013, 09:02:51 PM
Quote from: moomilk on May 28, 2013, 08:30:01 PM
Hm, I was not entirely following the 5D markII thread, what exactly is causing the write speed drop while using LV?

Just thought if LV mode is limiting the write speeds, maybe it will be also worth making a RAW video mode not involving the LV at all, for videos with static camera position that could mean a higher res possibilities.

I think a1ex implemented that at one point, but I don't think it made a difference in the end. No clue what's limiting the 5D2, maybe it's just the hardware limit, but we have been seeing gradual increases throughout the weeks. I've gained about +4-6 MB/s of write speed compared to when I first started playing with this, with the same card.
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 28, 2013, 10:35:20 PM
It,s implemented as a frozen image on the live view window and it helps a little.
Title: Re: uncompressed 14-bit RAW video recording
Post by: jasondhines on May 28, 2013, 11:49:37 PM
https://vimeo.com/67139126 (https://vimeo.com/67139126)

Please download the original, looks really sharp!

May 22nd build seems to be working really well for me. No vertical lines showing up and using ISO multiples of 100 seems to help make for a clean image. Using a 128GB 1000x Komputerbay card that benchmarked around 75MB/s to record 1920 x 900. Exported with a 2.39 aspect ratio at 1920 x 800. Looking forward to future developments and a completely rock-solid release.

Ordered a 128GB 1000x Transcend CF card to test. Hopefully it's faster than my Komputerbay. 
Title: Re: uncompressed 14-bit RAW video recording
Post by: nos on May 29, 2013, 02:10:02 AM


hi all, this forum is realy crazy!  ;)

a short "piano" in low light situation (iso 400) recorded with Build 130522 at 1280x720 with komputerbay 32gb 1000x

on 1920x1080 it drops frames after 3 seconds  ??? ....i have to test the cardspeed tomorrow

thank you very much magic lantern team!

Update: the video was scaled up to 1080p before uploading to youtube.
Title: Re: uncompressed 14-bit RAW video recording
Post by: agora1 on May 29, 2013, 03:06:28 AM
Hi, why for example in Photoshop they indicates only 8 bits
I know is 8,16,32 bits per channel so which software can tell us a
project is a 14 bits raw
Title: Re: uncompressed 14-bit RAW video recording
Post by: romeus on May 29, 2013, 03:35:21 AM
Hi guys I want to know if we can shoot for 10/15 minutes in a single take (like shooting an interview) in 1440 X 720  25p on Lexar 128 1000x?
Is the 5D Mark III can records audio when it shoot raw ? I did a search in the forum but I'm little bit confused :-\
is there any video tutorial to explain that
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 29, 2013, 08:06:33 AM
Quote from: romeus on May 29, 2013, 03:35:21 AM
Hi guys I want to know if we can shoot for 10/15 minutes in a single take (like shooting an interview) in 1440 X 720  25p on Lexar 128 1000x?
Is the 5D Mark III can records audio when it shoot raw ? I did a search in the forum but I'm little bit confused :-\
is there any video tutorial to explain that

Do you have a 5d3 and a Lexar 128 1000x card?  If so, fire them up and test it out.  You get slightly over 6 minutes per 32GB, so if that card is fast enough you should be good.  I personally wouldn't dream of recording anything super crucial while this software is being developed every day. :)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 29, 2013, 09:29:02 AM
Quote from: romeus on May 29, 2013, 03:35:21 AM
Hi guys I want to know if we can shoot for 10/15 minutes in a single take (like shooting an interview) in 1440 X 720  25p on Lexar 128 1000x?
Is the 5D Mark III can records audio when it shoot raw ? I did a search in the forum but I'm little bit confused :-\
is there any video tutorial to explain that
Actually, I have a silicon power 64gb 6000x card and I can go even a bit higher than that, around 1720 x 732 I believe. I think it,s 24p though but you should be fine I think. I can record audio and I have put my magic lantern firmware on a SD card(which is very slow, doesn,t matter) so the camera boots magic lantern from the sd-card while records to the CF-card. It does indeed record the audio to the SD-card but I can,t seem to sync the audio clips. I,m not sure what is going on when the audio gets recorded separately but it,s not reliable. SImply use an external recorder.

update* I read that someone stated that the audiofiles are working fine now with the may 28th build :). Havn,t tried it but that would be awesome :)

"With new build 28th(8ba641e), enabling the separate audio wave, there is no issues anymore.
Thanks as ever guys"

//D
Title: Re: uncompressed 14-bit RAW video recording
Post by: ted ramasola on May 29, 2013, 09:38:45 AM
Quote from: Danne on May 29, 2013, 09:29:02 AM
Actually, I have a silicon power 64gb 6000x card and I can go even a bit higher than that, around 1720 x 732 I believe. I think it,s 24p though but you should be fine I think. I can record audio and I have put my magic lantern firmware on a SD card(which is very slow, doesn,t matter) so the camera boots magic lantern from the sd-card while records to the CF-card. It does indeed record the audio to the SD-card but I can,t seem to sync the audio clips. I,m not sure what is going on when the audio gets recorded separately but it,s not reliable. SImply use an external recorder.
//D

I think you mean 600X not 6000X  ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Danne on May 29, 2013, 09:40:27 AM
Quote from: ted ramasola on May 29, 2013, 09:38:45 AM
I think you mean 600X not 6000X  ;)

haha, of course 600x sandisk   ;D
Title: Re: uncompressed 14-bit RAW video recording
Post by: dubzeebass on May 29, 2013, 02:07:39 PM
Hi folks,

Is anyone experiencing freezing when the HaCKeD mode is on?  The buttons still work but Liveview shows a frozen image, even with GD off.  Is this on purpose?

Before the lovely thread Nazis jump down my throat about SEARCH TEH THREAD1!!!!!! ... I recommend searching a thread about a hacked camera for the word hacked yourself :P

-B
Title: Re: uncompressed 14-bit RAW video recording
Post by: Stedda on May 29, 2013, 02:11:25 PM
Quote from: dubzeebass on May 29, 2013, 02:07:39 PM
Hi folks,

Is anyone experiencing freezing when the HaCKeD mode is on?  The buttons still work but Liveview shows a frozen image, even with GD off.  Is this on purpose?

Before the lovely thread Nazis jump down my throat about SEARCH TEH THREAD1!!!!!! ... I recommend searching a thread about a hacked camera for the word hacked yourself :P

-B

This has been explained several times already... the freezing is how you're getting a few more MB/s it's not a bug.
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 29, 2013, 02:33:18 PM
Quote from: dubzeebass on May 29, 2013, 02:07:39 PM
Hi folks,

Is anyone experiencing freezing when the HaCKeD mode is on?  The buttons still work but Liveview shows a frozen image, even with GD off.  Is this on purpose?

Before the lovely thread Nazis jump down my throat about SEARCH TEH THREAD1!!!!!! ... I recommend searching a thread about a hacked camera for the word hacked yourself :P

-B

If you Search for Hack3D you will find this thread:

Hack3D - Speed Improvements for RAW Recording
http://www.magiclantern.fm/forum/index.php?topic=5519.msg37539#msg37539

You got it.  Be Warned
Title: Re: uncompressed 14-bit RAW video recording
Post by: swinxx on May 29, 2013, 03:28:14 PM
Hello ml team!

We all know that when shooting raw and globaldraw is turned on, performance is suffering, but the info from raw histo is great for adjusting exposure (ettr hints ;))

So wouldnt it be possible to add a feature to turn globaldraw automatically of when starting to record?

That would be a great feature!

What i also noticed is, that anamorphic correction only works when globaldraw is set to pn, so i wonder if this menu shouldnt be moved to the globaldraw menu page,

Thx greets swinxx
Title: Re: uncompressed 14-bit RAW video recording
Post by: romeus on May 29, 2013, 04:24:33 PM
Quote from: Shield on May 29, 2013, 08:06:33 AM
Do you have a 5d3 and a Lexar 128 1000x card?  If so, fire them up and test it out.  You get slightly over 6 minutes per 32GB, so if that card is fast enough you should be good.  I personally wouldn't dream of recording anything super crucial while this software is being developed every day. :)

I don't have 5DmIII but I'ill get one next month for a summer project . I will Shot RAW  50% of the project :)
Are you sure that you'll get 6 minutes in 32GB card O_o I think you are wrong I did the math and it's like 11 minute for this resolution (without audio)

Quote from: Danne on May 29, 2013, 09:29:02 AM
Actually, I have a silicon power 64gb 6000x card and I can go even a bit higher than that, around 1720 x 732 I believe. I think it,s 24p though but you should be fine I think.

Actually It's not a big problem to me to shot 1440x 720, I want to shoot on this resolution to save space in the cards and to get 40 minutes per card

1440*720*25*14/8 = 45.4 M/s 5
am I wrong ?

I'm just wondering if I can get a continious recording for  15 minutes on lexar 128 1000x without having problem like dropping frames



Quote from: Danne on May 29, 2013, 09:29:02 AM
I can record audio and I have put my magic lantern firmware on a SD card(which is very slow, doesn,t matter) so the camera boots magic lantern from the sd-card while records to the CF-card.

why you don't just put your ML in the CF card ?
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 29, 2013, 04:30:08 PM
Quote from: swinxx on May 29, 2013, 03:28:14 PM
Hello ml team!

We all know that when shooting raw and globaldraw is turned on, performance is suffering, but the info from raw histo is great for adjusting exposure (ettr hints ;))

So wouldnt it be possible to add a feature to turn globaldraw automatically of when starting to record?

That would be a great feature!

What i also noticed is, that anamorphic correction only works when globaldraw is set to pn, so i wonder if this menu shouldnt be moved to the globaldraw menu page,

Thx greets swinxx

This is a Feature Request please re post on Feature Request:
http://www.magiclantern.fm/forum/index.php?board=24.0
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on May 30, 2013, 03:45:33 AM
Quote from: romeus on May 29, 2013, 04:24:33 PM
I don't have 5DmIII but I'ill get one next month for a summer project . I will Shot RAW  50% of the project :)
Are you sure that you'll get 6 minutes in 32GB card O_o I think you are wrong I did the math and it's like 11 minute for this resolution (without audio)

You might be right; I only shoot in 1920x1080, and it's a tad over 6 minutes per 32GB.
Title: Re: uncompressed 14-bit RAW video recording
Post by: dubzeebass on May 30, 2013, 04:46:47 AM
Quote from: RenatoPhoto on May 29, 2013, 02:33:18 PM
If you Search for Hack3D you will find this thread:

Hack3D - Speed Improvements for RAW Recording
http://www.magiclantern.fm/forum/index.php?topic=5519.msg37539#msg37539

You got it.  Be Warned

The menu specifically says HaCKeD mode which I asked about, but apparently I'm supposed to know it's Hack3D?
Title: Re: uncompressed 14-bit RAW video recording
Post by: dubzeebass on May 30, 2013, 04:49:33 AM
Quote from: romeus on May 29, 2013, 04:24:33 PM
why you don't just put your ML in the CF card ?

Most of the "This is how to do it" step-by-steps involve putting ML on the SD.  Six of one, half a dozen of another I guess!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Yomommassis on May 30, 2013, 12:16:56 PM
My friends and colleagues keep talking down the RAW video feature to the point where I haven't been too interested in testing it out myself
"Too impractical..Ridiculous file sizes..work flow is too long..not stable..get a real job..."

After shooting a few tests I think they should all go suck a lemon, the play room with these files is totally awesome
Definitely going to try more of this!

Title: Re: uncompressed 14-bit RAW video recording
Post by: squig on May 30, 2013, 12:33:13 PM
LOL. Yeah well some people lead, the rest struggle to keep up.
Title: 5D2 - repeated *buffer underrun-related automatic rec endings* w. dropped frames
Post by: hawkish on May 30, 2013, 04:28:52 PM
Hi - just venturing deeper into these RAW waters w my 5Dmk2 and a KomputerBay 64 GB 1000x
.
I'm experiencing repeated *buffer underrun-related automatic rec endings* w. dropped frames...
@ 1880x720 and even @ 1280x720

Would be great help to have a checklist here (couldn't locate one yet) on *optimizing buffer usage* parameters
(did i read somewhere about 'buffer doubling' or similar..? my screen says not enough memory "109+2048, would need 128" minimum)
and similar to get max. data write values. Altho my cam's benchmarks check shows good basic write >70values,
RAW recordings don't appear to be reliably homogenous  - or am i missing something basic...??

thanks for any swift help!

5Dmk2, KomputerBay 64 GB 1000x, ML nightly builld (13.05.2013)
Title: Re: uncompressed 14-bit RAW video recording
Post by: GregoryOfManhattan on May 30, 2013, 11:32:56 PM
reporting on 50D.
14bit 50D can record raw video at 1536x928 24fps data-rate 57 MB/s up to 4G limit
.raw can be processed to DNGs with raw2dng
(this is with build merged with today's unified e4fc7c7)

beyond 4G limit, spanning files are generated but cannot be processed by modules/raw_rec/raw2dng

more details on 50D and 40D thread http://www.magiclantern.fm/forum/index.php?topic=5586.0
Title: Re: uncompressed 14-bit RAW video recording
Post by: tonybeccar on May 31, 2013, 03:01:09 AM
Hello! I've followed almost all the threads in the forum, and I have one doubt. Given that I don't own a 5D3 and therefore cannot test the firmware.. I wanna ask:

Does the 4GB file splitting work in normal recording mode? Or just in RAW? If not, I guess you are like only a few steps away to implement it, right??

So happy with all the features you guys are making progress!!! Wishing someday it will be available for the 7D :)

Cheers from Argentina!
Title: Re: uncompressed 14-bit RAW video recording
Post by: Drewton on May 31, 2013, 05:54:47 AM
Some comparisons I did shooting RAW on 600D:

[spoiler](http://imageshack.us/a/img28/4074/grassh264.jpg)
H.264

(http://imageshack.us/a/img812/3735/grassraw.jpg)
RAW[/spoiler]

[spoiler](http://imageshack.us/a/img33/1801/treeh264.jpg)
H.264

(http://imageshack.us/a/img268/7421/treeraw.jpg)
RAW[/spoiler]
Title: uncompressed 14-bit RAW video recording
Post by: Redrocks on May 31, 2013, 06:10:35 AM
Quote from: hawkish on May 30, 2013, 04:28:52 PM

5Dmk2, KomputerBay 64 GB 1000x, ML nightly builld (13.05.2013)

That's either an ancient build or a typo, try this:

http://www.magiclantern.fm/forum/index.php?topic=5533.0
Title: Re: uncompressed 14-bit RAW video recording
Post by: nos on May 31, 2013, 01:00:25 PM


another short test with 5dm3. for me komputerbay 32gb 1000x doesn´t reach 1920x1080... 1280x720 works very good without any dropped frames. does anyone use this card and can shoot videos at 1080p?

later i will do a card benchmark to check the speed.
Title: Re: uncompressed 14-bit RAW video recording
Post by: agora1 on May 31, 2013, 05:02:27 PM
it is possible to open 4gig raw movie file canon 5d mark iii
from the latest raw2dng for mac I have this message "Error: This ain't a lv_rec RAW file"
Thanks
Title: Re: uncompressed 14-bit RAW video recording
Post by: dadinio13 on May 31, 2013, 10:25:09 PM
Hi to all!  :)
Firstly i'am a new member so i wanted to thank all the magic lantern team for their outstanding work.
I follow this topic from the first page and the developement of ML on the 5DIII and the project progresses very quickly.
I registered on the site now because I wanted to ask for more information about overexpose the image of raw video recording. Do I only overexpose the image serving me the apperture or what? for example, for highlights recovery is impossible to f2.8 at day with the sun at iso 100 and shutter 1/47 24 FPS?
sorry for my bad English... thank in advance for your help.
And again thank you for all efforts of the devs.
Title: Re: uncompressed 14-bit RAW video recording
Post by: RenatoPhoto on May 31, 2013, 11:17:12 PM
Quote from: dadinio13 on May 31, 2013, 10:25:09 PM
for example, for highlights recovery is impossible to f2.8 at day with the sun at iso 100 and shutter 1/47 24 FPS?

Shoot in video mode and the camera in manual.  Use aperture as you wish.

You will find many Tutorials here: www.magiclantern.fm/forum/index.php?board=15.0

Make sure you read and understand the Forum Rules here:
http://www.magiclantern.fm/forum/index.php?topic=934.msg1141#msg1141
Title: Re: uncompressed 14-bit RAW video recording
Post by: a1ex on June 01, 2013, 12:57:05 AM
Got an idea for squeezing a little more speed:

https://bitbucket.org/hudson/magic-lantern/commits/70d4c7456ed1
https://bitbucket.org/hudson/magic-lantern/commits/63d5922fdcd9

How it works:
- it also crops the source raw stream, hoping that it will reduce the DMA load, resulting in higher write speeds
- limitation: I only know how to crop it on the right edge, so for maximum effect, I had to move the cropping window to the far left. That's why the menu option is called "Framing: Force Left".
- I expect it to have maximum effect in 5x crop mode (where the source stream is ~3750x1350 on 5D3)
- You can combine it with hacked preview mode (which now has grayscale preview, done with CPU to keep DMA free)
- for 5x zoom, move the focus window a few steps to the right; that way, the recording window will be very close to the center of frame.

Also, ML preview is now zoomed, so it should help at lower resolutions.

Title: Re: uncompressed 14-bit RAW video recording
Post by: dadinio13 on June 01, 2013, 01:15:54 AM
Quote from: RenatoPhoto on May 31, 2013, 11:17:12 PM
Shoot in video mode and the camera in manual.  Use aperture as you wish.

You will find many Tutorials here: www.magiclantern.fm/forum/index.php?board=15.0

Make sure you read and understand the Forum Rules here:
http://www.magiclantern.fm/forum/index.php?topic=934.msg1141#msg1141

Thank you for your help! Sur, i read that rules right now ;)
Title: Re: uncompressed 14-bit RAW video recording
Post by: Shield on June 01, 2013, 01:56:15 AM
Quote from: dadinio13 on May 31, 2013, 10:25:09 PM
for example, for highlights recovery is impossible to f2.8 at day with the sun at iso 100 and shutter 1/47 24 FPS?

You need a neutral density filter to knock down the light amount.  Any camera @ F/2.8 (well, any larger sensor camera) is going to be about 4-6 stops overexposed in sunlight.  I use a variable ND filter or shoot at F/8-F/22.  You can adjust your shutter so it's a bit faster too, but watch out for things like flowing water / fountains else it'll be too jerky.

Google and read about ND filters if you want to shoot wide open in the daytime.  My ND filter (and I've owned Singh-Ray) of choice is the Tiffen 82mm ND; just under $200 at B&H and I don't need any step-up rings for my Tamron 24-70 2.8 USD VC.

Shawn
Title: Re: uncompressed 14-bit RAW video recording
Post by: dthost on June 01, 2013, 04:25:03 AM
Quote from: Yomommassis on May 27, 2013, 04:53:26 AM
I posted this in the 5d2 RAW thread but I felt this also applied to any of the cameras

after hearing about the new raw feature I was trying to find a cool practical use for it, and the big thing that came to mind was time lapses without adding shutter actuations

video link....

This test took 1 hour out behind my house, pretty dark aside from the full moon blasting the scene
5D Mark 2 May 20th build
640 ISO
T/1.5
FPS Override 0.178
RAW Video 1880x1250

I had to really adjust the color in post because the RAW kicked the magenta up a lot
Also had artifacts at the bottom of the frame

http://i.imgur.com/G1N6ya8.jpg

and there was this weird problem where every other frame was the first frame...
so instead of capturing 323 I actually captured 162

162 frame time lapse without adding any actuations

Yomommassis: have you done any further testing, and managed to get rid of the repeating first frame? Your still image looks excellent: I've been using the FPS override for timelapse a lot lately (stable build only), and have been very disappointed with the image quality (particularly sharpness)... doesn't cut well with timelapse shot using RAW stills only. Would be happy with a FPS overide that produced "only" 1000 RAW image files to work on later in Lightroom or similar.
Title: Re: uncompressed 14-bit RAW video recording
Post by: DerekDock on June 01, 2013, 04:35:16 AM
Finally got the post processing done on our shooting last weekend. Video looks great but the post workflow is a headache. Will really use this for a shot or two per shoot but never everything unless the clients are paying for it.

Check it out and see the details in the description: http://vimeo.com/67422760
Title: Re: uncompressed 14-bit RAW video recording
Post by: Audionut on June 01, 2013, 06:43:16 AM
Development discussion should continue in this thread (http://www.magiclantern.fm/forum/index.php?topic=5962.0)

General banter should continue in this thread (http://www.magiclantern.fm/forum/index.php?topic=5963.0)