This is a small change in the raw_rec module to enable timelapse video. It simply skips frames during the timelapse frame-interval, hence the slow card I/O is no more an issue.
The 'FPS override' somewhat gives similar results (usually desirable in a low-light condition). However, this frame-skipping option gives us better control over the FPS and the shutter-speed for timelapse, specially in a bright-light condition. It can still be used along with the 'FPS override'.
(http://dhilung.com/vault/ml/Magic_Lantern_frame_interval.jpg)
Test video
[updated to changeset: 7713:80e3ca539891]
raw_rec.c (http://dhilung.com/vault/ml/raw_rec.c)
raw_rec.mo (http://dhilung.com/vault/ml/raw_rec.mo)
Great ! Is it also available on the 5DIII ?
Quote from: SuperSix on June 13, 2013, 12:06:48 PM
Is it also available on the 5DIII ?
I have not tested, but I think it should work just fine in 5D3.
It's nice, but there seems to be a fair bit of flickering going on in the video. Nevertheless, I hope to see this implemented in the main builds, because this eliminates the need for a ND filter to accommodate the shutter speeds imposed by FPS override.
Super, dhilung. I just got through testing this module with my 500D and I am liking it, however there is a white stripe at the bottom of frame and I am wondering if this is due to the resolution (1584x636... which was highest i was allowed) that I used or if it is a bug of this mod. Thanks so much for this, and I am looking forward to using it as my last timelapse wasnt all that I desired. Keep up the good work. By the way, the white stripe is easy enough to cover with a black solid in AE...
~JoshuOne
Why not just enable the intervalometer and silent pics? You get DNGs straight from the camera.
Quote from: a1ex on June 13, 2013, 04:14:52 PM
Why not just enable the intervalometer and silent pics? You get DNGs straight from the camera.
I'm not sure if it's changed now, but the last time I tried it, silent pics came out really soft (even softer than H264 video), with a purplish tint (could be because I screwed the white balance, though it was on auto so I don't see how this could happen), with no ability to shoot with the screen off, and with some frames having issues like having a purple box at the bottom. Also it doesn't seem to shoot at the rate that I set the intervalometer to (it seems a lot slower - setting it to 1s interval resulted in something like 3s interval between pics)
Again, it could have changed since the last time I tested it, so I'm not sure if it has all these issues.
Honestly, a1ex, I do not know any reason not to just use the intervalometer and silent pics, I shot a little test just now to make sure it really was that simple and sure enough it seemed to be. I do think that I was given a limit on the number of frames, and if they are considerably less than those available with this module, that could be a reason. As it is, the straight DNGs from silent pics are missing the white stripe and are one less post-pro step from greatness! Thanks for the suggestion.
~JoshuOne
Quote from: JoshuOne on June 13, 2013, 04:57:35 PM
Honestly, a1ex, I do not know any reason not to just use the intervalometer and silent pics, I shot a little test just now to make sure it really was that simple and sure enough it seemed to be. I do think that I was given a limit on the number of frames, and if they are considerably less than those available with this module, that could be a reason. As it is, the straight DNGs from silent pics are missing the white stripe and are one less post-pro step from greatness! Thanks for the suggestion.
~JoshuOne
I just tested silent pics again today and from what I can tell, it doesn't keep up with the interval I set it to (1s). It may work for longer intervals, but it seems more like 2-3s between each picture from my quick test. This is strange because RAW video has no issues whatsoever doing 1fps video, apart from the 4GB limit, which you can just use the auto restart video feature (In timelapse, it doesn't matter if you miss a few seconds of frames, because depending on your interval, you may lose just 1 or 2 frames, which is not a big deal).
Honestly speaking, converting RAW to DNG doesn't take that long, so if we have this change implemented, I think we can do away with silent pics altogether. Last I checked, silent pics with intervalometer don't generate the post-deflicker XMP or UFRAW files anyway, so I don't see them being more useful than RAW video with this small change.
Also, after a quick test, I did not see the white stripe at the bottom that you described. I'm using the 60D though, so it could be different.
I think this would be most useful if Auto ETTR could work while recording too. I could not get it to work while recording; Auto ETTR only sets the exposure before recording despite setting it to always on (or am I doing something wrong?). It can work manually if you put it to set and press set while recording, but that's dumb because you'll have to repeatedly press set while recording, which is not an option for time lapse videos.
I'd love to try this on my 550D !
My main issue with SilentDNG + Intervalometer is
- the lack of Display off.
In 3sec intervals it just doesn't go off (too short ? )so battery dies quickly
- XMP files arent written, even if ETTR is measured right, so there will be no advantage over FPS override/skip
If XMP creation could be enabled also for SilentDNG I would prefer that option cause its an awesome feature.
Greetings, I just thought of one more reason to use this mod in stead of silent pics, I think we are limited to 1000 frames in silent pics.
~J1
-edit- I am testing now to see if this mod will let me go over 1000 frames or not, I have to go to a one-second or more interval to be able to record at 1584-636 on my 500D without filling the buffer.
-edit- As I suspected, I was able to shoot well over 1000 frames using this frame-skipping mod._
Is it possible to record 4K video at 1fps using the new Magic Lantern RAW video hack? Wouldn't 1fps eliminate the need to sacrifice resolution (due to limited data transfer speeds) in RAW video mode?
Is there a reason this sample video was shot at 1872x1250 and not 2048x1080 or 3840x2160?
Thanks!
Quote from: EOS4K on June 16, 2013, 07:39:32 AM
Is it possible to record 4K video at 1fps using the new Magic Lantern RAW video hack? Wouldn't 1fps eliminate the need to sacrifice resolution (due to limited data transfer speeds) in RAW video mode?
Is there a reason this sample video was shot at 1872x1250 and not 2048x1080 or 3840x2160?
Thanks!
I think they pull the RAW image from the live view buffer or something (at least that's how the 14 bit DNG silent pics work), so the max we can get is the live view resolution, which depends on your camera.
Quote from: brapodam on June 16, 2013, 08:35:10 AM
I think they pull the RAW image from the live view buffer or something (at least that's how the 14 bit DNG silent pics work), so the max we can get is the live view resolution, which depends on your camera.
Is THAT how it works? :o
The 5D Mk III has an LCD resolution of 1.04 dots (read 720x480 pixels). How are people producing RAW video in 1080p and even 2.5K from a Live View screen with DVD resolution? Perhaps I'm missing something here. In any case, I don't see why 4K would be impossible, seeing that 2.5K has already been demonstrated.
Any clarification would be greatly appreciated!
Yes that is how it works.
The RAW videos and silent pics are the images that are read from the sensor for the LiveView preview.
The max resolutions depend on the sensor resolution. My EOS 600D's sensor has a resolution of 5202 X 3465. The higest resolution that's possible for RAW is 1734 X 1155. That's because Canon doesn't read the full sensor and scales it to what ever we want but skips lines and columns to create their desired resolution. In the 600D's case, it skips every third line and column (5202 / 3 = 1734 & 3465 / 3 = 1155) to get a resolution that is close to full HD (1920 X 1080) in terms of width. So it can be used to create full HD H.264 videos.
Only if the developers would find a way to set the lines & columns skipping, higher resolutions might be possible on the 5D Mark III in 1X mode.
But it doesn't look like that is possible.
Quote from: a1ex on June 13, 2013, 04:14:52 PM
Why not just enable the intervalometer and silent pics? You get DNGs straight from the camera.
Yes, that also gives similar results, and I support having one implementation for one feature. However, as of now, I still see at least two advantages while using frame skipping 1) better fps precision at low frame intervals (<2s) 2) easier file management (a single file per timelapse session). Not to mention that all the current and future raw video related features, such as live cropping, can be enjoyed in a timelapse session.
Quote from: CFP on June 20, 2013, 05:19:48 PM
Yes that is how it works.
The RAW videos and silent pics are the images that are read from the sensor for the LiveView preview.
The max resolutions depend on the sensor resolution. My EOS 600D's sensor has a resolution of 5202 X 3465. The higest resolution that's possible for RAW is 1734 X 1155. That's because Canon doesn't read the full sensor and scales it to what ever we want but skips lines and columns to create their desired resolution. In the 600D's case, it skips every third line and column (5202 / 3 = 1734 & 3465 / 3 = 1155) to get a resolution that is close to full HD (1920 X 1080) in terms of width. So it can be used to create full HD H.264 videos.
Only if the developers would find a way to set the lines & columns skipping, higher resolutions might be possible on the 5D Mark III in 1X mode.
But it doesn't look like that is possible.
Thank you, CFP! I really appreciate your astute explanation for the current limitations with RAW HD videos from Canon EOS DSLRs. The 5D Mk III has a maximum sensor resolution of 5760 x 3840. 5760/3 = 1920. What a fabulous coincidence! Other sensors simply don't have such a perfect pixel count.
5DM2 build 25.06.13 can't load module - error (((((
tcc: error: undefined symbol 'raw_lv_share_right' [E] failed to link modules
Quote from: CFP on June 20, 2013, 05:19:48 PM
Only if the developers would find a way to set the lines & columns skipping, higher resolutions might be possible on the 5D Mark III in 1X mode.
But it doesn't look like that is possible.
i found the line skipping parameter already. its ADTG register 0x800C.
but when changing it, the screen seems to "overprint" multiple times.
looks like the bottom scanlines are printed over the top as the image gets larger in vertical dimension.
didnt find the register that handles line count / end address.
@dhilung
Could you post a diff file of your modification?
Better if is it against last commits. :)
Thank you.
Quote from: g3gg0 on July 01, 2013, 01:05:22 PM
i found the line skipping parameter already. its ADTG register 0x800C.
but when changing it, the screen seems to "overprint" multiple times.
looks like the bottom scanlines are printed over the top as the image gets larger in vertical dimension.
didnt find the register that handles line count / end address.
@g3gg0 that sounds exciting!
Quote from: gerk.raisen on July 01, 2013, 01:46:02 PM
@dhilung
Could you post a diff file of your modification?
Better if is it against last commits. :)
Thank you.
@gerk.raisen
Sure :) Here is a diff to the last commit.
(changeset: 7713:80e3ca539891)
90a91,94
> //for timelapse
> static int fskip_choices_x[] = {0,0.1,0.5,1.0,1.5,2.0,3.0,4.0,8.0,15.0,30.0,1.0*60, 2.0*60,5.0*60,10.0*60,30.0*60};
> #define FSKIP_CHOICES_X CHOICES("OFF","1/10 sec","1/2 sec","1 sec","1.5 sec","2 sec","3 sec","4 sec","8 sec","15 sec","30 sec","1 min","2 min","5 min","10 min", "30 min")
>
100a105,106
> //no config yet
> static int fskip_index_x=0;
177a184,185
> static int fcounter=0; /* frame counter for timelapse mode */
> static int fskip_count=0; /* frames to skip in timelapse mode. Calculate in fskip_update.*/
527a536,541
> static MENU_UPDATE_FUNC(fskip_update)
> {
> float fps = fps_get_current_x1000() / 1000.0;
> fskip_count = (int)(fps*fskip_choices_x[fskip_index_x]);
> }
>
1296a1311,1323
> /* frame skipping for timelapse mode */
> if (fskip_count>=1)
> {
> if (fcounter>=fskip_count)
> {
> fcounter=0;
> }else
> {
> fcounter++;
> return 0; //skip this frame
> }
> }
>
1950a1978,1986
> .name = "Frame interval",
> .priv = &fskip_index_x,
> .max = COUNT(fskip_choices_x) - 1,
> .update = fskip_update,
> .help = "Skip frames for timelapse.",
> .choices = FSKIP_CHOICES_X,
>
> },
> {
cant we keep that out of the raw_rec module and add another that handles frame skipping?
like bolt_rec was doing.
this would not take a lot of code. just implement the skip-check routines.
then you can "distribute" your own module that can add dozens of custom frame drop settings and stuff
without having to mess around with the raw_rec code.
(then it is more likely to get into ML source tree)
I still don't see a valid reason for using this and not FPS override or intervalometer.
not sure, but maybe due to the single container file? :)
Quote from: g3gg0 on July 03, 2013, 12:45:50 PM
not sure, but maybe due to the single container file? :)
Exactly, as a time-lapser, I can tell that life gets way easier during post production :) since I don't have to remember or hunt down 'that' transition of sequence. I can treat each timelapse session as a movie session (which essentially is), and directly work on it (e.g., directly import to AE using Ginger HDR etc.)
But again, technically, both ways give the exact same RAW data. This small modification was for a personal convenience, thought might be useful to others. I'll see if making a separate module makes sense.
In this case, I'd rather add a trigger to intervalometer (if raw video is recording, don't take pics, but trigger video frames).
@dhilung
I've tried your code on the last committ on my 60D but unfortunately seems doesn't work :(
The menu entry appear correctly.
I set frame skip to 4 second.
Then the RAW recording started OK but ignored the settings.
It "eat" all the about 150 frames at normal 30fps speed in about 5 second
(FPS override disabled, just to be sure)
@gerk.raisen
It needs to be set while in 'movie mode' (like aspect ratio setting) to be effective. I know it can get confusing :)
@a1ex
Sounds good. I'll look into that. First thing will be to fix the timing issue though.
You now have mili- and microseconds timers ;)
The intervalometer was designed when we only knew about the RTC clock, in seconds (which also took a big bunch of CPU cycles to read).
Is there any way to use this for a time lapse with exposure changes (i.e. sunrise/sunset/day to night time lapse)? I've tried many settings with this mod and just can't get it right. I was never really a time lapse guy but I'm still trying things out. The "demo video" in the first post was also full of flickering :(
-Movie auto exposure screws things up because it uses pushed ISOs which just don't work
-Auto ETTR doesn't work while recording; Manaul ETTR produces flicker
-Gradual Exposure with aperture priority (using the old version of the mod that allowed shooting RAW video in photo mode) goes crazy and produces corrupted frames and still has flicker in the video (does it use pushed ISOs?)
-HDR video doesn't work (only one ISO used)
Am I missing something?
@A1ex
I'm still watching about changing intervalometer to mili- and microseconds timers but it seems too difficult for me alone :( :(
Hint? 8)
@dhilung
Yes, I'm in Movie mode (also because on last versions RAW Video can only work in movie mode)
I tried enable it to 2 or 4 second and than pressed liveview button to star recording
Have you tried your patch also on last committs? Maybe one of the last changes broken it... :(
(FPS override disabled, just to be sure)
Frame skipping disabled (it's correct, isn't it?)
Quote from: gerk.raisen on July 05, 2013, 02:17:29 PM
@dhilung
Yes, I'm in Movie mode (also because on last versions RAW Video can only work in movie mode)
I tried enable it to 2 or 4 second and than pressed liveview button to star recording
Have you tried your patch also on last committs? Maybe one of the last changes broken it... :(
(FPS override disabled, just to be sure)
Frame skipping disabled (it's correct, isn't it?)
I've tried it on a very recent build (ML-RAW-60D-hourly_2013-07-04_00-00.zip) and the mod works.
@brapodam
I've just tested the exact build you write and the "Skipping interval" entry doesn't appear.
That build is compiled "vanilla" without external patches applied I think.
Only the "Frame skipping" entry are present (because it is in the standard ML code and isn't part of a patch)
Quote from: gerk.raisen on July 05, 2013, 02:51:44 PM
@brapodam
I've just tested the exact build you write and the "Skipping interval" entry doesn't appear.
That build is compiled "vanilla" without external patches applied I think.
Only the "Frame skipping" entry are present (because it is in the standard ML code and isn't part of a patch)
Yup, I meant that I installed that build and replaced the raw_rec.mo with the one dhilung made (of course I copied in the raw_rec.c as well). It works that way, at least for the intervals I tested (1s, 4s). Was just a quick test to see if anything changed.
Thank you brapodam,
the problem is the raw_rec.mo posted by dhilung is yet became a little outdated, most of the recent changes in ML are in raw_rec.c
For that I'm interested to apply the patch to last version, to have the feature but with last changes...but one of them kill the feature :(
hey guys!
no news regarding this? It'd be very very useful to have!
Hey! I'm new to the forum, but a long time ML user.
I downloaded this module, and it works great for timelapses (thank you so much!)
No shutter actuation, and RAW frames. It's just brilliant.
However, I was looking to do a timelapse in the night, and have stars move about. I noted that the FPS can't be changed and I was not able to capture the night sky with stars visible. I tried using FPS override but that brought me no luck, either.
So I was wondering -- would it be possible to decrease the shutter speed (ie go around 1-5 seconds per frame) when using this module? Or is there any other method I could follow to have a night timelapse (without shutter actuation/with RAW frames)?
Hey dhilung, sorry unrelated to the module but I was wondering what did you use for panning the video during the timelapse?
Quote from: kazeone on September 24, 2013, 09:06:04 AM
sorry unrelated to the module but I was wondering what did you use for panning the video during the timelapse?
I haven't used anything for panning so far, but I am looking to buy a dolly (or build one lol). :D
Quote from: kazeone on September 24, 2013, 09:06:04 AM
Hey dhilung, sorry unrelated to the module but I was wondering what did you use for panning the video during the timelapse?
I used a Radian (http://goo.gl/IXUCrg (http://goo.gl/IXUCrg)) with an extended arm (like this http://goo.gl/dVInoL (http://goo.gl/dVInoL)).
Quote from: ab19 on September 22, 2013, 01:20:30 PM
Hey! I'm new to the forum, but a long time ML user.
I downloaded this module, and it works great for timelapses (thank you so much!)
No shutter actuation, and RAW frames. It's just brilliant.
However, I was looking to do a timelapse in the night, and have stars move about. I noted that the FPS can't be changed and I was not able to capture the night sky with stars visible. I tried using FPS override but that brought me no luck, either.
So I was wondering -- would it be possible to decrease the shutter speed (ie go around 1-5 seconds per frame) when using this module? Or is there any other method I could follow to have a night timelapse (without shutter actuation/with RAW frames)?
Sorry to hear that it didn't work. Unfortunately, the exposure can be extended only as much as the FPS override allows. You might want to try a high ISO and a fast lens, or fall back to the real shutter actuation.
Quote from: dhilung on September 25, 2013, 10:04:30 AM
I used a Radian (http://goo.gl/IXUCrg (http://goo.gl/IXUCrg)) with an extended arm (like this http://goo.gl/dVInoL (http://goo.gl/dVInoL)).
thanks
Quote from: dhilung on September 25, 2013, 10:04:30 AM
Sorry to hear that it didn't work. Unfortunately, the exposure can be extended only as much as the FPS override allows. You might want to try a high ISO and a fast lens, or fall back to the real shutter actuation.
High ISOs are giving my 550D a hard time in the night at even f/1.8. :P I'll try with ML on my 5D3 soon, as well. In the meanwhile, I'll continue experimenting with FPS override.
Thank you for your reply. :)
hey!
Dear author, I've just tried it on my 5Dmk2 with magiclantern-Nightly.2014Apr02.5D2212 and it failed to load. I guess it's a bit out of date. do you have update for newer versions?
here's the error msg:
tcc: undefined symbol "FIO_Open"
tcc: undefined symbol "recording"
[E] failed to link modules
Quote from: sashk0 on April 11, 2014, 11:16:29 AM
hey!
Dear author, I've just tried it on my 5Dmk2 with magiclantern-Nightly.2014Apr02.5D2212 and it failed to load. I guess it's a bit out of date. do you have update for newer versions?
Yes, it is outdated :p
Unfortunately, I don't have any updated module. However, I'd recommend using regular Intervalometer with "silent picture" enabled (silent.mo) to get a similar output.
Any chance to bring this frame skipping option back into the recent nightly?
It would be very useful for situation where you need fast TV but only want to save every nth frame of the recording without FPS override. FPS override with low light optimization would overexpose the shot, while high jello fastTV optimization would not be fast enough, for example telephoto sunrise and sunset shots without using ND.
I've also noticed that FPS override with high jello fastTV optimization produces duplicated frames in the MLV with the last Aug nightly (probably a few builds prior to that as well, I haven't tested them to be sure), if FPS override is set low at 10fps while recording in 1x mode, 3x crop mode works ok with the same settings. This is on 5D2 212.
Silent shoot with intervalometer can't go lower than 1s, while setting it to shoot like crazy would sometimes yield inconsistent capture interval.
These are the reasons that made me looking for a nth frame skipping raw video recording option.