Messages

hoho, holy shit...
tried setting  PICQPC to 1.
this results in chroma_qp_index_offset being set to 1 and QPY 50, QPU/V 39.
the 1280 x 720 video looks like shot with a 320x240 webcam.
basically every 16x16 block is "worth" a pixel.

same if i set  chroma_qp_index_offset to -1..

when setting PICQPY from 0x1A to 0x19, then the pic_init_qp_minus26 goes down by one from -1 to -2

lol, is that cause PicQpC needs to be set by PicQpY?  Rather can changes be made directly to PicQpC without adjusting PicQpY?
(Forgive me if that question makes no sense cause I'm not entirely sure I even understand what I just said lol)

Anyway is that "720 looks like 240" trick basically what happens when setting a low CBR like 0.3x.  Should we expect a similar loss of quality when setting bitrate that low?

Yes, those are single frames taken from video. Average bitrate for the newsprint was about 150. Peak was 170. For the tree, average was 210, and peak was 260.

I did some high ISO tests with well lit trees last night and got some amazing bitrates - 290 peak and 260 average. It's 35 seconds and 1.04GB.

Thanks Akshayverma!

Those are very valuable numbers, my fastest card is averaging about 150 and peaking at 170 too (with a flush rate of 2). 

I found the best way to max out bitrate (for testing purposes only) is to set ISO to max (ISO 128,000) with a fast shutter (250 - 4000 depending on lighting) and point your camera at something that spotmeters under 75 on the 0-255 scale; this will shoot your bitrate up to max right quick!

Set to 20x with a Flush rate of 2 my "200x CF" card (benchmarked at 21 MB/s) can record 200 Mbps for one second after filling the buffer and shutting down recording but I found I could stably record anything to the full 4GB limit at a fairly constant 160 Mbps, which is the peak limit for 3.5x :/ guess I'll need those "600x CF" cards after all lol

I'll try some 160 Mbps high noise and tree details later today lol.

It's ~f/6.9 for the 7D due to the 18 MP sensor. However, with 1080p video, you're then talking about effectively a ~2.1 MP sensor so the airy disk can technically be larger. On top of that, since the video is created with line skipping, anything over f/7.1 may still in hibit resolution. One more thing I want to look into aside from the pixel-peeping effects of the VAF-7D.

Thanks Digital! You're always getting me to double check my work and learn more lol

You are right the Ariy disk does begin to overlap pixels around f/6.3 but it isn't really noticeable until f/8 so most people consider that diffraction limited.

Unfortunately I think with "line skipping" to down-res the image to 2.1MPs we do not benefit from an increased diffraction limit like we did with Photo mode's pixel binding method which physically makes each pixel sensor larger rather then in the line-skipping method we still have the same sized (4.3micron in this case) pixel sensors :/

Shorter GOPs are better for recording really fast or really erratic movements (like race cars) because it just records everything without having to try to "predict"your subjects movements (unlike with IPP and IPB) so there is less chance of artifacts caused by prediction errors (like ghosting). 

ALL-I will record the subject's movement more accurately but that needs more bandwidth.  If it doesn't have enough bandwidth, you'll still get the accurate movements but will get far more compression errors (like blockiness).

80% of the time I'd choose IPB (or IPP) but for those times you need "All-I" it's great to have, you need the bandwidth to use it though.

Also "All-I" is easier to edit and color grade cause each frame has its own pixels rather than having to depend on the frame that came before, or people can just transcode to an Intra codec in post

@g3gg0
Ah, I see.  So "Stock" would be like Flush = 24, which would flush the buffer every 24 frames which also happens to be every second when the recording is set to 24fps. 

So setting the Flush to 4 would flush the buffer every 4th frame instead of every second, yeah?

So it's clearing the buffer faster than once per second now right?

Thanks for clearing that up

...I don't have the video analysis tools but by looking at the video properties I found the following bitrates for
all my test subjects:

CBR       IBP        ALL-I
            kbps        kbps

No ML    97573        54588
1x          49541      43228
2x          88795        95941
3x          89605        150578
5x          90403        228832
10X        91294        268863
15X        96169        272493
20X        98747        243947

...

Note that the standard 7D video has a bitrates of 97573 kbps which is close to the 20X bitrate.  In All-I video the bitrates
go significantly higher but the artifacts show big time until you get above 2x and thereafter I do not see any improvements
as bitrates go up.

...

Any help to clarify this mystery would be greatly appreciated.

Something is not right here;  7D with no ML isn't 97 Mbps (h.264 Baseline 5 has it capped at 50 Mbps) and the 7D doesn't have "ALL-I" without Magic Lantern and never had IPB even with ML.

Yes "All-I" on the 5D3 will likely have less detailed image quality than its IPB mode.  We touched upon that in this thread: http://www.magiclantern.fm/forum/index.php?topic=3103.400

597 Mbps for a true 1:1 compression of a 1080p 4:2:0 image at 24fps
35 Mbps for the average 17:1 compression with "IPP" (Which is Canon's stock settings)
86 Mbps for the average 7:1 compression used for "All-I" (Canon gives around 95 Mbps for the 5D3 "All-I" mode)
and
We've seen the 7D maxed at about 250 Mbps which is about a 2.5:1 compression!

...since All-I "does less compression work" and since most compression artifacts and glitches happens due to round off errors during compression it would be expected that All-I would reduce the opportunity for artifacts. 

The problem is that All-I needs quite a bit more bandwidth than IPP and if it doesn't get it then most fine details get lost in the "All-I process". For example, the GH2 with h.264 IPB at 88 Mbps gave a near 1:1 compression look but the 5D3 with h.264 All-I at 95 Mbps didn't.

Being VERY untechnical about it, from what I've "heard":
ALL-I at 100 Mbps would look similar to IPP at 50 Mbps and IPB at 25 Mbps

Just to be clear, is a lower number for the "flush rate" speeding up the buffer? Is Flush=1 the fastest or is it Flush=20 and what is Canon's standard flush rate? 

Cause whenever I try to record with the flush rate any lower than 3 I get an err70 and when I set flush to 20 I see my buffer percentage stabilizes around 30% but when Flush is set to 4 it stabilizes around 5%.

Anyway my 1st CF card is benchmarking at 15MB/s (8, 8, 15) I set to CBR 20x and Flush to 3; it buffers out at 100 Mbps Average (125 Mbps peak instant) which is only equal to CBR 2.5x.  Not a very good card for testing high bitrates but I got another one that's benchmarking at 45MB/s that I'll try later.

I sharpened these with After Effects' built in unsharp mask at 200% (wanted to over sharpen a bit), radius: 1px, threshold:0.
Didn't use any anti-aliasing. I guess it's not there because the details are rather large (the pictures aren't cropped). Ink droplets on a newsprint are quite visible.

Hey Akshayverma,

Thanks for the tests what bitrates were you averaging and peaking at for those jpegs?  And I assume those were single frames taken from a video right?

@P337: Yes! I agree anything lower than f/8 would soften the footage, but a still image taken at the same aperture isn't that soft. I think it's because of the line skipping, as ilguercio pointed out. This time I shot at f/5.6 (which I think is the sharpest for this camera, or even f/6.3) and it's still not sharp.

Right, line skipping will also effect it as well as the 4:2:0 subsampling and frame compressions.  Even at 1:1 compression rates, about 597 Mbps, it would still only have half the chroma pixels :/ So I doubt it will ever equal to photo image quality

I'm using a SanDisk Extreme 8GB 60mbps card. I just shot some trees.

ISO 160
Shutter 1/250
Aperture f/11
Flush rate: 4
CBR 20.0x
Focus peaking, magic zoom, histogram, waveform and bitrate display were ON
Audio OFF

I recorded 1 minute, 42 seconds of handheld footage. Recording was stopped by me (it would keep recording for the whole 4GB). The file size is 2.38GB.

As g3gg0 rightly said, it's actually VBR. I got a maximum bitrate of 218 while the average was around 170. Although I focused it pin-sharp, the footage is blurry as is all 7D footage. Visually it looks just like ordinary 1x BR to me.

Could anyone please guide me about doing a better comparison?

It might help to know that once you hit an aperture of f/8 the pixels in the 7D start to diffract which will blur the image a bit more and more the further you go passed that threshold.

btw - its up to nearly 300MBit/s

!

...2/1 compression

uuuhhhh errrh. too much text for me

what is the advantage of all-I anyway?
about 90% of the P-frames are I-Macroblocks anyway.

setting GOP to 1 will enforce that all blocks (even ones that can be predicted very good) are definitely I-blocks.
this will improve only 10% of the single frames a little.
whereas increasing bit rate will improve quality of all macroblocks (I and P).

combining both would maybe give the best results.


@1%:
where did you patch the 600D to get all-I?
i found the GOP size variable but there are several checks for 12 and 15.
i would rather patch that somewhere in encoder routines so that canon firmware doesnt see that change.

@g3gg0
Well I agree that higher bitrates for IPP is more important than All-I, but since All-I "does less compression work" and since most compression artifact and glitches happens due to round off errors during compression it would be expected that All-I would reduce the opportunity for artifacts. 

The problem is that All-I needs quite a bit more bandwidth than IPP and if it doesn't get it then most fine details get lost in the "All-I process". For example, the GH2 with h.264 IPB at 88 Mbps gave a near 1:1 compression look but the 5D3 with h.264 All-I at 95 Mbps didn't.

Also about that GOP sizes, according to this: (http://learn.usa.canon.com/resources/articles/2012/ipp_ipb_all_i_compare.htmlp
) Canon's Interframe Codec uses GOP12 for 24fps and GOP15 for 30fps or 60fps.

The IPP compression method is used in many existing EOS models like the Canon EOS 5D Mark II, EOS 7D, 60D, and Rebel series models. This compresses video files in a way similar to that described immediately above. At 30 fps, the first frame in a group of 15 is a key frame, and the next 14 are predicted, based entirely on the data from the previous key frame and preceding predicted frames. With this compression method, each group of 15 frames (at 30 fps) is stored in what is known as a Group of Pictures (GOP). The drawback to this method is that frame-by-frame editing often results in slightly lower image quality.

Your initial math up on top is perfect so I don't know where you psyched yourself out and deviated from what you had. You summed your 8 bits per pixel before you multiplied them. Thus you computed a video frame that had a 24-bit depth. Damn, I *wish* the 7D could do that kind of dynamic range . A logical overview could explain the reasoning of your number based on doing file size calculations for the rendered frame in RGB, however. Each channel has an 8-bit color depth so you want to multiply each channel by 8 (2^8 = 256 which we love/hate for dynamic range). So...

Thanks again Digital Corpus!
I got lazy and added the resolutions together instead of the actual pixels lol
~((1920x1080+(960x540)*2) instead of ((2,073,600+(518,400*2)) 

I also adjusted the equation to keep all the channels 8bit, which sounds right, but still seems counter intuitive to me lol. (Can anyone explain why 3 8bit channels working together in one pixel still equals 8 bits per pixel?)

so now I get:
597 Mbps for a true 1:1 compression of a 1080p 4:2:0 image at 24fps
35 Mbps for the average 17:1 compression with "IPP" (Which is Canon's stock settings)
86 Mbps for the average 7:1 compression used for "All-I" (Canon gives around 95 Mbps for the 5D3 "All-I" mode)
and
We've seen the 7D maxed at about 250 Mbps which is about a 2.5:1 compression!

[So 4:2:0 color encoding requires 597 Mbps to be recored "uncompressed".]

Now remember me mentioning the 1 bit per pixel (bpp for short) for lossy encoded video? Take out number above, convert the 8 bits to 1 bit by dividing by 8, and that gives us a theoretical maximum bit rate of 74,649,600 bps or ~74.7 Mbps for optimal imformation capture. Granted, there are inefficiencies so going up to 100 Mbps should be okay, but 200-250 Mbps might be a touch excessive .

Thanks Digital Corpus!  You have obviously done a lot of homework on the subject but as you said each channel is 8-bit and each pixel has 3 channels (Y' Cb and Cr) so that's 24bits per pixel (3 8bit channels = 8+8+8) so wouldn't that be dividing by 24 in that theory? and I think it's Bpp(1 Byte per pixel) not bpp(1 bit per pixel) it's, but Bpp is uncompressed.

Anyway I tried crunching the numbers myself and this what I got:
Frame size: 1920x1080 resolution = 2,073,600 pixels

8bit Y' Channel: 1920x1080 resolution = 2,073,600 pixels * 8bits per pixel = 16,588,800 bits per frame
8bit Cb Channel: 960x540 resolution = 518,400 pixels * 8bits per pixel = 4,147,200 bits per frame
8bit Cr Channel: 960x540 resolution = 518,400 pixels * 8bits per pixel = 4,147,200 bits per frame

Total "YUV" Channels: (2,073,600 Y' pixels + 518,400 Cb pixels +518,400 Cr pixels) = 3,110,400 pixels * 8bits per pixel = 24,883,200 bits per frame (OR 3,110,400 Bytes which is 1 Byte per pixel)

Frame rate: 23.976 frames per second * 24,883,200 bits per frame = 596,599,603.2 bits per second

So 4:2:0 color encoding requires 597 Mbps to be recored "uncompressed".

Now H.264 has some compression method tricks up it's sleeve that I won't get into (cause I don't fully understand them all lol) but I've heard the average ratios are 7:1 compression for each "I-Frame", 20:1 compression for each "P-Frame" and 50:1 for each "B-Frame".  So assuming those are correct, average data rates for H.264 are:

H.264 I-Frame Compression: 24,883,200 bits per frame / 7 = 3,554,743 bits per frame (rounded up)
H.264 P-Frame Compression: 24,883,200 bits per frame / 20 = 1,244,160 bits per frame
H.264 B-Frame Compression: 24,883,200 bits per frame / 50 =  497,660 bits per frame

The 7D's H.264's compression mode is IPP and it records 24fps in GOP12 like this (IPPPPPPPPPPPIPPPPPPPPPPP for each second). That's 2 "I-Frames" and 22 "P-Frames" per second:

24fps IPP I-Frame data rate: 3,554,743 bits per "I-Frame" * 2 = 7,109,486 bits per second
24fps IPP P-Frame data rate: 1,244,160 bits per "P-Frame" * 22 = 27,371,520 bits per second
Total 24fps IPP data rate: (2 I-Frames) 7,109,486 bits per second + (22 P-Frames) 27,371,520 bits per second = 34,481,006 bits per second

So, if those "average compression ratios" for H.264 are correct then H.264 IPP at 24fps requires an average of 35 Mbps, a 17.4:1 compression of 1080p 4:2:0.  If we get "All-I" (Intra) working the average compression rate that should be used is 85,313,832 bits per second (86 Mbps) for a 7:1 compression.  Remember to get 4:2:0 subsampling to a 1:1 compression rate requires 597 Mbps.  So if we get Intra at 200Mbps that should give us a 1080p 4:2:0 image at about a 3:1 compression ratio ;-D

I'm pretty sure it's the 5D3, for starters the 7D is 32bit rather than 64bit (so no getting around the 4GB limit file sizes) but I do consider the 7D a close second now that it has ML, Thanks again g3gg0

It should be noted that for the same bitrate, shorter GOPs yield significantly poorer compression. GOP=18 is more efficient.

You mean requires higher compression rates.

Right, cause you're leaving less information headroom for each frame, that's why I think a higher bitrate is more important than changing GOP; a higher bitrate means less compression so you get fewer compression artifacts and more detail while shorter GOP means less predicted frames so you get more accurate fast movements and more natural motion blur.

Thanks for looking out Shizuka

Ah this guy shot with a ML beta 600D in All-I mode! And apparently I heard wrong; stock Canon encoding is GOP12 not GOP15. 24fps recordings are GOP12 while 30fps and 60fps recordings are GOP15

"Gettin' High - Super Neutral Log Picture Style Test" on Vimeo http://vimeo.com/andy600/gettinhigh

What means GOP 3?

It means only 2 frames are being predicted per "group of pictures" instead of 14 in the standard IPP compression.  "All-I" means no predicted frames so "All-I" is GOP1

GOP 3 with EOS 600D

http://vimeo.com/49988361

Oh cool, I didn't realize ML already did GOP controls!  GOP3 is close enough to ALL I for me, hope this can port to the 7D!

The GOP is it changed?

Very good job.
Thank you.

I assume it's still IPP, which (correct me if I'm wrong) is really only a concern for recording fast erratic movements.  Increasing bitrate helps resolve higher levels of detail in the image which I think is more important. (Though I've also heard the 5D3's "ALL-I mode" helps reduce morie and aliasing but haven't seen for myself)

I remember reading somewhere that in Canon's encoding process, 1 out of 15 frames are fully recorded and the rest attempt to "predict" the frame; in the "predicted frames" only the changes from the previous frame are recorded and anything that did not change is basically copied and pasted to the next frame so adding "B-Frames"(IPB) to that mix would be very helpful for those "Predicted frames" but I think that would require messing with Canon's code which ML doesn't do.  Making it Intra however might be easier by telling the encoding "OMG every single pixel changed since the last frame, so quick record it all!" for every frame lol.

But these are all guesses on my part, I'm not an engineer just a observant user

If it is possible to mitigate the 4gb limit and shoot continuously after hitting 4gb: yes! If it's not, well, we'll just have to shoot shorter scenes, so: YES!

Well at about 245 Mbps that's like 2 minutes of footage per 4GBs (if it's a constant bitrate that is) but that should be enough for one scene   and we would need at least a 30MB/s write speed so something like a "500x" 40MB/s Card would be ideal.  I've seen them sold for about $2.50 per gig so you could think of it as $5 per minute

Any chance of seeing a resolution chart video with the 7D at 200mbps :-D

AWESOOOOOOOOME!

Most probably not. Magic Lantern does not copy Canon code, see FAQ.

Ah I see, so the feature isn't already in there just limited to f/5.6, this would actually require writing/re-writing new code which Canon now owns.

Ok, thanks anyway.

So Canon's next 5D3 firmware update will include Clean 4:2:2 HDMI out and center point Phase Detect AF for f/8 which they've also recently enabled on the 1DX with a firmware update.

My question is, will there be any possibility of enabling f/8 Phase Detect Auto Focusing for the 7D through Magic Lantern?

Thanks for all your work! ;-D   

Hey guys, haven't had the chance to thank g3gg0 and the team yet, thanks guys! 

In hopes to clear up the AGC concerns, AGC is just Auto Sound Levels (Auto Gain Control) which can be useful but the preamps in these DSLRs are too noisey so when set to high levels (whenever it got quiet) you would hear a "hiss" in the background.  7D's 2.0 firmware gave us Manual control over the sound recording so now it won't automatically jump up and introduce that "hiss", just be sure to manually set your levels low.  What he may be referring to though is the "Digital" and "Analog" Audio Gain found in the other versions of Magic Lantern.


This build is working great, just one bug I found.  Whenever I turn my Canon EOS 7D Camera Body off then back on again Magic Lantern disappears! It's like it's only stored in the Camera's RAM or something... Just Kidding ;-P

I did find a potential crash though, but haven't been able to reproduce it.

I had all the Magic Lantern Video features on (except False Color and Ghost Image) and I had Trap Focus enabled.  I was switching between Canon's "Custom User Settings" (C1 C2 C3) to take video and another to take pictures.  When switching back from C3 (my video settings) to C2 (my photo settings) suddenly Magic Lantern was gone.  I was reverted to the official Canon 2.0.3 firmware but just had to "re-upgrade" to Magic Lantern.  Everything was back, none of my Magic Lantern setting or Canon Menu setting were effected.  I assume the RAM got dumped when switching from C3 to C2, maybe I didn't realize the card was busy when I switched and that caused it but I haven't been able to reproduce it yet.  Also I had my Canon Menu "auto power off" feature off so I know it didn't go idle.