Magic Lantern Forum

Developing Magic Lantern => Feature Requests => Topic started by: DigitalVeil on January 05, 2015, 06:30:00 PM

Title: Request: h264 GOP control in ML
Post by: DigitalVeil on January 05, 2015, 06:30:00 PM
Before bit rate control was merged from TL back into the unified ML branch, it was possible to adjust more individual parameters like slice and GOP.  Slice is now part of 'Qscale' but unfortunately the ability to change GOP has been removed.  It would be really nice if GOP control could be brought back in ML for at least the TL-supported cameras that originally had it, and possibly others as well.
Title: Re: Request: h264 GOP control in ML
Post by: Rshred on January 06, 2015, 02:39:06 AM
I agree! It would be awesome to have more control over the bitrate like that old tragic lantern build for the 600d. The only reason that I don't use that old build is because of new features like dual iso (for photos), ettr and mlvraw video. I would be nice to have all of these features combined!  :)
Title: Re: Request: h264 GOP control in ML
Post by: tupp on January 06, 2015, 06:38:39 AM
Request for GOP control seconded!
Title: Re: Request: h264 GOP control in ML
Post by: DigitalVeil on January 13, 2015, 06:43:45 AM
I really dont think it would be that difficult to bring the old code into the latest ML builds.  I would also like to see the 3.0x limit removed, basically allowing you to raise it until your particular camera can't write any faster.  I can't even think of a reason why these things would be removed in the first place, unless someone thought they were useless.  But just because it's useless to one person doesn't mean it's useless to everyone.  For those of us who can't shoot 1080p raw it would be nice to at least record 1080p h264 up to 40MB/s, setting GOP to All-I and QP to the highest you can get.

The old TL code is still readily available in the TL repo, and I think the 7D 20.0x bitrate experimental code is still in the ML repo as well.  My programming knowledge is shaky and limited and I'm still just beginning to familiarize myself with the code base so I'm nowhere near confident in my ability to do it, but if nobody else does it I might just try to make my own 700D and 70D builds with the old bit rate control code.
Title: Re: Request: h264 GOP control in ML
Post by: ItsMeLenny on January 13, 2015, 11:37:56 AM
Well I think once you hit 3.0x bitrate you might as well just switch to VBR(QScale).
Title: Re: Request: h264 GOP control in ML
Post by: DigitalVeil on January 13, 2015, 02:56:33 PM
Which I do, although I'm not sure Qscale -16 actually sets the highest possible QPs
Title: Re: Request: h264 GOP control in ML
Post by: dmilligan on January 13, 2015, 02:58:15 PM
AFAIK, the reason this stuff is not in ML is (this is just my understanding, someone correct me if I'm wrong):

- Just setting these h.264 settings causes Canon firmware to throw lots of errors.

- Using "cache hacks" (patching values in the cache to override Canon firmware behavior), 1% suppressed these errors to allow the settings to actually work.

- a1ex and other devs consider it "dangerous" to patch out and ignore these errors, without understanding *why* the Canon firmware is throwing them. Also, all this cache hacking code is very hard to understand.

Here's a summary from a1ex's code review of TL from a long time back. I've quoted the relevant sections below.
http://www.magiclantern.fm/forum/index.php?topic=9510.msg78084#msg78084
Quote from: a1ex
As I'm currently in the middle of porting some changes from Tragic Lantern, here's my detailed code review, based on this diff (ML 4b3129d vs TL 7a5ccc6). I hope it answers the questions about safety and about the differences between TL and ML.

...

Major concerns:

- In many places, Canon error checking code is disabled (examples: beep.c or bitrate-6d.c, especially the second one). This is a no-no for me, and it's like running with a bike downhill at full speed without brakes. Just try understanding that code if you don't believe me.

....

- Error handling in bitrate code is done by rebooting the camera in the middle of recording!

Minor concerns

- no checks before patching Canon code
...

Conclusion: personally I'm afraid to run Tragic Lantern on any of my cameras. Sure, I don't remember anyone bricking his camera with TL code, but I've learned about some dangerous things the hard way, by bricking my own cameras (and then learning how unbricking them). Of course, you should not trust me blindly, you should review the code on your own (even the code from the main repo).

Sorry if I sound like bashing or underestimating the contributions from 1%. I've just tried to do an objective review of Tragic Lantern, especially regarding code safety, and I've tried to back every argument with a link or a code snippet. These are the main reasons TL code didn't get merged into main repo; I've pointed them out many times, but the issues were not solved, so a little reminder shouldn't hurt.

Now, I have to admit I don't have a better solution. The main repo for 6D (and other cameras, e.g. EOS M) was not touched for months, so it's completely untested. I can't help with that, because I don't own these cameras, and I already have too many of them). I just want people follow some really basic development guidelines, and submit their changes to the main repository, where they will be subject to code review from me, g3gg0, nanomad and many others - both devs and nondevs). Now, the entire 6D/EOSM/7D development is done in a fork where the core ML developers have absolutely no influence, and most people are not aware of that.


So, it's not really a matter of getting the code to work, that's easy, it already works. If you want to see this in ML, then you need to prove that this stuff is not 'dangerous' (just testing it and saying 'hey it works for me' is not good enough). That will require a lot of work and reverse engineering/analyzing the Canon firmware.
Title: Re: Request: h264 GOP control in ML
Post by: DigitalVeil on January 13, 2015, 04:05:56 PM
Hmmm.  Thanks for digging that up for me!  I'm usually one to throw caution to the wind, but I understand that risky disposition should not be forced upon the entire ML user base by putting it in the unified branch.  Looks like if I ever want it, I'll have to try making my own build and maybe share it on a per-person basis as long as they acknowledge that they understand the risks.  At least for now a few models support GOP 1 in the default FW.
Title: Re: Request: h264 GOP control in ML
Post by: walter_schulz on January 13, 2015, 04:20:35 PM
I think you might want have a private talk with 1% about your plans. I'm afraid you just caused neck hair issues to some people.
Best intentions given, of course.
See
http://www.magiclantern.fm/forum/index.php?topic=11080
Title: Re: Request: h264 GOP control in ML
Post by: DigitalVeil on January 13, 2015, 06:04:17 PM
Right.  I certainly don't wanna cause another community split or anything like that.  It would just be a personal build really (if I could actually copy it successfully).
Title: Re: Request: h264 GOP control in ML
Post by: DeafEyeJedi on January 13, 2015, 09:26:10 PM
@dmilligan --Thanks for the enormous info regarding this particular requests... Definitely not worth fiddling with this due the points that @a1ex made earlier which I'm afraid it simply puts the foot on the floor!

CBR's can be a fun feature but is it really worth esp with the cons outweighing the pros?
Title: Re: Request: h264 GOP control in ML
Post by: DigitalVeil on January 14, 2015, 03:57:30 AM
Quote from: DeafEyeJedi on January 13, 2015, 09:26:10 PM
@dmilligan --Thanks for the enormous info regarding this particular requests... Definitely not worth fiddling with this due the points that @a1ex made earlier which I'm afraid it simply puts the foot on the floor!

CBR's can be a fun feature but is it really worth esp with the cons outweighing the pros?

Here's my personal take on it: yes, error checks were bypassed and it's not 100% "safe".  But it never bricked anybody's camera when it was being used in TL.  To me, it is worth it.  I'd like to squeeze every last drop of quality out of 1080p h264 that I can get, because I really don't like working with sub-1080p resolutions and I don't have $3.5K for a 5D3.  But like I said, I'm not a risk-averse person and I definitely understand now why it shouldn't be in the main public build.
Title: Re: Request: h264 GOP control in ML
Post by: pursehouse on January 27, 2015, 10:02:46 PM
yeah, GOP control would be nice... it would be nice to control all the settings that are possible to change. That way people can play with the settings as they see fit. How compressed things can get depends on how fast the encoding cpu(gpu?) is in the camera
Title: Re: Request: h264 GOP control in ML
Post by: ZachGoodwin on February 17, 2015, 06:52:48 AM
I agree with this, because I have seen the results right here:

https://www.youtube.com/watch?v=pchhGTB3VvI