Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - alMalsamo

#1
I got one of the unbootable KomputerBay 1050x 128GB CF cards, and it seems a very unconsistent performer in RAW recording benchmarks as well.  Even at 24 fps 1080p with no audio and Global Draw off, would I very often receive dropped frames early on.  I am going to send the card back to give another one a second chance.  In e-mail correspondance with KomputerBay, they claim it is impossible to acheive 1080p RAW recording at 30 fps with the 5D Mark III:


QuoteYou will not be able to achieve 1920X1080 @ 30fps without spanning. The max you can achieve will be 25fps. 30fps required a little more than 100MB/s (106~108MB/s) and the CF slot on the 5DM3 is limited to 100MB/s. You will get 2-34frames before your camera heats up and shuts down (also not recommended). The problem is not the card - but the camera's bottleneck. The benchmark we sent you is was for the 128Gb card - as you can see it will very well achieve above 110MB/s on the 5DM3 but the camera itself is limited to 100MB/s for continuous raw.


I may have missed this official discovery here on the forums, but I hope that they are incorrect!  UDMA 7 specification allows for 167 MB/s of throughput with CF version 6 specification, as far as I am aware.  It would be disheartening to hear that we could never acheive RAW 1080p/30 recording using the CF bus on the Mk3.

When I receive my replacement card, I will post benchmarks to this thread.  I hope to report good news!
#2
Quote from: Audionut on March 21, 2014, 02:26:11 AM
That has been possible for quite a long time.

Is is possible when recording to H.264?  Because that is what this feature request is about.


Quote from: Audionut on March 21, 2014, 02:26:11 AM
If you are unable to do things yourself, then you should probably not make assumptions about things being trivial.  Whether that assumption is based on conditions, or not.

I believe overcoming the barrier to saving dropped frames at the 4 GiB limit and implementing a solution where we could select recording to > 4 GiB files in the ML menu for H.264 mode as well is absolutely non-trivial.  I am not able to come up with a solution to this myself, not by a long shot.

However, if someone else comes up with a solution, I'd happily submit a patch to disable the option for cameras that do not support exFAT.  This is comparatively trivial, and within my abilities to lend a helping hand on.
#3
Quote from: Audionut on March 20, 2014, 05:51:41 AM
> 4GB recording capacity is pretty pointless without the removal of time limits, and to a lesser extent, vice versa.  The discussion about the removal of limits in H.264 encoding, is perfectly fine, contained in 1 thread.

I do not consider it pointless to overcome 4 GiB file limits without removing the 30 minute time limit in the 5D Mark III.  The time limit is at 29:59, but frames are already dropped well before this limit is reached.  Being able to actually have a continuous recording to 29:59 with NO dropped frames would already be an improvement. 

Furthermore, because it will probably be a different solution to recovering the lost frames at 4GiB file changes, I imagined that focusing on one barrier at a time would maybe be more efficient.  I am also open to a full-blown discussion for obtaining true continuous H.264 recording with no dropped frames, because that is an ideal end goal.


Quote from: Audionut on March 20, 2014, 05:51:41 AM
Quote from: alMalsamo on March 19, 2014, 10:40:48 PM
If the barrier to writing >4 GiB H.264 files can be overcome, then making this feature unavailable to builds of ML for non-exFAT cameras should be trivial.
As a1ex would say, if it's trivial, we look forward to your patch  ;)

My statement was conditional.  I said if the 4GiB barrier can be overcome, then making it unavailable to non-exFAT cameras would be trivial.  I didn't say designing or implementing a solution in the ML codebase is trivial.


Quote from: Audionut on March 20, 2014, 05:51:41 AM
For exfat enabled cameras, yes that would be the ideal solution.  Who cares about non exfat cameras anyway!   :o
Oh, and actually, non exfat cameras do have > 4GB raw capabilities.  It's called file splitting.  And since it's not limited in the same manner as H.264 encoding, there is no frame loss.

Honestly, any solution that yields no dropped frames would be a huge improvement.  Even if there are muliple files to deal with in post-production, we could at least write and distribute scripts to join continuous recordings prior to editing with no quality loss.  The crux of the problem is recovering the dropped frames during recording.  The discussion would ideally be centered first around recovering dropped frames, and only secondarily about how most efficiently to join them into a continuous file that could exceed 4 GiB.
#4
Quote from: g3gg0 on March 19, 2014, 12:46:15 AM
i talk about canon firmware patching. and solved means that we can patch that point in the firmware.

Ah, then I misunderstood what you referring to.

Quote from: g3gg0 on March 19, 2014, 12:46:15 AM
this resigned position contradicts to the euphoric last words in your post:
"to crack the 30-min limitation (obsoleting the need for the Movie Restart hack) would be a revolutionary feature.  Full frame cameras with unlimited continuous recording in both H.264 and RAW without dropped frames is a powerful end goal."

i dont get what you want with 16 min and 30 min frame drop.

Continuous recording with no dropped frames is the end goal, and right now on the 5D Mark III (probably the 6D as well?) there are two barriers to that end goal.  The first barrier is what this feature request covers: the 4 dropped frames when a 4GiB file finishes writing and recording moves on to the next file.  The second barrier is the forced stopping of recording at the 30 minute mark to put the camera in a different tax category.  The Movie Restart feature ML provides is a nice (but very imperfect) workaround for the latter, but yields dozens of dropped frames instead of only four.

Because the technological and "political" background for both of these barriers are discrete, I would think it best to have a feature request and subsequent discussion for both of them individually.  From my limited understandings of the inner workings of things here, I imagine the solution for having no dropped frames through both aforementioned barriers would be different as well.


Quote from: g3gg0 on March 19, 2014, 12:46:15 AM
no, why are you stating things as facts which you have not proven?
if you want the skinny details just ask and dont try to prove me wrong.

I actually do want as much details as possible about the situation so that I and anyone else who may be interested in the end goal of true continuous recording can become more educated about how best to overcome the current barriers.  I did not  try to prove you wrong... I was only stating the fact that there is no 4GiB limitation inherent in the H.264 (MPEG-4 AVC) video format itself.  While you are clearly much more knowledgeable than most about the issues as they relate to H.264 recording in Canon cameras, I just wanted to make clear to anyone who might have mistakenly took your words literally.


Quote from: g3gg0 on March 19, 2014, 12:46:15 AM
H264 videos on canon cameras are in a quicktime container.
this container format was not designed for file sizes >4GiB until somewhen in 2000 when quicktime 4.0 was released.
stco sections were simply just 32 bit offsets. later co64 chunks were added with that quicktime version.

canon uses the standard 32 bit set of atoms like stco.
this is a legit decision as it saves from a lot of trouble. especially from users.

file size limitation was already solved in A)

Thank you for being more explicit, as some of this information was new to me and helps me understand the situation on a deeper level.


Quote from: g3gg0 on March 19, 2014, 12:46:15 AM
additional encoders. well. we talk about a highly embedded, high volume consumer device.
do you expect an intel pentium inside?
there is nothing to upgrade. it is pure hardware.

On second thought, it is naïve to ask about support for encoding to other formats that the cameras do not have hardware acceleration support built-in.  Scratch the questions about other formats, but I am still not knowledgable enough to know if a different H.264 encoder (such as x264) would be able to take advantage of the camera's H.264 hardware encoding acceleration, or if that would even help in overcoming the barriers to continuous recording.  Insight from those more knowledgable clearing up either point would be appreciated.


Quote from: g3gg0 on March 19, 2014, 12:46:15 AM
again it is not about mlv_rec, its about H264 recording.
no exFAT = no files >4Gib.
and for people who want >4GiB H264 files, this is in fact an issue.


When I said I thought it wasn't a huge issue, I meant it shouldn't be an issue in terms of implementing a solution in the ML codebase.  If the barrier to writing >4 GiB H.264 files can be overcome, then making this feature unavailable to builds of ML for non-exFAT cameras should be trivial.  It should be as obvious to the users with older cameras that they can't use the >4GiB file feature with H.264 as it is that they can't use the >4GiB file feature with RAW recording.  Making the options simply unavailable to non-exFAT cameras seems to be the ideal solution in both cases.


Quote from: g3gg0 on March 19, 2014, 12:46:15 AM
files >4GiB were never meant to be handled by the camera firmware, so there are restrictions.
what if you have exFAT support but you never handle files >4GiB as movie recording is limited to that?
do you then still have to support large files?


Obviously Canon has chosen not to support large files, even though the underlying filesystem on the storage media may support them.  This the annoying barrier that can hopefully be overcome in H.264 mode as it was in RAW mode.  Is there a limitation on the version of the .MOV container that Canon uses as you alluded to in your post?  If so, can someone write a recording mode for H.264 that puts it into an updated .MOV container, or perhaps something else like MLV or MKV?


Any further technical analysis of how these cameras record to H.264 MOV files currently that might help to overcome the 4GiB barrier to continuous recording is very welcome!
#5
Quote from: g3gg0 on March 18, 2014, 03:05:03 PM
problem A:
  Recording software-limited to 4GiB
  -> solved

...but solved only for mlv_rec (which hopefully will lead to a breakthrough for H.264 recording as well).  While it is far more urgent in RAW recording mode because of the massive size of RAW video streams, it is extremely annoying that advertised "continuous" recording to 30 minutes for bodies such as the 5D Mark III are actually not continuous because they lose frames at the 4GiB file changes.

Quote from: g3gg0 on March 18, 2014, 03:05:03 PM
problem B:
  Recording software-limited to 30 min
  -> solved (on 600D only, right?)

I have tacitly accepted that the 30 minute limit is not going to be overcome anytime soon.  Don't get me wrong, I would love to hear that it could, but I am a least very thankful for "Movie Restart" feature to deal with this.  Expecting dropped frames every 30 minutes is a lot easier than expecting dropped frames every ~16 min *AND* ~30 min, hence this feature request asking to overcome only the first barrier.


Quote from: g3gg0 on March 18, 2014, 03:05:03 PM
problem C:
  H264 format doesnt allow sizes >4GiB out of the box
  -> creates buggy H264 files that need to be fixed using a tool (EOSMovieFixer)

Although I think you know what you mean when you say this, it would be ideal to be more clear with terminology for others: the H.264 video format itself has no limitations on filesize.  It is only the recording software in the non-libre Canon firmware that has this undesirable yet unmodifiable default behaviour of finishing writing to a file at 4 GiB of size... queue question about possibility of new H.264 recording mode handled entirely by ML.  Would a third-party encoder such as x264 be required to ship with ML?  If that is possible, then would this open the door to allow recording to other formats, such as VP9 or H.265?


Quote from: g3gg0 on March 18, 2014, 03:05:03 PM
problem D:
  Files > 4GiB only possible with exFAT
  -> many models support exFAT, but not all

I don't see this as a huge issue, since ML only supports a limited number of camera bodies.  It should be obvious that the > 4 GiB file feature for both mlv_rec and H.264 mode (if intrepid developers can solve it) should be disabled on any camera that cannot support exFAT.


Quote from: g3gg0 on March 18, 2014, 03:05:03 PM
problem E:
  Filesystem driver only supports < 4GiB
  -> some few models support large file offsets

I'm less educated about how the filesystem driver differs significantly from exFAT support.  Are you implying that some cameras that DO support exFAT have a fs driver that cannot support files > 4 GiB?  That seems counter-intuitive, as that is one of the basic specifications that makes exFAT implementation more flexible and desirable.


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


If ML is able to liberate H.264 recording from the dropped frame hell the bodies currently are stuck in, it would make these bodies unbelievably useful for many people in a wide variety of use cases.  I opened a feature request to solve the glaring 4GiB limitation first, but indeed to crack the 30-min limitation (obsoleting the need for the Movie Restart hack) would be a revolutionary feature.  Full frame cameras with unlimited continuous recording in both H.264 and RAW without dropped frames is a powerful end goal.


In the meantime, tackling this highly annoying 4 GiB limitation and using Movie Restart (plus a second body to cover up the video hiccups at 30 minutes) would be of HUGE importance to event videographers!
#6
Quote from: g3gg0 on March 18, 2014, 09:01:45 AM
the only reasonable way would be using mlv_rec to record H264 frames, a case which was considered in the file format.

if someone comes up with a piece of code which allows me to capture H264/JPEG in LV mode without canon firmware being recording, then i can integrate that into mlv_rec.
this requires no change to the MLV format, as it was designed as container format and i designed it even for that use case.

If the H.264 encoder is embedded into the proprietary Canon firmware, then it obviously cannot be modified itself as you have explained.  But is it possible to call the Canon H.264 encoder from the ML Live View mode?  Or perhaps possible to ship a custom encoder like x264 or even for other formats such as VP9 or H.265 when those formats' encoders become more mature?

I am totally unfamiliar with the ML codebase to know what is possible in terms of calling external video encoders (or even calling the Canon H.264 encoder from within mlv_rec), but it seems like > 4 GiB video files is not possible (or very difficult?) unless ML itself handles the recording of the frames.

Quote from: 1% on March 17, 2014, 09:56:35 PM
It can be patched the same way as on 600D, again the meta data would probably have to be rewritten. I thought about doing it but figured why bother since its supposed to auto split the files. Its more work to re-write the videos just for 4 frames.

Can you elaborate on this please?  I hope you understand that missing 4 frames is the difference between true continuous recording for 30 minutes and the need to have a second camera setup on hand to transition to in post-production to hide unprofessional video skips.  I would love to hear your thoughts on the method of video recording on a 600D with ML might help to create > 4 GiB H.264 files up to 30 min, or at least save the missing 4 frames in a separate file so we can write scripts to merge the data in post.
#7
Quote from: a1ex on March 17, 2014, 08:19:08 PM
1)64GB cards are now working for first-time install!!!

(I only had to rename the FIR to 8.3 characters, not less)

Ah, I see why you think it has already answered.  I did notice that my exFAT cards were not working with the March 16th nightly  before the .FIR 8.3 filename fix, so I did the initial install with a 32GB SDHC card and thereafter made my SDXC cards bootable manually.  Since the 8.3 firmware fix, I can take a brand new, unmodified exFAT-formatted SDXC card and put the fixed nightly data onto it, and the .FIR-flashing process makes the card bootable and works great.  I am happy to report that I have tested it multiple times on 4 separate SDXC cards at 128GB capacity, all exFAT.  ML for 1.2.3 runs great from these cards with the new .FIR file to set it up, and it was also running when I made the cards bootable manually via make_bootable.sh/EOSCard.exe after initial installation on-camera with a 32GB SDHC card. 

My problem however is with my Komputerbay 1050x CF card.  I cannot get ML to load from this card.  If I try to make it bootable on my computer *OR* if I try to make it bootable using the .FIR update method (yes, even with the 8.3 filename fix), it creates a completely unusable card.  Whereas it starts as a perfectly mountable/usable exFAT filesystem, after trying to make it bootable via either method, the camera cannot recgonise it and asks to format it.  The laptop fails to mount the partition on the CF card when in the card reader, complaining:

ERROR: invalid VBR checksum 0x258ed093 (expected 0x30c2a1c9).

Furthermore, fdisk reports the filesystem to be type "W95 FAT32 (LBA)" after making it bootable.  Before, fdisk correctly read the partition table and detected the filesystem as type exFAT.  I have more SDXC cards around that I can always boot ML from, so I do not strictly need ML bootability from the CF card, but I want to know if there is something wrong with my CF card itself or the ML install process.



Quote from: a1ex on March 17, 2014, 08:19:08 PM
2) I bet you have enabled FPS override.

Oops!  You are correct.  Sorry about that.  I changed to 1080p/24fps in the Canon firmware menu and disabled FPS override.  Obviously audio recording is working again!  I just wanted to do my RAW testing with 25 fps + audio, but I guess that's not possible atm...  ???
#8
Quote from: gary2013 on March 17, 2014, 05:08:48 PM
I was wondering if it was possible for ML to keep recording those four frames or so into a memory buffer and then it records those four frames into another split file when the next clip starts recording.

Are you suggesting a temporary split file for the 4 frames to be saved to a separate file when the camera creates the 2nd file?  So we would end up with 3 files in the end for a 30 minute clip, or would you suggest having ML join the files at the end of the record session to get one continuous 29:59 file at the end of it?


I am not sure the best way to implement this, but finding out how the > 4 GiB file setting for mlv_rec works may be helpful in determining how easy this is to implement for H.264.
#9
Quote from: a1ex on March 17, 2014, 01:15:46 PM
1) already answered

I had already lookd through this entire thread as well as did an forum search for the entire site to find the technical reason why CF cards are non-functional for booting ML even if made bootable via normal methods (EOSCard.exe or make_bootable.sh).  Seeing as you know that it was discussed elsewhere, it may be best not to spend more time in the 1.2.3-specific thread asking about it, but it is still not clear why booting from CF is not technically possible at this time.  Any links to further information would be appreciated.

Quote from: a1ex on March 17, 2014, 01:15:46 PM
2) H.264 sound works out of the box, no modules needed

The audio was broken on the March 16th nightly, but I just upgraded to the March 17th and now I can record audio on H.264 mode as well.  The March 16th build was flashing a "Audio recording disabled" error as soon as I hit the Start/Stop button to begin recording and it never showed the audio recording meters in Canon firmware mode or in the ML live view with the larger ML audio bars up top.  Only the menus in the Canon firmware would show mic activity unless mlv_rec and mlv_snd were enabled.

March 17th build fixes the audio recording being disabled for me.
#10
We do quite a lot of event videography, and one of the nicer features of the 5D Mark III is that the default camera firmware allows for recording 30 minutes straight without needing ML's movie restart feature.  We are very greatful to the movie restart feature for 30+ minute recordings, but this feature request is not about that.

As you know, the 5D Mark III still splits up video files into 4 GiB chunks, even when the CF or SD card uses the exFAT filesystem.  This limitation is the basis for the mlv_rec RAW video recording option labeled

QuoteFiles > 4 GiB (exFAT)


This request is for the same feature in H.264 mode.  During post-production, we have noticed that even with the camera supposedly being able to "continuously" record to 29:59, there are dropped frames during the 4GiB file changes!  The first 4GiB file generally records for 16-17 minutes, after which the next file is created: in between, there are about 4 video frames that are lost in this transition, meaning the 5D Mark III is not able to truely record to 29:59 straight through!

It is frustrating because when using exFAT, there is no need to make 4GiB splits.  Ideal ML behaviour would be to auto-detect the filesystem on the recording media and simply keep recording until the 29:59 limit is reached in a single file if exFAT is used.  If this ideal behaviour is too difficult, then simply giving a menu option as with the mlv_rec's "Files > 4 GiB" option would be a huge improvement for all of us who need continuous recording with no dropped frames.


It would, at least in our case, be a killer feature of the ML software!  We are otherwise being unfairly held back by FAT-era limitations in a post-FAT world.
#11
Installed and working!  Another success!!  :-)  Amazing...

Nightly.2014Mar16.5D3123 here.


I have 2 questions however...

๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏

1)  Installing and booting from a CF card is still not possible?  I am currently running ML off an 128GB SDXC card which works great (had to make it bootable with make_bootable.sh for it to work however).  I tried to also set up my 128GB 1050x Komputerbay CF in the same way, but with just the CF card in, the camera is completely unbootable and non-responsive until I take out the battery.  If both the SDXC and CF cards are inserted, I get the message:

QuoteML on both cards, loading from SD.

and ML loads fine and I can shoot/record to either card.  Also, if just the SDXC card is inserted, I can also use ML with no issues.  According the the message, ML detects a copy of itself on the CF card, but cannot load from there (and makes the camera seem broken until I pop out the batteries!).  Is this normal behaviour?  Is it impossible to boot from CF cards?

๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏

2) Audio recording in movie mode, even with H.264, seems to not work at all unless mlv_snd is loaded and enabled.  Furthermore, mlv_snd causes many errors unless mlv_rec is also enabled.  Therefore, there a couple of things that need fixing.  Most important is being able to record normal H.264 video and audio without having mlv_snd enabled.  As far as I understand it, the mlv_snd module is for adding audio recording to MLV-style RAW video recording.  This is also what the module description indicates:

QuoteWith this module, mlv_rec is extended by sound recording support.

So why do we need this module enabled to record normal H.264 videos with audio when using ML?  In the Canon firmware menus, the audio recording is enabled and volume up, but when starting to record H.264 videos without mlv_snd disabled (default configuration!), it gives an error that audio recording is disabled.  This is confusing and seems to be broken.  Does normal H.264-mode audio recording really depend on mlv_snd, or is this a bug?

On a related note, enabling mlv_snd but leaving mlv_rec disabled causes many ML errors to the point of disfunctionality.  This is a user interface issue, really.  If one module depends on another to function, then the UI must load not only the user-selected modules, but its dependencies as well.

As it stands, after a default install, audio recording seems broken unless *BOTH* mlv_snd and mlv_rec are enabled.  So if you want normal H.264 video recording with audio, you need to enable RAW video recording modules but disable the RAW video recording.  This is not only counterintuitive, but it gives users a default broken configuration for normal (non-RAW) video recording.

๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏๏

If anyone else can confirm any of the issues I have outlined, it would be appreciated!
#12
5D MIII owner here, and *extremely* happy to hear that some others are getting clean HDMI output whilst recording RAW onto CF cards... the prospect of merging the official features of Canon's 1.2.3 with improved integration and stability of ML is incredibly exciting for me!

I'd love to ask those of you who have acheived simultaneous clean HDMI & RAW recording about the details of the video stream being recorded via HDMI.  What device (if any) are you using to record the clean HDMI signal, and is the uncompressed data being encoded to a different format, such as ProRes, prior to saving?

I understand that the HDMI output, whilst "clean" and without any menus overlayed, is only YCbCr 4:2:2 with an 8-bit colour palette, which should make it inferior to the RAW video that ML enables with a full 14-bit colour palette.  If the HDMI output at 4:2:2 is saved uncompressed (as it is sent via HDMI with Canon 1.2.3), then it should make for some very interesting comparisons if we can make the same shots with both 4:2:2 8-bit & RAW 14-bit uncompressed files saved concurrently!


Thank you all for your creatively intelligent hacking to port all existing functionality to 1.2.3, and even fixing previously broken features along the way!!  Three thumbs up!  ;D
#13
General Help Q&A / Re: Latest 5D Mark III version??
October 26, 2013, 12:17:31 AM
I finally just upgraded to a 5D Mark III, but it's very unfortunate to find out that I must downgrade the base Canon firmware version in order to use ML!  My newly purchased 5DMIII shipped with version 1.2.1, so an out-of-the-box firmware downgrade in order to try out even ML nightlies is somewhat disheartening!

Even so, the benefits of ML will surely be larger than what official changes Canon provided in the 1.2.1 update.  I lack the reverse engineering abilities myself in order to get ML working with 1.2.1, so I am not complaining, but just letting it be known here that it's not an ideal situation to have to downgrade firmware of a new body.


I'm going to try to downgrade and use a nightly ML build from 128GB SDXC cards, for lack of a better option.... here's hoping for 1.2.1 support being worked on!
#14
Quote from: a1ex on January 16, 2013, 10:06:58 AMI don't understand what changes you did to make_bootable.sh, other than removing the autodetect feature (which works for me).

I don't understand why myself, but as I said the files linked on the Google Groups page had CRLF for their line breaks when I downloaded them, when they should be just a LF character.  I didn't otherwise remove any features, so I'm not sure what you are referring to.


Quote from: scrax on January 16, 2013, 10:24:17 PM
Great, so you used the fixed exfat_sum.c on a card bigger than 16GB without issue?
Cause so far I have reports of only 16GB Exfat card confirmed working.

Yes, it's working with no issues on my 128GB SDXC Lexar card with exFAT!
#15
I've solved the obstacles to getting exFAT bootable under GNU+Linux!  I never tried running EOSCard with Wine, as I wanted a native solution to get this working.  I will explain what I found out for those who are curious, and provide attachments to the fixed files for those who want something quickly working.

Problem #1: make_bootable scripts have incorrect line breaks

The files on the Google Groups page are problematic and ideally should not be linked to in the official ML instruction PDF and wiki as they are.  The make_bootable scripts are not executable on my systems (I tried on multiple distributions of GNU+Linux) because the endings of the lines are CRLF which stands for a Carriage Return character followed by a Line Feed character.  This is the default newline notification for text files under DOS/Windows, but in Unix-based OSes (including GNU+Linux and Mac OS X), newlines are notified by only a Line Feed (LF) character, with no CR in sight.  Bash was unable to execute the files as they exist on the Google Groups page because it needs all CRLF to be converted to LF in order to sanely determine where the line breaks are.  It's tricky because in a normal text editor, you do not see the difference, but if you use for example the 'file' command on the bash scripts, you will see output similar to this:

make_bootable.sh: Bourne-Again shell script, ASCII text executable, with CRLF line terminators

This last part of the output of 'file' tells us why this script is not executable by default.  To convert CRLF line breaks to LF ones, there is a utility called dos2unix (most likely available as a package of the same name in your distribution of choice) which does the job quickly and easily.  After running this program on the make_bootable script, file now reports it as such:

make_bootable.sh: Bourne-Again shell script, ASCII text executable

With no mention of CRLF in sight.  After doing this change, the script posted by 'arm.indy' on the Google Groups page became executable on my systems.  Unfortunately, while the scripts seem to execute and exit with success, they caused my SDXC cards to not only be unbootable, but completely unmountable and unreadable on both my camera and PC.  Debugging the mounting errors on my PC, I found the fault to lie with the exfat_sum program, which brings me to...


Problem #2: exfat_sum.c was written by a 32-bit OS user, for 32-bit OS users (won't work on 64-bit!)

After getting the make_bootable script working, I found out that the exfat_sum step was calculating the wrong checksum on my systems, which are all running 64-bit operating systems.  The problem lies in the exfat_sum.c source code file, where there are 4 instances of "unsigned long".  long numbers are 32 bits on a 32-bit OS, but they are 64 bits on a 64-bit OS, meaning for those of us on 64-bit OSes will have different (incorrect!) values calculated by the compiled exfat_sum program.  To fix this, I changed every instance of "unsigned long" to "unsigned int" in the source, because int values are always 32 bits no matter what platform you are running.  Changing these lines, recompiling, and running my fixed make_bootable.sh script finally produced a mountable, bootable exFAT filesystem. 

I copied the ML directory and autoexec.bin file to the exFAT filesystem, and Magic Lantern was instantly working on my 128GB cards!  Note that I had previously initialised my camera with a 32GB SDHC card with FAT32.  I do not believe you can skip this step and still have a working SDXC/exFAT card with ML on it, even if you successfully make the exFAT filesystem bootable using the above methods (or other known working methods such as EOSCard utility on Windows).


If you are uninterested in duplicating the steps I gave, I have attached the fixed versions of the 2nd make_bootable.sh script and exfat_sum.c files in this post.  I hope this information or files are useful to other *nix users down the line!
#16
Quote from: nanomad on January 12, 2013, 03:44:15 PM
give EOSCard a try and see if it works for you

I don't have a Windoze box around, and it would be a damn shame if I can't set this to be bootable without using a Windoze-only GUI app.  Almost any other filesystem I can make bootable from the CLI easily, but Microsoft just HAD to use their power and money to get the SD Card Association to "standardise" on yet another one of their heavily patented, proprietary filesystems for SDXC as a trojan horse....   /end mini-rant


I guess I will try to run it with Wine.  Or perhaps I should try the firmware update method with the SDXC card?  Is there a reason why the instructions only mention the bootable method for SDXC cards?  Does the firmware update method not work for SDXC cards?
#17
General Help Q&A / Re: Bit Rates
January 12, 2013, 06:53:33 PM
Quote from: Lumiére on January 12, 2013, 06:22:16 PM
                     720x1080   50fr     color space 4.2.0      overall bitrate  47.3 Mbps
Nikon 5100   720x1080   50fr     color space  4.2.0     overall bitrate   12 Mbps ( poor Nik)

720x1080?  What kind of aspect ratio is that... taller video than it is wide by 360 pixels?

Maybe you mean 1280x720?
#18
Hallo, first time ML user here, and I am very excited to get going!

I got ML 2.3 working great on a 32GB SDHC card using the firmware update method on my 60D, so I have a small taste of how things should be running!  I normally only use 128GB Lexar SDXC cards in this body, however.  I want to set up every SDXC card I have to make ML run as soon as I load the card, but I can't get my first one working!


I am running Debian GNU/Linux at the moment, and I am trying to use the make_bootable.sh script found in the Google Groups page linked in the instructions PDF.  It is annoying that something seemingly as simple as turning on a bootable flag in the SDXC card's exFAT filesystem is tripping me up, but here is the output I have from.  I had to use the third version, make_bootable3.sh as the other two were not working. Also, I compiled 'exfat_sum.c' and set both the resulting exfat_sum executable and make_bootable3.sh script file +x permissions.


# ./make_bootable3.sh /dev/mmcblk0p1

MagicLantern card pacher Only

Applying EXFAT parameters on /dev/mmcblk0p1 device:
writing EOS_DEVELOP at offset 130
writing BOOTDISK at offset 122
recompute checksum. old=0x0, new=0xf801fffff801ffff


After this finishes, I can no longer mount the card on the computer, nor the camera, and I must reformat again in the camera for it to be functional again.   :'(


Getting ML working on my 128GB cards is a must, so anyone with some tips on how to properly set exFAT filesystems to be bootable under GNU+Linux (or even Mac OS X CLI), please impart your wisdom!


Thanks!