Defragmenting ExFat ?

[Kharak]

I recently upgraded from 5D MK II to MK III.

So now I'm in Exfat world.. On MK II where I was running my cards with Fat32 I regularly defragmented my cards because after filling a card with RAW footage and exporting to computer the fragmentation would be around 50% sometimes higher, sometimes lower.

But as I understand by googling a little, Exfat can not be defragmented. Is this because Exfat does not get fragmented ? or its simply not possible?

I find it hard to believe that it does not get fragmented when it is so severe in FAT32. So I am worried that I am running around with slower than need to be cards because of fragmentation.

Anyone know something more about this and how to defragment Exfat? (if need be)

Thank you.


(Maybe this is in the wrong section, but it's to do with RAW recording)

[Walter Schulz]

Is there any data what fragmentation on a flash drive actually does to performance?
On a spinning drive where the read/write sensor's arm has to be moved to the sectors's location, waiting for the spinning sector and so on ... yes, fragmentation exists. But on flash?
Hard data, please!

EDIT: In the early flash aera I was trained just to format the card which will result in overwriting FAT only (FAT=file allocation table) and therefore reduced write stress to the cells. And fragmentation is gone, too.

[Frank7D]

Do you really need to defragment an empty drive? I would think after you format your card (or delete all files except ml) the space freed is contiguous and no longer fragmented.

[chris_overseas]

Flash memory is composed of pages and blocks. A page is the smallest unit that can be read or written, and a block (made up of multiple pages) is the smallest unit that can be erased. To overwrite an existing page the entire block has to be erased first. It's this last characteristic that has some very important implications for flash memory performance.

There's a good example given here that shows how performance can suffer when writing to flash memory: http://www.anandtech.com/show/2738/8  The entire article's worth reading if you have time (it's an old article but still relevant today).

From the above you can see that defragmenting isn't going to give you a speedup for the same reason it does with a HDD. What you really want to do is erase any pages that are no longer used (eg that contain deleted photos). If the defrag process does that then it can be a win. More recent CF cards have Trim support however which does a lot to negate this problem. Overall, I doubt that defrag is beneficial and if it doesn't erase blocks that are no longer used after the defrag it might even make things worse. It also uses up write cycles, though the chances of that ever being a problem are almost none.

[edit: worth a read: http://www.pcworld.com/article/2047513/fragging-wonderful-the-truth-about-defragging-your-ssd.html]

[Kharak]

Thank you for all your replies, this has given me a much greater understanding of Flash.

But this TRIM command, is this also functioning within a DSLR?

And do you think the severe fragmentation I saw when running the cards with FAT32 are the blocks/pages that are "empty" or ready for rewriting ? Never seen a HDD with 40-60% fragmentation.

And just to clarify. The best way to ensure a peak performance is to run a Quick Format, correct?

[Audionut]

A smart defragger doesn't defrag an SSD, it send the trim command to erase the pages.  I assume this holds true with all flash memory.  I'll see if I can find some relevant links.

[a1ex]

More recent CF cards have Trim support however which does a lot to negate this problem.

I managed to run a small part of hdparm (the "identify" command) on 5D3, using g3gg0's direct CF access routines from the cf_acc modules, and here's the log for Komputerbay 32GB:

Code Select Expand


Load modules...
  [i] load: CF_ACC.MO

[...]

hdparm identify...

CompactFlash ATA device
Model Number:       SILICONMOTION SM2236AC                 
Serial Number:      AA00000000000103384
Firmware Revision:  20121203
Standards:
Likely used: 6
Configuration:
Logical max current
cylinders 62041 62041
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors:   62537328
LBA    user addressable sectors:   62537328
LBA48  user addressable sectors:          0
Logical/Physical Sector size:           512 bytes
device size with M = 1024*1024:          80 MBytes         ; <-- probably printf bug regarding int64 values
device size with M = 1000*1000:           0 MBytes (0 GB)
cache/buffer size  = 1 KBytes (type=DualPort)
Capabilities:
LBA, IORDY(may be)(cannot be disabled)
bytes avail on r/w long: 4
Standby timer values: spec'd by Vendor
R/W multiple sector transfer: Max = 1 Current = 0
Advanced power management level: disabled
DMA: not supported
PIO: pio0 pio1 pio2 pio3 pio4
     Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
Enabled Supported:
    Power Management feature set
   * Write cache
    WRITE_BUFFER command
    READ_BUFFER command
    NOP cmd
    CFA feature set
    Advanced Power Management feature set
   * 48-bit Address feature set
   * Mandatory FLUSH_CACHE
   * FLUSH_CACHE_EXT
CFA max advanced io_udma cycle time: 80ns
CFA max advanced mem_udma cycle time: 80ns
   * CFA Power Level 1  (max 500mA)
Integrity word not set (found 0x0000, expected 0xaaa5)
done!
Save configs...


So... it doesn't seem to support TRIM, nor secure erase

Note: the hdparm -i command refuses to run on the PC if I use a USB card reader.

[Walter Schulz]

Note: the hdparm -i command refuses to run on the PC if I use a USB card reader.

AFAIK there is no USB cardreader able to do that. Same for other programs trying to get into the controller (F3, h2testw, ...)
I tried to contact KB support via ML forum PN (but no avail) about PATA-adapters. Those I found are UDMA-6 at best and KB is telling about damaging cards in non-UDMA-7 devices.

If you know about an UDMA-7 device (other than ML cams, of course) allowing access to controller ...