macOS Made a File That Reports Zero Bytes and Occupies Sixty Kilobytes
I was writing a storage analyser, which means walking every file on the disk and adding up what it costs. Somewhere around the 140,000th file the progress line printed a path that does not exist: /Users/me/Applications/Chrome Apps.localized/localized There is no localized in that folder. There is a .localized, and there is one other thing, and that other thing turned out to be two separate problems wearing one filename. The file $ ls -b ~/Applications/Chrome\ Apps.localized/ Icon\r Not Icon followed by something. The name is the four characters I, c, o, n, and then byte 0x0D — a carriage return. Python is blunt about it: >>> os.listdir(d) ['Icon\r', '.localized'] macOS wrote that. Every folder with a custom icon gets one: Finder → Get Info → paste an image, and this appears. The trailing CR is a fossil from classic Mac OS, where it was the line terminator, and it is there to keep the file out of the way of anything that lists a directory naively. It has outlived the reason. Problem one: it eats your log line A filename on a POSIX system may contain any byte except / and NUL. That includes carriage return, newline, escape, and backspace. Almost nothing you write treats a path as anything other than text you can print, and this is where that assumption gets paid. Here is what a progress line actually emits: $ find box -type f -print0 | while IFS= read -r -d '' f; do printf 'scanning %s 42,318 files\n' "$f" done | hexdump -C 00000000 73 63 61 6e 6e 69 6e 67 20 62 6f 78 2f 49 63 6f |scanning box/Ico| 00000010 6e 0d 20 20 34 32 2c 33 31 38 20 66 69 6c 65 73 |n. 42,318 files| Look at offset 0x11: 6e 0d — the n of Icon, then 0d, the carriage return. The terminal receives it and does what a terminal has always done with a CR: it moves the cursor back to column zero. The rest of the line then overwrites what you just printed. You do not see a corrupted path. You see a line that is missing its front half, spliced onto whatever came next — which is precisely as confusing as it sounds when the count in front of it is also changing. The same byte does the same damage to a single-line label in a GUI, a CI log, and anything that later parses its own output. And the CR is the polite one: a filename containing \n will break find | while read line outright, because that loop's entire contract is that filenames do not contain newlines. Hence -print0 and read -d '', which exist for this reason and not for style. If you render paths anywhere a human will look, strip the control range first: final safe = path.replaceAll(RegExp(r'[\x00-\x1f\x7f]'), '?'); Problem two: it is zero bytes, and it is sixty kilobytes This is the one that actually mattered for a disk tool. stat -f %z (st_size) 0 bytes ls -l 0 os.path.getsize() 0 ------------------------------------------------ st_blocks × 512 61,440 bytes du -k 60 KB The data fork is genuinely empty. The icon — 61,010 bytes of it — lives in an extended attribute: $ xattr ~/Applications/Chrome\ Apps.localized/Icon^M com.apple.FinderInfo com.apple.ResourceFork com.apple.provenance com.apple.quarantine com.apple.ResourceFork is the resource fork, the other half of a classic Mac file, kept alive on APFS as an xattr and reachable through a magic path: $ stat -f %z ~/Applications/.../Icon^M/..namedfork/rsrc 61010 st_size describes the data fork. It always has. So a scanner that sums st_size — which is the obvious thing to sum, and what os.path.getsize hands you — reports this file as free. Sixty kilobytes, invisible, once per folder with a custom icon. Sixty kilobytes is nothing. The habit is not nothing. st_size is a statement about content; st_blocks is a statement about the disk, and a disk usage tool is answering the second question. They come apart in three directions and resource forks are only one of them: Sparse files report a large st_size and occupy almost no blocks. Compressed files — macOS transparently compresses much of /System and many app bundles — report the uncompressed size and occupy less. Resource forks and xattrs report nothing and occupy real space. Sum st_size across a volume and your total will not match About This Mac, in either direction, and the user will believe Apple. Correctly. du has always used st_blocks; that is the whole reason du and ls -l disagree, and I had read that fact several times without it meaning anything until a zero-byte file turned up holding an icon. The catch, if you write Dart Knowing to use st_blocks and being able to are different problems. Dart's FileStat exposes size, mode, type and three timestamps. There is no blocks. File.length() is st_size as well. The number you want is in the stat struct the VM already called and did not surface, so reaching it means going around the standard library: // dart:ffi against stat(2), or a platform channel to // URLResourceKey.totalFileAllocatedSizeKey on the Swift side. Which is the actual reason so many size-summing tools get this wrong. It is not that st_size is chosen over st_blocks; it is that in most high-level languages st_size is the only one handed to you, and the difference between "how big is this file" and "what does this file cost me" never comes up until something reports zero and takes sixty kilobytes. Two rules I would now write down before starting a tool like this: Sum st_blocks, not st_size, and if the language will not give you st_blocks, that is a dependency you take on day one rather than a refactor you do after the totals disagree with About This Mac. Strip [\x00-\x1f] from any path before rendering it. Paths are bytes, not strings, and one of them on your Mac right now ends in a carriage return. The file is still there. It is on your Mac too — find ~ -name 'Icon?' -type f will tell you, and the ? is the only way to match it that does not require you to type a carriage return into your shell. Originally published at devshakib.jumyn.com. I write about Flutter, Dart and the parts of shipping that are genuinely awkward — and publish the packages that came out of them at pub.dev/publishers/jumyn.com.
This is a summary aggregated from Dev.to. Read the complete article on the original site:
Read full article at Dev.to