Oof. Been a minute, hasn’t it? Still, what’s eighteen months between blog posts, eh? I mean, yeah, it’s eighteen months, but then again I have friends and family that I like to put on a two-to-fifteen-year face-to-face rotational, so the fact that I’ve let this thing slide for the better end of a year and half is both probably on me and also not bad by my personal, very lax standards.
Still, I only seem to have time to work on things like this when I have gaps in Actual Client Work™ that I am actually doing for Actual Clients™, and there’s been a lot of that of late which – let’s face it – for the freewheeling independent IT consultant can only be a good thing. Once in a while, though, I come up for air, look at this thing, and try and think of something useful or interesting to jot down, and this is one of those times.
Today, Gentles all, I am going to write about how to spend a lot of time in order to save a little time and not a lot of space. I am going to talk about archiving.
Businesses bloom, and business expand, and then now and again, businesses wither away. Often by design; after, say, designing reticulating sprockets for forty years, a reticulating sprocket-designer has had enough and wishes to jack it all in and go and sit on a beach and never think about flanges or fittings again, and this is a fine and noble state of being. Still, the market for things that attach to other things and also turn around continues apace, so businesses or clients are sold as assets, taking with them all the assorted data and debris that those clients have accumulated over the years, which takes me to my current discussion of What To Do With All That Data.
Some data is relatively small. If you added up forty years of Mac text files – going all the way from the initial version of WordPerfect back in the 80’s to land somewhere in the recent Office365 incarnation of putting-words-on-a-screen, you might reasonably expect to have a healthy handful of gigabytes – but probably not more. Text files are, by-and-large, small and uncomplicated things. Large CAD or design files, however, can cheerfully run into the healthy-handful-of-gigabytes range, and forty years of that kind of stuff gets… pretty huge. If you’re talking about a couple of dozen client directories with that kind of amount of data then you’re talking about some serious numbers in the multiple-terabyte range.
“So?” I hear you scoff. “Modern hard drives can handily accommodate that scope of data.” You are, of course, right about that, but the waters get muddier when you start to consider that due to the sensitive (cough, secret, cough) nature of some of these client files, there are some requirements for compressing and encrypting that data before storing it.
Enter my favorite place on the Mac: the relatively obscure toybox of /System/Library/CoreServices, with a special shout-out to its attendant subdirectory – Applications. Let’s take a look at that, shall we?
Truly a treasure trove of delights. This folder is a place where the OS dumps applications that it needs to break out now and again, but that don’t necessarily warrant a spot inside the regular Applications or Utilities folder. Some of them used to live in the regular Applications or Utilities folders, but have recently been shunted to this sort of dusty shelf full of lost toys (I’m looking at you, Keychain Access. I know, I know, you didn’t deserve this). Some are just there because they’re so rarely used that putting them under people’s noses would just be annoying. After all, how often do you want to fire up the Expansion Slot Utility if you don’t have a computer with Expansion Slots? And while the Ticket Viewer is handy for working with Kerberos, the vast majority of Mac users have never heard of a Ticket Granting Ticket (or Kerberos) and are – quite frankly – all the better off for that.
Still, a couple of those things are absolutely worth knowing about, and while I can wax on for another handful of paragraphs about the splendor that is Directory Utility, let’s stick to the task in hand, which is archiving, and thus handily addressed with the appropriately-named Archive Utility. Here, drink the icon in, in all it’s glory:
The Archive Utility is separate and distinct from the Finder when it comes to making archives. Oh, you can just right-click on a folder or file in the OS and choose “Compress”, and that’ll make you a nice, neat .zip file after a little while, which is all well and good – but it won’t do much more. And it’s sloooow.
Here’s an example – a relatively small 22GB folder containing a mixture of text documents and executables, compressed using the Finder as a regular .zip file. The CPU Usage chart is shown to demonstrate that the CPU itself is barely ticking over – the six bars on the left represent the six Efficiency cores in my M3 Pro MacBook Pro (the dual “Pro” name is ridiculous, I know), and the six bars on the right show the Performance cores.
As the Finder’s approach to running this compression is to use what appears to be a single-core process, it doesn’t leverage the full potential of the CPU, and this takes a… long time. Five minutes and forty-four seconds, to be precise. It’s also not encrypted, which isn’t ideal. Happily, Archive Utility has some ideas about that.
When fired up, Archive Utility doesn’t give you much to look at, but if you choose “Settings” from the “Archive Utility” menu, you get this tasty little snippet:
I will draw your attention to “Apple Encrypted Archive”, because that’s what we’re going to play with. Select that, then hit Command-K or choose “Create Archive…” from the File menu, select your directory, and it’ll pop up a window with a pre-filled Password key and an option to store that password in the Keychain. I advise you to this – or at least note that password – as it doesn’t appear to be editable. Apple’s attitude in this department seems to be that they’re doing all the work here, so if you don’t like their passwords then you’re perfectly welcome to not use them, so… there. Anyway:
Hit “Encrypt” and then – if you’re me writing this – frantically move things around to take a screenshot because the Archive Utility isn’t restricted by the ridiculous concept that you might want your computer to use one (1) of it’s cores to run this task, when you can use all twelve (12), and you have a limited amount of time to get said screenshot because this entire operation takes nine seconds.
Nine seconds. Nine. As opposed to Three hundred and forty-four. That’s, what, thirty-eight times faster? Give or take?
“Now,” I’m sure you’re saying to yourself, “that’s great, but does that really warrant a whole blog piece?” I’m going to argue yes – because this test folder was twenty-two gigabytes. The Actual Client’s™ file folders? There are upwards of two hundred of them, and the smallest one is a hair over six hundred gigabytes – which Archive Utility was able to handle in four minutes and fifty-two seconds. Compressing this using the Finder? Two hours, eighteen minutes and eleven seconds.
Thankfully, I am not required to be the person sitting at the desk, packaging up and compressing and encrypting those files. Well, okay, sure, that wouldn’t be the worst thing in the world because for this kind of thing I’d charge by the hour and that new Rivian looks cool, but you get my point. With the volume of data we’re talking about, using Archive Utility allowed the client to have someone sat at a desk with a Mac Studio and get the entire job done in two full work days. Extrapolated out, using the regular Finder utility would have eaten up somewhere around fifty-eight work days. Fifty-eight.
I don’t know about you, but that seems like a tool that should be taken down from the shelf, dusted off, and put back into your regular rotation…
(PS: See you all in another eighteen months. Maybe.)