Month: December 2020

This is, believe it or not, the busiest part of the year if you’re an IT consultant. There are excellent reasons for this; businesses are generally keen to close out budgets, or make purchases and roll out products prior to the end of the tax year, or even just feel the universal urge to do whatever it takes to tie a neat knot around the year and go into the next one loaded for bear. The practical upshot of that is that I haven’t written anything for this thing for a while because I’ve been far too busy running around and actually working (which is exactly the kind of problem you want to have if you’re me).

Still, I have a running list of things I wanted to touch on because I think they’re interesting and because this blog is as much as anything else a resource that I can go back and look at when I need to remember some piece of syntax that gets shoved out of my aging cranium to make space for something more critical. One of those things concerns my first love (at least in a professional sense), which is homebrew.

(Okay, honorary shout-out to Synology as my other first love. It’s the weirdest, nerdiest form of polygamy.)

Homebrew is fabulous. I like to support it financially when I can because it’s a simple and effective way of putting together tools that enable me to do my job. I mean, sure, there are other ways of building programs and tools that are perfectly functional, but if IT consultants are plumbers then homebrew is an organization that just hands out wrenches for free. You can’t beat that value, and you’d be a fool to try. Still, now and again open-source tools run afoul of the rest of the world, and it can be jarring to reach for a wrench only to find that – against all expectation – it’s not where you left it.

I’m referring in this case to YouTube-dl, and the recent debacle over it’s equally recent removal and reinstatement on GitHub. You can read that link, or I can outline the rough lines of the story, which is pretty simple. YouTube-dl is a tool that you can feed a streaming video URL to, and which will then process that URL to extract the video and audio feeds and download them to your computer. Despite the name it’s a tool that works on, well, basically every service that streams non-DRM encoded video, and while it’s (fairly predictably) used to scrape video content from the internet that providers may not want scraped, it also has a raft of legitimate and fair use applications. I work with educators who’ll use it to pull academically-licensed video down to machines for presentations and research purposes, for example.

Still, the problematic use cases of the tool (notably the bit about being capable of illegally downloading and saving copyrighted music and video) ran afoul of the RIAA, who complained to GitHub that they were hosting a tool that was in contravention of copyright law, and in turn GitHub pulled the tool and left a lot of proverbial plumbers without proverbial wrenches.

Fortunately, all parties saw sense and restored Youtube-dl within days, but during the outage I had to do some very specific poking around about how to find and build the tool from non-GitHub resources, and in turn how to use it to manually select video and audio formats, which turned out to be rather interesting.

As anyone who’s uploaded video to YouTube will tell you, it’s not simply a process of lobbing it a file and then going and having a cup of tea while it puts all the ones and zeroes onto a webpage for your delectation and delight. That’d be convenient, but there’s more to it; YouTube is accessed by all kinds of client computers and devices over all kinds of connections, and as such it likes to have a lot of different versions of those video and audio files to serve out to those computers and devices. After all, if I’m watching a video on my iPhone on an LTE connection and the only video they can send my iPhone is the same full-resolution 4k file they’d send to my desktop wired to fiber then I’d stand in the rain watching the thing spool for a very long time while I asked myself whether I really needed to spend thirty minutes waiting to watch a cat video, and whether I should have considered my life choices with greater attention to time-management and the ownership and use of an umbrella.

Fortunately, YouTube-dl makes it easy to look at all the different versions of audio and video streams associated with a YouTube video, and then allows you to pick and choose which versions you’d like to download. Let’s start with a classic educational staple – Dr. Richard P. Astley’s rendition of the immortal classic “I Shall Never Give You Up.”

The YouTube URL for this is:

If you copy that URL and paste into YouTube-dl while invoking the -F option then you’ll get this impressive looking list:

From looking at that list, we can see that there are four audio-only formats, seventeen video-only formats, and one combo option right at the very end. YouTube takes this menu, looks at your connection and the device you’re using, and then selects the best options from the list, but we can use YouTube-dl to make our own choices by invoking the -f option. Let’s say we want to download the highest-possible quality video file (the 1080p mp4 option – number 137 on the list ) and the worst quality audio file (the 49k webm option – number 249 on the list). To download that using YouTube-dl you’d use the command:

youtube-dl -f 137+249 https://youtu.be/dQw4w9WgXcQ

The output will look something like this:

…et voila – you now have a copy of the file in the home folder on your computer.

All talk of plumbers and nonsense aside, I can kind of see the concern about the use of tools like Youtube-dl. It is, after all, a tool that can be used for both legitimate and illegitimate use alike, and how it’s wielded is largely a matter of personal judgment and policy. On a practical level, I suspect that the prospect of it being used wholesale as a mainstream piracy tool is limited by the fact that unless you want to go spelunking into video and audio formats a default invocation of the command will feed you a pretty good copy of a video that’s really no better than just viewing the content for free on the internet. Further, if you’re of a mind to go digging around and trying to pull down higher-quality files then you’ll still usually end up with a sub-perfect product quality-wise, as well as something that will eat up probably a lot of storage space – in short, there are easier and better ways of getting to content than adopting this as a default part of your arsenal…

I spent most of last weekend in a warehouse in Carpinteria, spouting an ever more specific series of salty oaths and curses.

This isn’t – just so we’re on the same page here – the way that I normally like to spend my weekends. It’s terribly important to maintain a healthy work/life balance (particularly in These Trying Times), so keeping work and personal matters separate is important and a flagpole of mental health, and it’s vital to stay grounded and in touch with the people who are most important to you.

This is by way of saying that when I’m issuing salty oaths and curses on most weekends they are chiefly directed at my family, who are quick and open about returning them in kind.

Still, now and again the nature of honest toil involves going and working on a weekend, which is fine. A lot of substantive IT work gets done at hours when it’s less likely to cause massive disruption. Like most IT consultants, I’m no stranger to walking into a client office at 5pm and walking out at 8am the next morning. Or decamping to an onsite location for a weekend, for that matter. This is the nature of the gig; you can’t make fundamental changes to infrastructure while said infrastructure is being actively… infrastructed. It’s like repairing a car engine while the thing is hauling down the freeway. It can be done, but it’s not going to end well, there are going to be enormously destructive crashes that cost everyone a lot of money and time, and someone’s probably going to end up in the hospital.

So, last weekend should have been pretty straightforward. The migration from the client’s ancient and ailing Mac mini server to a nice, shiny new Synology NAS had been completed without incident – chiefly because Synology makes a solid, well-designed product – and all that was left to do was to install a remote access application on each Mac desktop so that the client could use their cloud-based accounting package. It was a simple matter of installing some applications, doing a little light configuration, then being home in time to sink a couple of cocktails replete in the general glow of a Job Well Done.

Except that, no, it wasn’t a simple matter. The remote access application flatly refused to launch on about half the Desktops for no discernible reason whatsoever. Same hardware, same exact operating system and patches, but while about half of them worked perfectly, the other half not only refused to launch but refused to even bounce in the dock.

This is unusual. Well-written applications either run just fine or give you some kind of polite-if-terse indication why they fail to do so. They don’t as a matter of course just sit there, unresponsive, glowering at you from the Dock while you wrack your brain and try and work out what’s wrong. A peruse of the Console.app showed an error message thus:

Termination Reason: Namespace CODESIGNING, Code 0x1

…which is the kind of thing that makes your blood run cold once you figure out what it means. Essentially, the program won’t run because the OS has decided that it either isn’t signed (see last week’s article on Gatekeeper) or because its signature is invalid. Downloading a fresh copy of the app from the Mac App Store made no difference, which pointed me in the direction of the OS thinking that the signature was invalid because anything you download from the App Store is, by the nature of the transaction, signed.

So, how to fix?

My first thought was that maybe – somehow – Gatekeeper on those Macs was somehow at fault. Other downloaded apps worked just fine, though, which rather scuppered that theory. My second thought was that maybe there was some issue with the app being flagged as damaged by the Macs, so I tried manually adding the apps to quarantine using xattr, like so:

sudo xattr -rd com.apple.quarantine /Applications/Microsoft\ Remote\ Desktop.app

(Spoiler – the app was Microsoft Remote Desktop).

Finally, I stumbled across the codesign command (installed as part of Xcode Command-Line tools). I’d run into it before while tinkering around with homebrew, and on reading the man page found that it had options for removing, altering, and replacing existing code signatures. Downloading the Xcode Command-Line tools can be done from the Terminal.app like so:

sudo xcode-select --install

The first move was to remove the existing code signature:

sudo codesign --force --deep --remove-signature - /Applications/Microsoft\ Remote\ Desktop.app

Next, now that the existing signature has been removed, we can re-sign the app (using the --force flag to actually replace the existing signature and --deep flag to ensure that any sub-hosted code signatures are also replaced) by issuing the following command:

sudo codesign --force --deep --sign - /Applications/Microsoft\ Remote\ Desktop.app

Thankfully, this worked like a charm, allowing all parties to return to their regularly scheduled weekend drinking. I mean families. Right? Right.