Developer Tip: How to Install the Ink System Preferences Pane Without a Graphics Tablet

On twitter, the question was posed if the Ink System Preferences pane could be installed without having a supported graphics tablet.

I immediately jumped to answer: YES, there is!
I remembered because I’ve had a need for this myself, way back when the OS X sandbox was introduced.

Why I Know This

Every now and then, I get an eMail from a graphics tablet user, describing how they suddenly aren’t able to drag files out of Yoink anymore. No warning. No apparent signs to why Yoink wouldn’t do its duty anymore. And all the console would say is “sandbox: deny hid-control”.
Thankfully, now, I know what it means and I can give users directions on what to do. However, after all these years, Apple still hasn’t done anything (not one thing!) about this bug in the sandbox, and it’s insanely irritating.

For me to be able to confirm that it had in fact to do with OS X’s handwriting recognition, aka Ink, i had to find a way to activate it, without actually having a graphics tablet at my disposal.

I tried doing the obvious thing (navigating to /System/Library/PreferencePanes/ and double-clicking Ink), but that only resulted in this error message:

Ink Install Error

Install The Ink System Preferences Pane

If you, too, need to test some stuff with Ink, even though you currently don’t have a graphics tablet handy, here’s how to do it.

After a lot of trial and error, this is the working solution I came up with:

  1. In Finder, navigate to /System/Library/CoreServices/Menu Extras
  2. Double-click onto Ink.menu

This item will appear in the right part of your menu bar (notice how it’s perfectly updated for the dark appearance of the menu bar… NOT) :

Ink menu bar icon

Now, click on Ink Preferences… and it will launch System Preferences for you, installing the Ink preference pane:

Ink preference pane

A restart of System Preferences removes the preference pane again, though you’ll continue to be able to install it using the menu bar item.
Should you have the wish to remove Ink from your menu bar, hold down the command key (⌘) on your keyboard and drag the item off, then release the mouse button. The poof-animation will appear, indicating that the item has been successfully removed.

Eternal Storms Software Logo

– – – Do you enjoy my blog and/or my software? – – –

Stay up-to-date on all things Eternal Storms Software and join my low-frequency newsletter (one mail a month at most).
Thank you :)

Under-The-Hood Improvements in Yoink 3.2

Yoink 3.2 comes with a handful of new features. In this post, I will … NOT tell you about those. Instead, I’d like to focus on what changed under the hood.
I think it can be interesting to reflect on how things change from release to release and what improvements were made. And how.

I’ll write about performance improvements and perceived speed-ups in the app and how I managed to improve Yoink’s handling of resources within the OS X sandbox to be able to hold more files than it was previously able to.

Performance Improvements and Speed-Ups

Let’s begin with low-hanging fruit, shall we?

I improved the responsiveness and perceived performance of Yoink in a couple of subtle, but nonetheless important places:

Make “Clear All” Faster

This is the perfect case of “put work on another thread to make the app seem faster”.

Files that don’t exist on the disk yet (like images from websites, text snippets from documents, a file from an FTP client, etc.), Yoink has to create and therefor, when they’re removed from Yoink, they need to be deleted from disk as to not waste space.

For this, I have a method named -prepareForRemoval. It checks if the file was created  by Yoink itself and deletes the file from the disk.

Now, the way “Clear All” works is fairly straight forward:

  1. Iterate over each item in Yoink
  2. Check if the file is currently pinned in Yoink. If it is, skip it; if it isn’t, continue with
  3. Call -prepareForRemoval that cleans up any files that we don’t need anymore
  4. Update the tableView
There’s a few things I’ve improved here:
For starters, for removing files out of the array that powers the tableView, I had used another array that held the items to be removed and after the loop was done, removed the files from the main array.
This I’ve replaced with -filterUsingPredicate:. It’s not only much cleaner to read, but also more performant and using less resources (getting rid of the superfluous array and the second loop for iterating over it for removal).
Secondly, I took step #3 (calling -prepareForRemoval for each item) out of the filtering method and put it on a background thread that gets executed concurrently with everything else, making the filtering much faster.
It doesn’t make any difference for just a few files, but once there are more than 20 files in Yoink and they’re all cleaned out at once, you can definitely begin to notice it.
It’s almost instant now, no matter how many items there are in Yoink.

Make Splitting Up Stacks Faster

A Stack in Yoink is an item that encapsulates multiple files, created by dropping more than one file to Yoink in one single file-drag. The Stack can be split up so individual files can be dragged out of it.

A Stack in YoinkA Stack in Yoink

Splitting up works fine, but again, if a Stack holds more than just a few files, there might be slow-downs. I can’t entirely get rid of those because of the work that needs to be done, but I can speed up the process.

It used to be a simple for-loop, iterating over the files in the Stack and creating a new item in Yoink for each one.
Now, the method uses concurrent enumeration, by calling
[array enumerateObjectsWithOptions:NSEnumerationConcurrent usingBlock:^(aBlock)].
It makes the system create as many threads as it sees fit under the current system load and configuration and loops over the array concurrently, making splitting up larger Stacks noticeably faster.

Accepting a Drop

Accepting a drag in Yoink

Because of changes equivalent to brain surgery (which I’ll discuss below), I had to speed up accepting files in Yoink, otherwise it would be perceived as slower than the previous version, and that’s unacceptable.

Again, NSEnumerationConcurrent to the rescue. It can do wonders.

On the other hand, sometimes it can be slower than an ordinary for-loop, because there’s the overhead of the thread-creation that goes on behind the scenes.
For example, the equivalent to NSEnumerationConcurrent for sorting is NSSortConcurrent. If you use it for something very simple, though, like comparing two strings, it doesn’t do much good. For more complex comparing, it might speed up your method. Another example, NSEnumerationConcurrent would not have given me any speed-ups before the brain-surgery in Yoink.

Removing Sandbox Limitations in Yoink (a.k.a Brain Surgery)

In older versions of Yoink, the app would start behaving incorrectly once a certain amount of files were added to it over the course of the app’s session lifetime. It’s a direct result of how the sandbox hands NSURLs to your app.

Files can be added to an app via the Open dialog (known inside the sandbox as Powerbox) or via drag and drop – which is what Yoink is doing.
These files come in the form of NSURLs. In the sandbox, NSURLs that hold references to files can’t just be used nilly-willy, access to them has to be started (-startAccessingSecurityScopedResource) and stopped (-stopAccessingSecurityScopedResource).
When a file is added through the powerbox or a drag, the system automatically calls -startAccessing(…) for that NSURL for you before it is passed to you in code (without telling you. Also, it causes problems: see Addendum #1).

The documentation has the following to say about -startAccessing(…):

If you fail to relinquish your access to file-system resources when you no longer need them, your app leaks kernel resources. If sufficient kernel resources are leaked, your app loses its ability to add file-system locations to its sandbox, such as via Powerbox or security-scoped bookmarks, until relaunched.

So, -startAccessing(…) and -stopAccessing(…) have to be balanced, otherwise resources are leaked and no further files can be handled by the app – which is exactly what caused earlier versions of Yoink to misbehave once a certain amount of files have been added to it, because I didn’t balance the -startAccessing(…) call the system made before handing the NSURL to me.

The obvious solution, then, is to indeed call -stopAccessing(…) once you receive the NSURL and call -startAccessing(…) once you really need to access the file. But there’s where things go downhill. Once you stop accessing an NSURL, the sandbox believes you never had access to it in the first place, and any subsequent call to -startAccessing(…) fails. Unless…

NSURL’s Security-Scoped Bookmarks (the actual Brain Surgery Part)

Unless you create a security-scoped bookmark of the NSURL first and then call -stopAccessing(…). In versions prior to 3.2, I created bookmarks only when the app quit, in order to keep access to files over relaunches and restarts.
While the app was running, though, I only handed around and worked with NSURLs (with the limitation that once ~1.500 files were added to it, the app would seize to work properly and needed a restart).

This is what I do now, to allow for many, many more files than ~1.500 to be added during the app’s run:
The first thing I do when I receive an NSURL from the drop is create a bookmark from it. Subsequently, I call -stopAccessing(…) (important: without first calling -startAccessing(…), as the system has already done that for me at that point) and the resource is freed – the app is now in a state just like right from launch – as if no file had been added yet.

But now I have an NSData object where an NSURL object used to be – pretty incompatible. That’s where the brain surgery came in. It sounds trivial, but switching from NSURL to the bookmark and making sure the resource is accessed in all the right places (and not unnecessarily) and also cleaned up was an unexpected amount of work.

But long story short, the brain surgery worked, and the patient is up and well. No spasms. No drool.

Eternal Storms Software Logo

– – – Do you enjoy my blog and/or my software? – – –

Stay up-to-date on all things Eternal Storms Software and join my low-frequency newsletter (one mail a month at most).
Thank you :)

Resource Usage Improvements

In the second part of this post, I’d like to explain how I was able to reduce not only Yoink v3.2’s memory footprint, but its overall resource usage.

In Yoink, I want to be able to reflect changes in filenames and be aware of file deletion – it’s a better user experience when, if a file is renamed in Finder, that file also updates its filename inside Yoink.
And there’s not much use in having a file in Yoink that’s been deleted a while ago.

Watching Files for Renames

For this, I use dispatch_sources to watch NSURLs.

GCD provides a suite of dispatch sources—interfaces for monitoring (low-level system objects such as Unix descriptors, Mach ports, Unix signals, VFS nodes, and so forth) for activity.

I want all the information I can get right from the start – probably a habit I picked up during the development of flickery, where flickr counts the number of API requests and once you hit a certain limit, you’re cut off.
So up until now, I created a dispatch_source for every single file that was added to Yoink – better safe than sorry, right?

Well, wrong! dispatch_sources are limited as well, as you need to call open() on the files you’d like to watch. The limit lies around 2.500 files (which, if the NSURL-issue I described above hadn’t already limited the amount of files Yoink could accept, would limit the amount of files Yoink could accept).

No. I needed another, saner and resource-friendlier approach.
Sane and resource-friendly? Bye-bye, polling. You’re out of the race. Good riddance.
Now, three facts are painfully obvious, in the case of Yoink. There are three promising places for improvement, as I don’t actually need to watch:

  • … files that were created by Yoink (images from websites, text snippets, any files that haven’t actually been written to disk yet…)
  • … files that are scrolled out of view in Yoink
  • … files that are encapsulated in a Stack

Files that were created by Yoink don’t need to be watched for name-changes

Text Snippet in Yoink

Files Yoink creates itself are inside Yoink’s sandbox container (at /Users/name/Library/Containers/) and managed by Yoink. No need to watch those.
Text snippets especially – their names are created based on their contents.

Files that are scrolled out of view don’t need to be watched for name-changes

Out of view files in Yoink

Imagine this: You have 9 files in Yoink, separately. You have it set up to show 3 at a time. Currently, you see files 4, 5 and 6. You definitely want to be notified when the name of any of these three files changes in Finder, as the user can currently see them in Yoink, so the changes need to be reflected right away.
But what about files 1, 2, 3 and 7, 8, 9? Do they need to be updated if the user can’t see them right now, anyway?

No!

We watch only visible files, and already we’ve reduced the files to watch for name changes from 9 to 3, saving ~67% of valuable resources – which also means more performance (no extra code execution for those 6 files) and less memory usage. (API side-note: In an NSTableView, you can get the currently visible rows by passing the rect you get from -visibleRect to -rowsInRect:).

If a user scrolls, though, the visible files change and the watched files might now be out of view, being completely useless.
Due to this, Yoink needs a mechanism to update which files need to be watched for filename changes. They are updated when:

  • files are added
  • files are dragged out or removed
  • the window’s size changes
  • Stacks are split up
  • a scroll ends
The update mechanism discards the dispatch_sources for previously visible files and sets up watching the newly visible files.

Files that are in a Stack don’t need to be watched for name-changes

Stacks in Yoink

Yoink displays Stacks like you see in the picture above. An icon, with the number of files inside the Stack. There’s actually no need to watch those files, as file name changes wouldn’t be reflected anyway.
This has “opportunity” for resource-savings written all over it.

When to actually update filenames for files that aren’t being watched

So, when are those filenames updated?
Files that are currently not visible might be visible later because the user scrolls to them or files are removed and dragged out.
Files in Stacks might be exposed by splitting them up.
There needs to be some way to update those filenames, even if they’re not actually being watched by the app right now.

Files in a Stack are actually easy – the filenames are created as soon as the user decides to split the Stack up, as Yoink gets the NSURLLocalizedName for each file.

Files currently not visible require a bit more work, but nothing too difficult.
OS X thankfully provides notifications for when a scroll occurs. Depending on the version of OS X you’re targeting, there’s either NSViewBoundsDidChangeNotification (OS X 10.8 and earlier) or the more fine-grained NSScrollViewWillStartLiveScrollNotificationNSScrollViewDidLiveScrollNotification and NSScrollViewDidEndLiveScrollNotification (OS X 10.9 and newer). For a caveat regarding “legacy” mice, see Addendum #2.

When the *WillStart* notification is sent, I start updating all filenames in Yoink in a background thread (all filenames, because I don’t know where the scroll is going to end).
Once *DidEnd* is called, I update the files to be watched with my usual routine of getting the -visibleRect. Then I update the filenames for the now visible files, just in case the background-thread is still running and hasn’t come to those files yet.

Watching For File Deletion

A warning message when a file residing in Yoink is moved to the Trash in FinderYoink 3.2 displays a warning once a session when a fie is moved to the Trash.

Watching for files being moved to the trash or deleted altogether is another beast entirely.
For this alone, I’d have to watch every single file – being in a stack, not currently visible, etc. -, effectively subverting every single improvement I had hoped for above.
There had to be a better way.

Luckily for me, there is.

I had the idea that, if a file or folder could be watched, the Trash itself should be watchable too, meaning that instead of having to watch every single file if it was moved to the trash or deleted, I’d only have to watch one Trash folder for each volume attached to the Mac (because external volumes have their own Trashes).

Since dispatch_source is my go-to-API for watching files (and I have experience with it), I set up a test-app that watched all Trash(es) folders on the Mac. Surprisingly, it worked only for the internal volume, external volume’s Trashes folders couldn’t be watched. Busted.

FSEvents to the Rescue

Just when it looked like all hope was lost, I remembered the FSEvents API:

[The FSEvents] API provides a mechanism to notify clients about directories they ought to re-scan in order to keep their internal data structures up-to-date with respect to the true state of the file system. (For example, when files or directories are created, modified, or removed.)

It was worth giving it a shot, and – again, surprisingly – watching external volume’s trashes for changes worked this time around. Success.
Now I can tell whether a file in Yoink was moved to the Trash or deleted, without having to set up a dispatch_source for every file and without having to do any polling.

In Closing

Having had code in Yoink that could render the app useless after a couple of thousands files had been added makes me uncomfortable and is pretty embarrassing to admit, but I’m glad I got rid of it.
I’m also very happy with the improvements made with the upcoming update.
In many places, Yoink feels much more responsive because of them.

Addenda

Addendum #1

An NSURL you receive from the powerbox or a drag’n’drop operation already has started access to the resource it points to. This causes the problem that file drags with more than ~1.500 files behave incorrectly (your mileage may vary). The first ~1.500 files are added just fine, but suddenly you start seeing something like this in your logs:

Yoink[74545]: Consume sandbox extension for itemIdentifier (9035) with path: /some/file/path from pasteboard failed!

And worse, the NSURLs you receive beyond that point are useless, you can’t access the files they point to.
My understanding is that this would be a non-issue if the system didn’t automatically call -startAccessing(…) before it hands the NSURLs to you and lets you decide when to startAccessing the NSURLs.
As far as I can tell, this is an issue that can not be worked around at this time, and I’ve tried a couple of things. I hope Apple is working on it. One should believe since 10.7, they’ve had enough time.

Addendum #2

A legacy mouse, as in, a mouse that has a traditional scroll wheel, will only call NSScrollViewDidLiveScrollNotification, not *WillStart* or *DidEnd*.
I handle that in Yoink by calling the update method delayed, and canceling the delayed call should another *DidLiveScroll* notification occur in the meantime, this way, I “simulate” *WillStart* and *DidEnd*:

[NSRunLoop cancelPreviousPerformRequestsWithTarget:self selector:@selector(scrollingDidEnd:) object:nil];
[self performSelector:@selector(scrollingDidEnd:) withObject:nil afterDelay:0.15];

Eternal Storms Software Logo

– – – Do you enjoy my blog and/or my software? – – –

Stay up-to-date on all things Eternal Storms Software and join my low-frequency newsletter (one mail a month at most).
Thank you :)

What I… (March 2016)

Another busy month over. Time flies recently, doesn’t it? I feel like as soon as I’ve published the most recent “What I…” post, I have to sit down again and write another one, though an entire month has passed. Weird. And with the recent daylight savings time change here in Austria, I’ve become a little more messed up with my time. It always takes me a little while to acclimate to the “lost” hour. But damn, is it light out now at 7 p.m. I love that.

Enough talk. Here’s March 2016’s edition of What I…

… Did

Interviews for Contract Work
I had two interviews for contract work recently (none of which I’m allowed to talk about), but that was a first for me.
I’ve never done any work for people other than friends (I did some work for green&slimy) and family (I coded Reach ZEN, an iOS game my cousin came up with), so this was a first for me, going to meet people who determined if I was fit for a development job.
The first time, I was a little nervous and ultimately declined the offer, the second one I was more confident and I’ll be helping bring an awesome app to live soon.
Obviously, my own apps won’t come short because of it, that’s something I’ve considered very carefully.

Achievement Unlocked: 9to5mac Reviews Yoink (+ Video) (9to5mac)
I’ve never had an app of mine reviewed on 9to5mac before, so this is an important milestone for me. It’s a very favorable review, and the video that accompanies it is just great.
Thank you, Jeff (@jeffbenjam on twitter).

Optimizing Yoink (website)
I did a lot of optimizing and improved resource management in Yoink. I’ll go into detail in an upcoming blog post.
Suffice to say, I fixed an issue I’ve been battling a long time, where Yoink couldn’t hold more than a couple thousand files simultaneously and denied accepting more after that (in earlier versions, they didn’t even have to be in there simultaneously – it accumulated over time).

Mac Developer Tip: Simulate Memory Pressure (blog)
A short blog post about how to trigger a memory pressure event so you can test your apps under memory sensitive circumstances.

Sponsorship of Beautiful Pixels, The Platter and Digital Dojos (beautifulpixels, theplatter, digitaldojos)
I decided to sponsor a couple of sites again, get the word out about my apps (mostly about Yoink, see the Beautiful Pixels sponsored post and The Platter’s ad).
I usually wait for a good deal, and those sites currently had very nice ones.
It was all pretty spontaneous, except for Digital Dojos, that’s been in the making for a while (my fault, not theirs). Speaking of Digital Dojos, I’ve never sponsored them before, but they seem very enthusiastic about what they do and having me as a sponsor, so I’m very happy with them, too.
I’m very much looking forward to the result of these campaigns ;)

… Downloaded

Duck Tales

Duck Tales on the Apple TV (App Store)
I got it for the iPad some time ago, now they released it on the Apple TV as well (and it’s free for those who bought it already on the App Store).
I love it on the big screen, but the controls (via Siri Remote) are a little clunky and needs getting used to.

Tayasui Sketches+ Icon

Tayasui Sketches+ (App Store)
I guess this will really shine on one of those iPads Pro, but it was on sale, so I gave it a shot, because I’d heard good things. It’s really very nice.

The Executive Icon

The Executive (App Store)
I’m not really into fighting games, but it’s nice if you need to blow off some steam :P

… Read

Disney Infinity 3.0 Not Getting Updates on Apple TV (toucharcade)
<sarcasm>They said the future of TV was apps, not games</sarcasm>
All joking aside, this is disconcerting. Seems like Apple TV app sales aren’t doing so well.

Mac Ransomware KeRanger Distributed Through Transmission (click)
Larger market share sadly means more creeps are interested in the platform…

Craig Federighi: The FBI Wants to Roll Back Security to 2013 (washingtonpost)
I’m a big fan of Federighi’s and this strengthens my opinion of him.

The Simple Rules That Could Transform How You Launch Your Product (launchkit)
Some interesting and potentially helpful tidbits.

How a $2.99 Recipe App Became a Top 2 Paid App in the App Store (medium)
“Launch into an existing community, or piggyback one”

Distributing Apps Outside of the Mac App Store (devmate)
If I were to sell my apps outside of the Mac App Store, I’d probably use DevMate.

The Most Important Apple Executive You’ve Never Heard Of (bloomberg)
A visit with Apple’s chief chipmaker, Johny Srouji.

An Indie’s Guide to the Press (curtisherbert)
“Getting the attention of the tech press can feel hard as an indie”

How the iPad Pro Changed My Illustrating Career (medium)
An open letter to Tim Cook by a 15 year old girl, Zoe, about how the iPad Pro made her a better illustrator and even landed her an illustrating job of a children’s book.

Apple Needs A Preferred Developer Program (medium)
Some good points are made here, but I don’t think creating a two-class-society on the App Store is a good solution to the problems the App Stores are facing.

Inside Apple CEO Tim Cook’s Fight With the FBI (time)
“There’s financial information. There’s your conversations, there’s business secrets. There’s probably more information about you on [your smartphone] than exists in your home.”
You can read the full transcript of the interview here.

… Listened To

Encryption Tightrope Apple / FBI Hearing (youtube)
The (lengthy) hearing of the two parties involved. Very insightful, so if you have some time, be sure to watch it.
I listened to it like a podcast, which you can do as well, it doesn’t need to be watched, imho ;)

… Watched

Columbo DVD Box

Columbo (itunes)
My girlfriend gave me the entire Columbo series for my birthday and we couldn’t stop watching. One of the greatest detectives – ahem, lieutenants, of course – of all time.

Last Week Tonight with John Oliver on Encryption (youtube)
A fun take on the whole Apple / FBI thing.

… Ate

Spicy Bombay PotatoesSpicy “Bombay Potatoes”

… Went to See

Innsbruck, AustriaInnsbruck, Austria

The For a quick weekend getaway, my girlfriend and I took a spontaneous short-trip to Innsbruck.
In the three days we were there, we walked about 30 kilometers.