A Tale of Middleware and Woe

May 3rd, 2019 | by | general

May
03

(This is a more technical article than usual. Programmers might find it interesting. Everyone else, just trust me: it is, every word, sage and true.)

Middleware is computer software that provides services to software applications beyond those available from the operating system. It can be described as “software glue”.

Middleware makes it easier for software developers to … focus on the specific purpose of their application.


Wikipedia

Part I: Fool me once

It started with the decision to use a cross-platform library for Fade In Mobile to try to cover both iOS (iPhone/iPad) and Android with one codebase.

I wanted to make a slimmed-down version of Fade In for mobile devices, for reading, reviewing, and editing on the go. At this time I had no real experience with either Android or iOS development, although I’d had a relatively good experience taking a similar approach to developing the desktop version of Fade In on Windows, Mac, and Linux. Specifically what I didn’t realize at the time was how completely divergent the two mobile operating systems were. For instance, while desktop OSes seem quite different on the surface — they look different, have specific interface idiosyncrasies, and each has its own dedicated and often vocal adherents — when it comes right down to it, the main ones are more the same than different, particularly structurally and even somewhat at a technical level. Or, just as importantly, they can be abstracted1 to a degree where cross-platform support, i.e., writing a mostly shared codebase that compiles and runs on multiple platforms, is reasonably achievable.2

Mobile operating systems, on the other hand, not only use entirely different languages — on Android, you’re (mostly) limited to Java, and on iOS you are, or at least were at the time I embarked on this adventure, (mostly) limited to Objective-C — but their very application models and programming architectures are so completely different that overall app design and structure cannot be transferred cleanly from one mobile platform to the other.

I did not know any of this at the time.

A product called Appcelerator Titanium promised rather audaciously to bridge that mobile development gap by providing a common API (Application Programming Interface) written in Javascript — something that started off as a toy language for running simple code on early webpages and has since grown distressingly in use, and which now most regular users might be familiar with through warnings that sites using Javascript may expose them to all sorts of vulnerabilities.

Anyway. Javascript. It’s not a terrible language. Other than being type-unsafe, generally unsafe (depending on the application), slow (and even slower at the time), inconsistently implemented, difficult to debug (and even more difficult at the time), unpredictable in its memory management, and a host of other things. A lot of working in Javascript involves learning how to work around Javascript. People do use it, though, and not all of them are terrible people.

The appeal of Titanium’s approach is obvious and seductive: instead of having to write and maintain two entire different apps from the ground up — one in Java and one in Objective-C, neither language I was anywhere near expert in — a single app could be written (okay, in Javascript, but at least I had a little familiarity with it) for both mobile platforms at once! And it is…so easy to get rolling. With just a little bit of work you can get a basic app running on both platforms with no trouble at all!3

And so you dive in.

That first basic app is so easy.

Or, as the dealer says: first one’s free.

I’m not a terrible swimmer, but I assume if I ever tried to swim across, say, a small lake (which I never have) then there’d almost certainly be some point at which I realized this was a really bad idea, and would have to consider in a hurry whether it would be smarter to turn back or keep going, and which choice involved the lower probability of drowning.

That was the feeling I had, after a while, with Fade In Mobile and Titanium.

Turning back would mean rewriting, from scratch, the app for each of Android and iOS, getting up to speed on the programming language for each, the platform architecture, the libraries and components, etc.: in other words, a big, time-consuming job. On the other hand, the app(s) would be, hopefully, better and faster and more readily extensible to take advantage of the unique and powerful features of each platform.

To keep going would mean continuing to fight with the Titanium layer, which despite its intentions just wasn’t up to the promised task, which had issues with memory management, was trying to reconcile the differing application models, was trying to provide a common interface for things that were, for instance, synchronous on one platform and async on the other (usually by throwing locks and semaphores and timeouts at it), was trying to implement a UI layout system that worked with the two wildly divergent underlying layout systems and in doing so worked smoothly with neither, and the behavior of which had a tendency to break or at least change significantly enough with new Android or iOS releases that it meant the introduction of more and more version-specific conditional code.

Some of the problems I began to address by delving into the Titanium source code. Titanium’s source was available to view and modify (locally4), but to maintain a separate codebase with my modifications that was still in sync with the official development distribution wasn’t a tenable long-term plan. Plus it meant getting up to speed on Android and iOS platform-specific development, anyway: precisely the thing that using Titanium was supposed to avoid.

During more than a few stretches, the work being put into the little Fade In Mobile app far outweighed the work being done on the main desktop application: a much bigger, much more powerful, much more complex codebase being used by more people for more projects.

So a couple of years ago I decided enough was enough and got serious. And by that I mean I made two new folders in the Fade In development directory: mobile/android and mobile/ios.5

I already knew a little Java and figured, from what I knew and had heard about the two operating systems, that the Android version would be harder (plus my walking-around phone is an Android) so I tackled that first. It took about a year, alongside other work, to get a complete and stable version of Fade In Mobile completely rebuilt from scratch as a native Android app. It turned out pretty well — and was a genuine improvement from the previous version.

Then I turned to the iOS version. In the time since I’d started this all, iOS development had switched from Objective-C to Apple’s own in-house-developed language Swift.6 So that took a little more time. Swift is sufficiently C-like (and therefore C++-like, and Java-like) to get going quickly on, but sufficiently un-C-like to generate regular moments of bewilderment and staring dumbly at the screen. At any rate, it’s necessary to pick up Swift in order to do iOS development, at least for now, at least until Apple changes its mind again. And more importantly, after a few more passes and refinements this time around, the iOS version was ready to release.

Part II: Fool me twi — dammit

The title of this little article is “A Tale of Middleware and Woe”, and its central message is one about the problems that can await a developer who puts too much trust in a third-party’s software. In this case, for the first version of Fade In Mobile, that was the Titanium platform, which in the end just wasn’t up to the task, and the app was written natively for each of Android and iOS.

The thing is, it occurred to me that, since everything was being rewritten from scratch anyway, it would be highly desirable and maybe even necessary to add syncing for Google Drive and Microsoft OneDrive (and iCloud on iOS) in addition to the previous Dropbox-only option.

Now of course, and surprising no one, Dropbox and Google Drive and OneDrive and iCloud have completely different ways of authenticating and accessing and interacting with them. Whereas in an app, users want consistency. For that matter, developers want consistency, too. And there was a solution: a company named CloudRail made a Universal API to access, among other things, cloud storage for all the services I wanted to target, using a single, consistent set of functions. This is literally the very thing that middleware is for. It also promised to be future-proof in that service requests did not go through their server (as they did for another, similar library I looked at) but were dealt with on the device and sent directly to the service itself. So I signed up for CloudRail and used it as the cloud service layer for Fade In Mobile.

And then?

And then CloudRail dumped their Universal API product overnight without a word.7 As it turned out, it was not future-proof — since authentication was still going through their server, or would be if their server was still there — and Fade In Mobile was left without functioning cloud service access (at least on iOS; the Android authentication seemed to still be working). Meanwhile a whole host of iOS users who were now using the new app were running into difficulties with it simply not working.

After eventually realizing what had gone wrong and cursing CloudRail eternally, I set about reimplementing the cloud service layer as quickly as possible, writing all-new implementations while apologizing to those users for whom things were not working right and enjoying some new one-star reviews. (I’ve got to say this about one-star reviews: I understand you’ve got a complaint. But it’s far more constructive and helpful — that is, if you actually want the problem addressed — to communicate intelligently with the developer instead of just stamping your feet, posting your protest rating, and hoping they’ll realize how mad you are and fix the problem. If they see your review at all.)

So what’s the moral of this story, honestly? Don’t trust anyone? Do everything yourself? We’re all going to die alone but by golly we’re going to do it compiling our own self-contained codebases? Admittedly that’s a pretty bad moral.

If you’re reading this, you might (also) be a filmmaker or screenwriter, and you know that collaboration is the name of the game and it’s literally not possible, nor is it advisable, to do everything yourself.8

When it comes to software development, though, I guess it’s more a case of caveat programmator. Software is an ongoing concern. It never gets “done”. There are a million moving parts, and when some of those parts are other people’s black boxes, then by definition you’re going to be insecure. And these days nothing is save. Even Google dumps stuff left and right. (Remember all the projects hosted at code.google.com?) But the more you do it, the more you’ll get a sense of where you’re exposed. Pay attention to your spider-sense. Learn from the misfortune of others.

Have a backup plan.

Or plenty of time.


  1. From a programming point of view, abstraction is about reducing things to a common denominator and hiding all the messiest bits.
  2. One of the guiding principles of Fade In development has been to support as many users and as many platforms and as many operating system versions as possible. Personally, while I use a Mac the majority of the time — largely because I like my Macbook Pro — I’m as platform-agnostic as anyone I know. Put another way, I’m an equal-opportunity complainer. I will grumble about Apple every day with dedicated regularity. When working on Windows I will wonder how it can possibly get worse. When on Linux I will raise my eyes to the heavens and pray for a quick and merciful end. That said, other people’s reasons for using one operating system over another range from economic to political to religious, and I’ve never wanted their choice to affect the writing tools available to them if I could help it.
  3. There were a couple of other alternatives to Titanium at the time, making similar promises and using similar approaches, but Appcelerator’s product seemed — thanks to some effective marketing claims — to be the best choice.
  4. It wasn’t open source in the true sense of the descriptor.
  5. As I found out, I wasn’t the only one who reached this point and took this step.
  6. And, in predictable Apple ecosystem fashion, Objective-C had fallen in Apple developer eyes from a language that received glowing praise and acrobatic justification for all its quirks and shortcomings to one of the most dreaded.
  7. They “pivoted” to another product, this one some sort of IoT thing. Who knows. I think the Universal API might’ve actually been a pivot from a previous pivot.
  8. Although sometimes, budget-wise, it’s unavoidable.

Authored by

Comments are closed.

© 2019 Kent Tessman

▲ Back to top