Easy Mode: Synchronizing Multiple Processes with File Locks

Thanks to the introduction of new language features and frameworks such as C#’s async keyword more and more developers are learning how to use parallelism and multi-processing in production contexts. One of the consequences of this, however, is having to learn how to use synchronization mechanisms to serialize access to shared memory and other resources.

In mobile app development the entire conversation around synchronization is focused on synchronizing access to resources within the context of a single running application – because there are no major mobile app platforms that allow you to run multiple parallel instances of the same application.

In desktop app development, however, it’s totally common for end-users to be able to launch multiple instances of your application and therefore it becomes more important to understand how to synchronize access to shared resources across multiple distinct processes.

There are two common use cases for process synchronization:

  1. The “single application instance” pattern – this means using a synchronization mechanism to prevent the user from launching more than one instance of your application. If you’re writing a Windows tray client that runs in the background and does file synchronization, remote backup, or anything else like that then you need to make sure that the user can’t accidentally kick off multiple instances of your backup software in parallel.
  2. The “shared access to resource” pattern – suppose you have multiple processes that can all write some critical data to the user’s registry on Windows or perhaps an important settings file; you have to pick some method for synchronizing reads and writes to this data if your application can have multiple instances running in parallel.

In the case of the MarkedUp, we need interprocess synchronization between our customer’s apps for the following two cases:

  1. Ensure that only one instance of a customer’s MarkedUp-enable desktop application can empty its “retry” queue – this is specific to a single customer’s application; if an end-user’s machine goes offline or if there’s an issue with our API, the MarkedUp SDK will save all of its undeliverable messages into a retry queue on-disk and retry sending them the next time the application runs OR if Internet connectivity is restored. If there are multiple processes running, we need to make sure that only one process can do this at a time.
  2. Ensure that only one In-app Marketing message can be displayed to a customer at a time, across multiple MarkedUp-enabled applications from different software publishers – this is trickier, because we have to synchronize the In-app Marketing TrayClient across all MarkedUp-enabled applications running on a single user’s desktop.

Given these requirements, we decided that the safest approach to inter-process locking was to use an old-fashioned file lock!

How File Locks Work

The concept of a file lock is simple and it’s fairly ubiquitous in the real-world – anyone who’s done a lot of work with Ruby (Gemfile.lock) or Linux has used a file lock at some point in time.

A file lock is implemented by having multiple process look for a file with the same name at a well-known location on the file system – if the file exists, then each process knows that the lock has been held at some point in time. The lock is acquired when the file is written to disk and the lock is released when it is deleted.

 

Typically the lock file will contain some data about the ID of the process that last held the lock and the time when the process acquired it – that way processes can have “re-entrant locking” and lock timeouts, in the event that a process was forcibly killed before it had a chance to explicitly release the lock.

Here’s an illustrated example:

file-lock example

Process 1 acquires the lock by writing markedup.lock to the file system – and then it does its work against the shared resource. Process 2 attempted to acquire the lock but since the file is already in use, it has to wait until Process 1 releases the lock.

file-lock example step 2

Process 1 releases the lock by deleting the markedup.lock file. Process 2 attempts to acquire the lock later and discovers that the markedup.lock file is missing, so it successfully acquires the lock by recreating the markedup.lock file and does its work against the shared resource.

File Locks in Practice (in .NET)

We picked a file lock over an OS-level mutex lock primarily because we couldn’t guarantee that the same thread that acquired a lock would be the same one that released it (required on Windows) – plus we wanted to have our own recovery mechanism to timeout locks.

We weren’t able to find a C# implementation of a file lock, we wrote and open sourced a simple .NET 3.5+ file lock implementation on Github – it’s licensed under Apache V2.0 so it should be safe for you to use in commercial software.

Here’s a quick demo of it in-action, synchronizing 50 .NET processes serially.

Want to use this inside your own application? Make sure you check out the source for file-lock on Github!

MarkedUp 1.3.5: Now with .NET 3.5 and .NET 4.0 Client Profile Support

Earlier this evening the MarkedUp team pushed MarkedUp v1.3.5 to NuGet – the changes include some relatively minor bug fixes, but the major new change is the addition of both .NET 3.5 and .NET 4.0 client profile support for Windows Desktop applications.

NuGet will automatically install the appropriate version of MarkedUp into your application, depending on your Visual Studio project settings.

We added support for these platforms at the behest of end-users, so make sure you submit your ideas and feature requests via MarkedUp’s UserVoice!

Speaking of which, we’ve had a number of requests for strong-naming our analytics client for .NET 3.5 / .NET 4.0 / .NET 4.5 – what do you think? Should MarkedUp add strong-naming support to its NuGet packages by default? Let us know!

Full Support for Windows Desktop Applications– Native C, C++, WPF, and Windows Forms

We promised it, and now we’ve delivered it. As of today, MarkedUp now boasts full support for all flavors of Win32 / Windows Desktop applications, including native Win32 applications written C/C++, Windows Presentation Foundation, and Windows Forms.

win32 announcement

Our documentation has already been updated to include the SDK reference and integration requirements for all three of these platforms, so you can find all of the good technical stuff there. We support everything that the WinRT and Windows Phone SDKs do.

Some statistics:

  • We’ve successfully integrated MarkedUp Analytics for Native C/C++ into popular, well-known applications whose codebases are 22 years old (originally written for Windows 3.1) without any issues. If MarkedUp can run there, it can run anywhere.
  • MarkedUp has been successfully deployed on over 250,000 Windows XP, Vista, Windows 7, and Windows 8 system without any problems. If MarkedUp gets deployed onto a system that’s missing one of our system requirements, MarkedUp is smart enough to simply no-op all of its calls and fail gracefully for those users.
  • The entire MarkedUp SDK for Native C/C++, including the tray client for In-app marketing, adds just 500k to the size of an install package when compressed, and has had no measurable impact on download completion rates or install rates for any of our beta customers (a good thing.)

Now, here’s the exciting part…

Introducing MarkedUp In-app Marketing Automation for Windows Desktop

MarkedUp In-app Marketing has been our secret skunkworks project for quite some time now, and we’ve made it available first for Windows Desktop applications.

A sample in-app marketing notification for Windows Desktop

MarkedUp In-app Marketing allows app developers to improve user retention, time spent in app, conversion, and revenue through targeted campaigns of push notifications that are delivered instantly to end-users based on their behavior inside the app.

 

Once you’ve installed MarkedUp into your app, you never need to update your application again in order to create new campaigns, new messages, or target new groups of users – you can do all of this on-the-fly from your MarkedUp dashboard.

You can learn more about MarkedUp In-app Marketing by reading today’s announcement, but suffice it to say – there is nothing like this that exists for Windows Desktop, web, or mobile developers today.

We’ve done what MixPanel, Urban Airship, and a boatload of other well-funded companies should have figured out a long time ago: horizontal integration between event analytics and marketing automation makes things really simple for everybody.

Add Your First Windows Desktop App to MarkedUp

Want to try all of this good stuff out? Good – it’s still just as easy as before.

Create a new app and select any of the following from the “primary platform” list:

track-new-desktop-app

And that’s it – you’re off to the races.