Easy Mode: Synchronizing Multiple Processes with File Locks

Thanks to the introduction of new language features and frameworks such as C#’s async keyword more and more developers are learning how to use parallelism and multi-processing in production contexts. One of the consequences of this, however, is having to learn how to use synchronization mechanisms to serialize access to shared memory and other resources.

In mobile app development the entire conversation around synchronization is focused on synchronizing access to resources within the context of a single running application – because there are no major mobile app platforms that allow you to run multiple parallel instances of the same application.

In desktop app development, however, it’s totally common for end-users to be able to launch multiple instances of your application and therefore it becomes more important to understand how to synchronize access to shared resources across multiple distinct processes.

There are two common use cases for process synchronization:

  1. The “single application instance” pattern – this means using a synchronization mechanism to prevent the user from launching more than one instance of your application. If you’re writing a Windows tray client that runs in the background and does file synchronization, remote backup, or anything else like that then you need to make sure that the user can’t accidentally kick off multiple instances of your backup software in parallel.
  2. The “shared access to resource” pattern – suppose you have multiple processes that can all write some critical data to the user’s registry on Windows or perhaps an important settings file; you have to pick some method for synchronizing reads and writes to this data if your application can have multiple instances running in parallel.

In the case of the MarkedUp, we need interprocess synchronization between our customer’s apps for the following two cases:

  1. Ensure that only one instance of a customer’s MarkedUp-enable desktop application can empty its “retry” queue – this is specific to a single customer’s application; if an end-user’s machine goes offline or if there’s an issue with our API, the MarkedUp SDK will save all of its undeliverable messages into a retry queue on-disk and retry sending them the next time the application runs OR if Internet connectivity is restored. If there are multiple processes running, we need to make sure that only one process can do this at a time.
  2. Ensure that only one In-app Marketing message can be displayed to a customer at a time, across multiple MarkedUp-enabled applications from different software publishers – this is trickier, because we have to synchronize the In-app Marketing TrayClient across all MarkedUp-enabled applications running on a single user’s desktop.

Given these requirements, we decided that the safest approach to inter-process locking was to use an old-fashioned file lock!

How File Locks Work

The concept of a file lock is simple and it’s fairly ubiquitous in the real-world – anyone who’s done a lot of work with Ruby (Gemfile.lock) or Linux has used a file lock at some point in time.

A file lock is implemented by having multiple process look for a file with the same name at a well-known location on the file system – if the file exists, then each process knows that the lock has been held at some point in time. The lock is acquired when the file is written to disk and the lock is released when it is deleted.

 

Typically the lock file will contain some data about the ID of the process that last held the lock and the time when the process acquired it – that way processes can have “re-entrant locking” and lock timeouts, in the event that a process was forcibly killed before it had a chance to explicitly release the lock.

Here’s an illustrated example:

file-lock example

Process 1 acquires the lock by writing markedup.lock to the file system – and then it does its work against the shared resource. Process 2 attempted to acquire the lock but since the file is already in use, it has to wait until Process 1 releases the lock.

file-lock example step 2

Process 1 releases the lock by deleting the markedup.lock file. Process 2 attempts to acquire the lock later and discovers that the markedup.lock file is missing, so it successfully acquires the lock by recreating the markedup.lock file and does its work against the shared resource.

File Locks in Practice (in .NET)

We picked a file lock over an OS-level mutex lock primarily because we couldn’t guarantee that the same thread that acquired a lock would be the same one that released it (required on Windows) – plus we wanted to have our own recovery mechanism to timeout locks.

We weren’t able to find a C# implementation of a file lock, we wrote and open sourced a simple .NET 3.5+ file lock implementation on Github – it’s licensed under Apache V2.0 so it should be safe for you to use in commercial software.

Here’s a quick demo of it in-action, synchronizing 50 .NET processes serially.

Want to use this inside your own application? Make sure you check out the source for file-lock on Github!

MarkedUp 1.3.5: Now with .NET 3.5 and .NET 4.0 Client Profile Support

Earlier this evening the MarkedUp team pushed MarkedUp v1.3.5 to NuGet – the changes include some relatively minor bug fixes, but the major new change is the addition of both .NET 3.5 and .NET 4.0 client profile support for Windows Desktop applications.

NuGet will automatically install the appropriate version of MarkedUp into your application, depending on your Visual Studio project settings.

We added support for these platforms at the behest of end-users, so make sure you submit your ideas and feature requests via MarkedUp’s UserVoice!

Speaking of which, we’ve had a number of requests for strong-naming our analytics client for .NET 3.5 / .NET 4.0 / .NET 4.5 – what do you think? Should MarkedUp add strong-naming support to its NuGet packages by default? Let us know!

Full Support for Windows Desktop Applications– Native C, C++, WPF, and Windows Forms

We promised it, and now we’ve delivered it. As of today, MarkedUp now boasts full support for all flavors of Win32 / Windows Desktop applications, including native Win32 applications written C/C++, Windows Presentation Foundation, and Windows Forms.

win32 announcement

Our documentation has already been updated to include the SDK reference and integration requirements for all three of these platforms, so you can find all of the good technical stuff there. We support everything that the WinRT and Windows Phone SDKs do.

Some statistics:

  • We’ve successfully integrated MarkedUp Analytics for Native C/C++ into popular, well-known applications whose codebases are 22 years old (originally written for Windows 3.1) without any issues. If MarkedUp can run there, it can run anywhere.
  • MarkedUp has been successfully deployed on over 250,000 Windows XP, Vista, Windows 7, and Windows 8 system without any problems. If MarkedUp gets deployed onto a system that’s missing one of our system requirements, MarkedUp is smart enough to simply no-op all of its calls and fail gracefully for those users.
  • The entire MarkedUp SDK for Native C/C++, including the tray client for In-app marketing, adds just 500k to the size of an install package when compressed, and has had no measurable impact on download completion rates or install rates for any of our beta customers (a good thing.)

Now, here’s the exciting part…

Introducing MarkedUp In-app Marketing Automation for Windows Desktop

MarkedUp In-app Marketing has been our secret skunkworks project for quite some time now, and we’ve made it available first for Windows Desktop applications.

A sample in-app marketing notification for Windows Desktop

MarkedUp In-app Marketing allows app developers to improve user retention, time spent in app, conversion, and revenue through targeted campaigns of push notifications that are delivered instantly to end-users based on their behavior inside the app.

 

Once you’ve installed MarkedUp into your app, you never need to update your application again in order to create new campaigns, new messages, or target new groups of users – you can do all of this on-the-fly from your MarkedUp dashboard.

You can learn more about MarkedUp In-app Marketing by reading today’s announcement, but suffice it to say – there is nothing like this that exists for Windows Desktop, web, or mobile developers today.

We’ve done what MixPanel, Urban Airship, and a boatload of other well-funded companies should have figured out a long time ago: horizontal integration between event analytics and marketing automation makes things really simple for everybody.

Add Your First Windows Desktop App to MarkedUp

Want to try all of this good stuff out? Good – it’s still just as easy as before.

Create a new app and select any of the following from the “primary platform” list:

track-new-desktop-app

And that’s it – you’re off to the races.

The Future of MarkedUp: Support for all Major Native App Platforms

MarkedUp Users and Customers,

If you’ve logged into your MarkedUp Analytics dashboards in the past day or so, you might have been surprised by the new look and feel of our site. We hope you enjoy it!

And it’s not just the look and feel of our site that’s different.

MarkedUp: No Longer Only for Windows 8, Windows Phone

The first thing you’ll see on markedup.com today is this:

Future native app plaftorms MarkedUp Analytics will support: Win32, Android, iOS

Our team has enjoyed working with WinRT and Windows Phone developers over the past year and a half, and we want to bring our services to developers working on other platforms too!

Our goal is to enable developers of all shapes and sizes ship better, more satisfying software to their end-users – and we want to do this on every platform developers care about!

Sign up for early access to Win32, iOS, and Android updates from MarkedUp

We’ll keep you updated on our latest progress for each platform, including access to early binaries and platform-specific services.

Click on any of the links below to sign up!

Coming Soon: MarkedUp Analytics for Windows Desktop Applications

win32

The first new platform that MarkedUp will fully support is Win32 – the platform used for building traditional Windows desktop applications.

We may have mobile and touch platforms like iOS and Android to thank for the resurgence of native application development over the past five years, but the Windows Desktop is the original developer platform.

The Windows Desktop economy is strong and growing, doing over $100b+ a year in license + services sales directly to end-users annually.

However, it’s a market that largely predates the Internet – and thus it’s had trouble adopting all of the software sales + marketing + product best practices that are commonly used throughout modern software development shops, largely due to lack of third party services and tools.

Modern software development practices depend on connected services like analytics and marketing automation to in order to make data-driven decisions, and it’s MarkedUp Analytics’ intention to finally make some of these services available to Windows Desktop developers.

Sign up for MarkedUp Analytics for Windows Desktop (Win32) waitlist.

The details

MarkedUp Analytics will support the following flavors of Win32 development:

  • Windows Forms: .NET 3.5 and later
  • Windows Presentation Foundation: .NET 3.5 and later
  • Native C/C++: Windows XP and later

Holy crap, you’re supporting C/C++ applications for Windows?!?!?!

Yes! Our native C/C++ components will support the all of the same APIs that are available in the .NET flavors of our Win32 SDK.

Worth noting: our C/C++ SDKs will depend on .NET.

How will your WPF / WinForms support compare to your WinRT and Windows Phone APIs?

They’re virtually identical. The only major difference is that our .NET 3.5 / .NET 4.0 / .NET 4.5 SDKs for WPF and WinForms development will expose more methods for manually handling events such as app start / termination.

Win32, by its very nature, is a much more open platform than WinRT / Windows Phone and thus the developers have a lot more options when it comes to how they manage the lifecycle of their applications.

Thus, we decided it would be inappropriate for MarkedUp to try to automate some of the things we do on WinRT and Windows Phone.

How can we access the Win32 binaries for MarkedUp Analytics?

As with our WinRT and Windows Phone SDKs, you will be able to download MarkedUp’s WinForms and WPF SDKs via NuGet.

For the native C/C++ SDKs, we will make downloads available via our CDN.

When will you make the Win32 SDKs available?

Find out.

iOS and Android

There are a ton of choices for iOS and Android when it comes to app analytics and reporting – when we were developing apps ourselves, we frankly felt that many of these services were difficult to use, had sub-standard reporting, and non-existent service.

We still feel that way.

Thus, we are making it a goal of our to provide support for iOS and Android in the near future! They’re further out than Win32 support, but we’ll keep you updated on the latest.

Best,

-The MarkedUp Analytics Team