MarkedUp Analytics Beta for Windows 8 is Now Available

We are thrilled to announce that MarkedUp Analytics for Windows 8 is now available for public use!

markedup-microsoft-surface

Our team worked lots of long hours delivering MarkedUp in time for WinRT developers to get their arms wrapped around it well before the public ship date for Windows 8 (October 26th, 2012) and we are really, really happy with what we built.

MarkedUp is currently in-use in several Windows 8 apps already being distributed in the Windows Store, and it works equally well both for WinJS and C# / XAML.

The best part though is the pricing – while in beta, MarkedUp is absolutely 100% free as in “free beer” for developers. Eventually we will have pricing tiers, but there will always be a perpetual free tier so developers never have to open their wallets when starting a new project.

How to Get a Free Beta Registration Code

There’s been a lot of pent-up demand for MarkedUp, and while we want to make it available to everyone soon, we’re capping registration with limited-use codes.

Want a free MarkedUp registration code? Send us an email to support@markedup.com and we will send you one!

How to Get Started with MarkedUp

So you’re one of the lucky people who’s received a registration code, how do you get started working with MarkedUp?

I recommend watching MarkedUp’s tutorial video for Windows 8 developers – it explains everything including the data dashboard and shows you how to get live data out of one of your Windows 8 applications in 10 minutes or less.

Getting Started with MarkedUp for Windows 8 Developers

If you still need more help, take a look at MarkedUp’s documentation or drop us a line at support@markedup.com

Please let us know if there’s anything else you need – we are eager for feedback and we need users like you to help guide the development of the next versions of MarkedUp!

MarkedUp’s Core Values when it Comes to Software Developers

With the first version of our product just about ready to go to market, we’re thinking a lot more about recruiting and how we want to expand our team at MarkedUp.

MarkedUp is a developer-centric company: we’re building products for developers, by developers. Given that, everyone on the team should have no trouble understanding what it’s like to be in the shoes of our customers (Windows 8 developers.)

So naturally, top of mind for us is recruiting developers – specifically .NET and HTML5 / JS developers who want to make a difference at an early stage company and tackle tough problems like large-scale analytics, data visualization, building developer-facing SDKs and services, and more.

MarkedUp’s business is deeply technical, and so thus the culture we’re building follows suit – and everyone we ultimately want to hire should share our values and be as passionate about solving hard problems as we are.

So what really matters most when it comes to building a company that revolves around technology? Breadth or depth of experience? Volume of published work? Contributions to notable open source projects?

Having had to put some of this in writing recently, I thought I would share it more widely – here are what we at MarkedUp consider to be the three most important values for a developer-centric company:

1. Recognize that software development is an inherently social, team-driven experience – everyone on the team should get into a rhythm of regularly communicating and working together on tough issues, whether it’s done remotely / asynchronously / in-person – whatever.

No one codes off in a corner by themselves at MarkedUp; good programmers do their best work when they’re working with effective teams.

We do our best to make sure our developers have suitable tools and processes for managing their projects and work in concert with their teammates.

2. The ability to communicate concisely is a developer’s most important skill, even more than ability to code – everyone on the team should be able to articulate new ideas, improvements, and issues. If you can’t concisely explain a problem or a new idea, how else are you supposed to win the support of your teammates?

Developers at MarkedUp are encouraged to blog about writing software, give talks at conferences and user groups, and contribute to open source projects – because these are all forcing functions that improve a developer’s most important tool: the ability to concisely and understandably communicate technical content.

3. Continuously learn and improve – a developer’s job is to model the complexity of the real world, and to do that well you need to learn more about the world itself and improve the tools you use to model it.

MarkedUp fields a wide range of technical challenges we have to solve everyday, so we invest a lot into helping our team members improve their skills, learn new technologies, and play with new tools.

We even have a little perk to make sure that every employee at MarkedUp gets a Kindle allowance so they can snap up some new books each month and teach themselves something new. Or read a Twilight novel if they’re into that.

4. Pragmatic programmers only – at a company as early stage as MarkedUp, we have to optimize for flexibility and speed of iteration.

Everything must be designed in a way that it can be changed quickly if needed so we can frequently deliver new versions of our products. Performance, “scalable” design, and all of that other /r/Programming stuff only becomes a concern once it becomes a real problem rather than a theoretical problem.

This is how developers at MarkedUp are expected to approach problems: pragmatically. There is no room for design / platform / language dogma in our shop.

—–

And so, off we go – we’ll probably refine this list over time ;)

MarkedUp is currently looking to hire two .NET developers to join our team – if you have any interest in joining an early stage company and make a difference in our product and your career, send us an email with the subject line “.NET developer” at team@markedup.com

C# Feature Dossier: Tasks Based Asynchrony with Async and Await

If you haven’t been living under a rock for the past year you’ve at least probably heard about the new C# async and await keywords coming in C# 5 that’s shipping in with Visual Studio 2012. In a nutshell what the feature allows you write asynchronous code and make it look like it’s synchronous. That means that you no longer have to write callback handlers to handle what happens when the async operation you invoked has returned. An illustrative example using good old HttpWebRequest should drive the point:

In this first example we used the traditional Begin/End async pattern on HttpWebRequest to request a website and print it to the console. Notice how you have to split your logic between setting up the request and dealing with the result once results come in. In .net 4 this became a bit easier with the introduction of the Task Parallel Library (TPL) and the Task class.

A way you can think about a Task is that it represents a promise of a future result rather than the actual. In other words sometime in the future I’ll have the value you request. What’s nice about this abstraction is that it allows you to combine asynchronous operations into a single operation thanks to since your returning a promise rather than the actual result.

One thing you might be asking yourself at this point is great I can write an async method in one go now by return a promise of something to come. What if I need to act on the result before I return it to the caller? Glad you asked! And that’s where Task continuations come in. They basically allow you to specify an action to take on a Task once it completes. So enough talk, let’s see this stuff in action:

There are a couple things going on here but take note that unlike with the begin/end async pattern everything is neatly contained within a single method with a Task being returned that will contain the async result being returned.. Let’s break this down starting off with the call to the Task.Factory.FromAsync method. FromAsync is a static method provided by the TPL base class library for bridging the traditional async begin/end pattern over to the new Task based async pattern. The method returns a Task representing the result the begin/end async invocation.

That in and of itself is pretty neat but the real magic of this examples lies in the continuation that’s being defined of the Task we got back from the FromAsync method. All tasks have a method called ContinueWith that then allows you act upon the future result of the Task you call ContinueWith on. Basically, you’re saying what you should do once the parent task or antecedent has completed. No longer do you have to define call backs and event handlers to handle async. You can simple call a method and pass in a delegate. This makes writing async methods immensely easier since your async code is all in one place and reads much better.

The last part is this example is returning the task that was given to us by the call to continue with. This the Task that calling code will wait on for the final result of this asynchronous method. Pretty, nifty right? So now that we’ve seen TPL and Task based asynchrony in action how do C#’s new async and await come into the picture. They make writing and consuming task based async methods so easy that they look like synchronous method calls. No callbacks, no wrapping delegates, events hooks. Just code. (Well it’s all just code in the end but you get my drift).

Seeing is believing so here’s what are method looks like with async and await.

 

Like I said, just code :) The interesting to point here are the introduction on two new keywords. The async keyword in the method signature and the await used in front of the methods that have async at the ends of their names. The async keyword marks a method as being asynchronous and allows the use of the await keyword within it.

The await keyword is the bread and butter of all this. What it does in a nutshell is unwrap a Task by introducing a point of asynchrony into the method at which the compiler can logically pause code flow execution until the Task completes. Point of emphasis here “logically pause” code flow execution. Note I didn’t say block but pause.

One thing to be aware of when writing task based methods with async and await is that the method returns after the first await in the method. Which could or could not be what you want. On WinRT and asp.net the thread is released to do other work and will reschedule itself once the Task that is awaited on completes and continue execution after the await. If you try and await on your main method (in actuality you’ll get compile error, but work with me) in a console app you’ve now released your app’s main thread and the program will terminate. Point of this being while async and await do make async work easier to manage you do still need to be aware of their nature.

Hopefully, this you’ve found this high level overview of async and await at least somewhat helpful. But we’ve only just scratched the surface of what the TPL is capable of. We have yet to talk about CancellationToken that help provide unified cooperative cancellation, scheduling of Tasks with the TaskScheduler, concurrent collection primitives like ConcurrentBag and ConcurrentQueue and many more. One last thing parting gift I’d like to leave you with is that if you’re still on .net 4 you can still leverage the async and await keywords in your app. Since async and await are really just a compiler transform all you really need is the C# 5 compiler in Visual Studio 2012 and the set of supporting types which the C# team has made available to .net 4 developers as Nuget package which you can find here.

Introducing Hircine: a Stand-alone Index Builder for RavenDB

RavenDB

Here at MarkedUp, we rely on a variety of technologies to help produce meaningful analytics and reports for WinRT developers who want to know how their applications actually get used in the marketplace.

One of the technologies we use is RavenDB, a “second generation” document database built on top of C#, Lucene.NET, and Esent.

We use Raven for some of our “origin storage” – the unprocessed, raw data we receive from apps that have installed our WinRT client.

For the TL;DR crowd: if you’re tired of having RavenDB build its indexes on application startup, which is slow and painful, use Hircine to build your indexes at compile-time. Check out “Getting started with Hircine” if you need help figuring out how to use Hircine for the first time.

How RavenDB Normally Builds Indexes, and Why It Sucks

RavenDB indexes are C# classes which derive from Raven’s AbstractIndexCreationTaskobject.

When the RavenDB client builds indexes on your remote server what it’s really doing is taking instances of your index classes and serializes them over HTTP into a format that the remote RavenDB server can use to define a searchable / sortable index internally.

Most people typically use this method of the RavenDB client to build all of their indexes at once, and usually this happens on application startup (Global.asax):

Raven.Client.Indexes.IndexCreation.CreateIndexes(typeof(SimpleIndex).Assembly, documentStore);

In this instance SimpleIndex is a type derived from AbstractIndexCreationTask and documentStore is an initialized IDocumentStoreinstance connecting to a live RavenDB database.

When this method is called, Raven iterates over all of the classes defined in SimpleIndex‘s assembly and creates instances of every type that is assignable from AbstractIndexCreationTask – once it has that full list of indexes, it synchronously builds those indexes against theIDocumentStoreinstance, which can be really slow and can even fail sometimes.

Image doing this every time your application starts up, particularly if you have a large number of indexes or have to connect to multiple RavenDB servers. Global.asax is not a great place to build your indexes.

How RavenDB Indexes Are Built by Hircine

Hircinedoes things a little differently, but let’s talk about what it has in common with the RavenDB Client approach:

  • Hircine also looks for index definitions contained inside user-defined .NET / C# assemblies.
  • Hircine uses the same method (ExecuteIndex) as the RavenDB client for building each individual index, so the instructions being sent to the RavenDB server are consistent with what the RavenDB Client does.

So what does Hircine do differently?

  • Hircine runs in its own stand-alone executable (hircine.exe), rather than inside your application.
  • Hircine can builds all of its indexes in parallel by default, using multiple HTTP requests and threads to get the job done faster.
  • Hircine can build indexes found in multiple user defined assemblies at the same time, rather than relying on successive calls to IndexCreation.CreateIndexes.
  • Hircine can build indexes against multiple RavenDB servers in parallel, rather than doing them one at a time like the RavenDB client.
  • Hircine works asynchronously against both remote databases and embedded ones for rapid testing.

From this set of behaviors, you start to appreciate Hircine’s goals:

  1. To fully decouple RavenDB index-building from application startup;
  2. To make it trivial and painless to build indexes from multiple assemblies against multiple servers;
  3. To make the process of index building really, reallyfast; and
  4. To provide a simple interface that developers can integrate into their own build or continuous integration processes.

Hircine in Action

Want to see what Hircine looks like in action? Well here’s a peek, which you can see by running the rake command inside the root directory of the Hircine repository on Github.

image

You can learn more about Hircine’s command-line options from the getting started page.

Installing via Nuget

Hircine ships in two different Nuget packages:

Nuget – Hircine – stand-alone executable that runs directly from the command-line.

Nuget – Hircine.Core – core Hircine engine in a callable assembly, for people who want to integrate Hircine into their own projects.

Contributing to Hircine

Hircine is licensed under Apache 2.0 and the source can be found in Hircine’s Github repository.

MarkedUp happily accepts pull requests on its open source projects, so long as any significant changes are accompanied with an acceptable level of test coverage.

Windows 8 App Developer Registration is Now Open! Here’s What you Need to Know.

Today’s an exciting day if your an individual Windows 8 app developer. The store now open. Before you go off and open your shinny new developer account here are few last minute good to know.

If your a STUDENT, MSDN subscriber or BizSpark member you get your Windows 8 developer account for FREE! The links below will get you to the right place. You’ll be given a registration code to enter when you sign up for your Windows 8 developer account.

When registering you’ll enter the code your given in a registration code field as shown below:

If none of the above apply an individual Windows 8 will cost USD $49.00 to register.

Now without further adu the link below will take you directly to the Windows Store dashboard:

Good luck! Hopefully we’ll see you on the flip side as an official Windows 8 developer.

Azure Mobile Services: For Those Who Just Want to Code

If you follow Azure news on the web you might have noticed a new Azure service being offered in preview form called Mobile Services.

In this blog post we’ll be doing a high level overview of what Azure Mobile Services is and what it gives you. In subsequent posts we’ll dive deeper into how to get your hands dirty and actually use the thing.

With this initial preview the Windows 8 app development scenario has been enabled. We’re likely to see other platforms such as iOS and Windows Phone (my personal favorite) come on board as the Azure mobile team develops the new service.

Why should I Care?

The idea behind mobile services is to provide is quick and easy way to bootstrap client app development by providing a supporting backend. If you want to write client apps and don’t want write all of backend web service logic yourself from scratch then you should take look at mobile services.

So here’s what we get out of the box mobile services:

  • Data storage
  • Authentication
  • Push notification

Like any other Azure service you also get a nice dashboard that shows a graph of your mobile service activity. In this case you’re able to see how the number of API calls, CPU time and data going out that service has made over time.

Data Storage

The data service is particularly interesting as it’s backed by Azure SQL Server and uses the JSON version of the OData protocol to communicate on the wire. One of the more interesting features of the data service is dynamic schema support which automatically updates the tables that backs the model as it changes. This makes it extremely easy to get up and running quickly as you no longer have to setup a database schema ahead of time, you can just let the service take care of creating the columns and choosing the appropriate data types in SQL server. Note that before you go into production you should turn off dynamic schema support to ensure that the schema can’t be changed outside of development.

The notion of dynamic schemas should by familiar to those who play with NoSQL document databases such as Mongo and RavenDB. The development experience is also very similar to that of entity framework code first if you ever used that before.

The C# primitives that mobile services is able to map out of the box include the following:

  • Integer types (int, long, etc.)
  • Floating points (double, decimal)
  • String
  • DateTime and DateTimeOffset

For any other data type like TimeSpan or a custom type you will need to convert to string or provide a client side converter that implements IDataMemberJsonConverter and handles the JSON serialization for that type to simple JSON types.

Authentication

One really nice thing about the mobile service is its client authentication support. To set it up all you need to do is register the web service with Windows Live, drop in your client id and secret with the mobile service and call a few methods on the client app and hey presto you’ve got authenticated calls going though.

The authentication service in mobile services is backed using Microsoft Accounts or Windows Live accounts if you haven’t got the branding memo yet. In order to make use of authentication in the client app you will also need the Windows Live Connect SDK. The Live SDK is responsible for displaying the sign in UI and eventually giving you an authentication token which you in turn hand over to mobile services client so it can perform authenticated requests to the backend. The code for this is actually simpler than it sounds.

And that’s all the code it takes to authenticate with our mobile service backend on Azure.

Push Notification

Last but not least we have push notification support which helps us to do push notifications back to our mobile client. The way push notifications work in the mobile services preview today is that they are created using server side scripts that are authored by the developer in the Azure mobile service portal. These scripts are written in JavaScript which get executed on the server when create, read, update and delete (CRUD) operations occur on data in the mobile service.

These scripts are responsible for creating push notifications will comprise the core of your server backend logic as well. They are responsible not only for creating push notifications but for validating and modifying in coming requests and performing authorization logic based on the incoming authenticated identity.

Under the covers, these scripts are actually being executed by node.js, Azure mobile services exposes a small subset of node modules that can be used by the scripts to do their work. We’ll go into more details in another blog post on how to use these scripts and what you can do with them. This is a pretty big topic in and over itself and a simple blub wouldn’t do the topic justice.

Anyways, now that we know where push notifications get created on the server we need to know the how. In order to leverage push notifications on the server the client app is responsible for obtaining the push notification channel URL from the PushNotificationChannelManager from within their app and uploading it to the service. Typically this gets saved as property on the model that you use to save data to mobile service.

The code for this would look something like this:

Once the notification channel is saved to the mobile service the server side scripts will then be able to use that push notification channel url to send push requests back to our app.

Hopefully this post gave you a good overview of how Azure mobile services work. In future blog posts I’ll be doing some deep dives on using specific pieces of the Azure mobile services like scripts, push notifications and the data service.