Deep Learning for Sensor Fusion and Sequence Classification

Deep Learning (DL) has achieved remarkable performance on ‘cognitive’ tasks such as object recognition in images, real-time spoken language translation, etc. These are monumental achievements that have transformed our thinking about what machines could do for us.

DL, it turns out, can also perform well on more mundane machine learning tasks such as sequence classification of high resolution multivariate sensor data. This post presents such a model which utilizes the Microsoft Cognitive toolkit (CNTK).

Continue reading

Recognizing Patterns in Noisy Data using Trainable ‘Functional’ State Machines

Update: A version of this talk is also on Channel9

Finite state machines (FSM) are a common technique for recognizing patterns over streams of data. However basic FSM cannot easily handle noisy data such as sensor data from a wearable device that is generated using human limb movement. This session presents a novel variation of FSM using techniques from functional programming to construct ‘functional’ state machines (FnSM) that are computationally as efficient as FSM but can handle probabilistic patterns almost as well as more elaborate techniques such as Hidden Markov Models.

I presented a talk on this subject at the recent Commercial Users of Functional Programming conference in Vancouver (2015). The slides from my talk are now on SlideShare ( Or use the direct link here (cufp2015 1).

In this talk, I described a real-world application of FnSM; an Android smartwatch app that recognizes gestures performed using wrist and forearm movements. Also described is the use of an evolutionary computing algorithm to optimize the performance of the FnSM by selecting better parameter values for the various state-transition decisions.

We are increasingly living in a word where data processing needs to happen in real-time to enable quicker decision making. FSM are an essential technique for recognizing patterns over sequential data. This talk describes techniques that leverage FP to succinctly define complex FSM. Additionally, such constructed FSM can be trained with the help of a suitable evolutionary computing algorithm to also handle noisy data.

Stormin’ F#

Apache Storm is a scalable ‘stream computing’ platform that is fast gaining popularity. Hadoop and Storm can share the same cluster and the two complement each other well for different computing needs – batch for Hadoop and near-real-time for Storm.

Storm provides a macro architecture for executing ‘big data’ stream processing ‘topologies’. For example, one easily increase the parallelism of any node in the Storm topology to suit the performance requirements.

For streaming analytics, however, Storm does not offer much help out of the box. Often one has to write the needed analytic logic from scratch. Wouldn’t it be nice if one could use something like Reactive Extensions (Rx) within Storm components?

Luckily Nathan Marz – the original author of Storm – chose to enable Storm with multi-language support. While Storm itself is written in Clojure and Java, it implements a (relatively simple?) language-independent protocol that can be used with basically any mainstream language.

FsStorm is an attempt to allow F# (and .Net Rx) to be used for defining and running Storm topologies.

Storm and Rx are big topics in and of themselves. If your are interesting in leveraging FsStorm, it would be best to first understand Storm and Rx using the official and supplemental documentation (blog posts, videos, etc.). It took me a while to get a decent enough grasp of Storm to start putting together FsStorm. Be sure to view one of Nathan’s videos on Storm.

Given sufficient Storm knowledge, take a look at the sample project FsSimpleTest. It includes a topology a spout and a bolt. The topology is described with StormDSL (which IMHO makes defining topologies much simpler than in Java).

In order to run FsSimpleTest first setup a Storm cluster under Windows (for now). Download the repo and compile FsSimpleTest. To submit the topology to Storm, use the Submit.fsx script, included in the project but before you run the script make sure the ‘binDir’ variable is set correctly for your environment. Also the “jar” command – which comes with Java JDK – should be in the path (it is used to package the built components into a Jar file which is uploaded to the Storm cluster).

You should be able to see the running topology in Storm UI – the browser-based console.

While the repo does not include a sample that uses Rx (yet) I have been successfully running a 3-component Storm topology that leverages Fsharp.Control.Reactive (the Rx wrapper for F#).

FsStorm does not utilize all of the capabilities offered by Storm. In particular Distributed RPC and Transactional topologies are not supported.

This is a very early but promising start for stream computing with F#. Contributors are welcomed!

Android Wear Starter Project – F# + Xamarin

Wearable computing is undeniably a hot topic currently, what with all the major mobile players (Apple, Google and Microsoft) having entered the market. There is also a plethora of smaller players such as Timex, Nike, Pebble, etc.

Cheap, ubiquitous wearable computers – along with Internet-of-things – may be the next big wave in technology. As they say, “possibilities are endless…”.

As this blog is dedicated to F# and functional programming, here is a starter barebones Android Wear project for Xamarin – written entirely in F# :

Please go through the repository documentation (“” file). It will give you an overview of the Android Wear app deployment process. There you will also find a list of potential ‘gotchas’ to avoid and tips on debugging and troubleshooting.

VL – a F# DSL for the Layout of IOS Controls

Starting with IOS 6 Apple added Auto Layout – a language for specifying constraints on the layout of IOS controls. With Auto Layout one can express the height, width, sequence, spacing, margins, anchoring / docking, etc. of controls, as constraints.

Auto Layout constraints can be expressed in an untyped way as strings or one can use the underlying API. While feasible for Objective-C code, these approaches become awkward for other languages such as F#.

The VL DSL is meant to easily create IOS layouts in code in a strongly typed manner. The DSL makes use of F# Algebraic Data Types (Discriminated Unions), pattern matching and operator overloading. (Note that at the time of writing this post Xamarin has announced Xamarin.Forms which may make VL redundant).

Auto Layout :       H:| -10-[ctrl1]-[ctrl2]-10-|
VL                :       H [ !- 10. ; @! ctrl1 ; sp ; @! ctrl2;  !- 10. ]

The two constraints are equivalent; they layout two controls horizontally with 10 point margins at the left and right edges and a standard space in between.

Continue reading

Xamarin F# – Experience Report

For the past few months I have been working on an Android application that collects automotive CAN bus data – ultimately for analysis with BigData technologies such as Hadoop. I have already described some aspects of this project here and here. Thus far close to 2 Terabytes of data has been already collected.

In this post I would like to recount my experience of using F# with Xamarin for a real-world mobile application.

Continue reading

OpenXC based Android App

Recently launched an OpenXC ( based app for internal use. The app connects to the OpenXC dongle and collects CAN data for analytics. It is interesting to note that daily capture is in 100 megabyte range (depending on the vehicle and how long it is driven). The app is based on the Xamarin platform and written entirely in F# (also see this post). Here are some screen shots:

Message Rate


Message Frequency Histogram


Talking to your car – with OpenXC, Android, Xamarin & F#

Virtually all vehicles today operate an internal network called the CAN bus. Different modules (or ECUs – electronic control units) in the car communicate with each other via this network (e.g. engine, transmission, dashboard, etc.). In this post I reference a CAN bus reader built with F# and Xamarin.Android based on the OpenXC platform (

Update: Also see this post.

Continue reading