MultiLens is a multi-touch controlled WPF application which uses Magic Lenses to help exploring large node-link diagrams. The goal was to provide an alternative to traditional menu based and mouse controlled systems. Therefore we use multi-touch input to quickly manipulate the lenses, change filters and adjust parameters. Furthermore, Lenses may be combined with other lenses to create generic multi-purpose tools. MultiLens was developed at the Interactive Media Lab Dresden as part of the DFG GEMS Project and resulted in two publications: A demo at ACM ITS 2014 and a full paper at ACM ITS 2016.
General
MultiLens underwent several iterations: It was originally developed by a team of 4, including me. Afterwards, I was the sole developer for roughly half a year. Afterwards I collaborated with a colleague of mine, and several students worked on smaller aspects of the prototype (see history).
The application is written in C# using WPF for the GUI. The whole program is strictly structured regarding the MVVM pattern making excessive use of data bindings to automatically generate and update views from an underlying data model. The lens system in particular allows the combination of multiple lens functions into a single lens to form a single, unified output. For better Performance, a Direct2D Renderer utilizing SharpDX was created as an alternative to WPF’s front end.
For the organizational part we used redmine as a bug and issue tracker and SVN for version control.
Details and Contribution
When this project originally started, using WPF was one of the requirements. As I had never worked with WPF before, I took this project as an opportunity to get acquainted with its associated patterns and best practices. Therefore I made the decision to use the common MVVM pattern and make excessive use of WPF’s data binding functionality. I spend lot of effort on the general design, i.e., how models, views, and view models are structured and organized. This included designing and implementing the graph data structures and the ones for the lenses as well. As a consequence I’m also responsible for much of the initial look and feel of the application.
I’m particular proud of the lens and lens combination system. When to lenses are moved so that their areas intersect, users would expect their functions to be combined. As a Magic Lens can in principle have any arbitrary function, this is easier said than done. The goal to this problem was to organize each function based on precedence. Whenever a function changes the data mode, instead of applying this change, an individual stack is build for each individual aspect than can be changed. This stack can than be combined using one of several available combination strategies, e.g., by calculating the average between every change.
While I spend a lot of time making sure the framework was as flexible and efficient as possible, performance problems appeared with bigger graphs or lots of individual lenses. The reason for this issues were most likely rooted in WPF’s data bindings and the excessive rebuilding of the GUI it triggered. In an attempt to improve performance I implemented an alternative Direct 2D renderer using SharpDX for the bindings. While the performance was indeed much better, there were some timing issues with the refresh of the WPF window and the DirectX context which resulted in random, but frequent black screens. As all attempts to address this issue proved futile, we reluctantly switched back to the normal WPF front end for the publications and demonstrations.
History
The project was developed over the course of several years. Starting in 2013 as part of a student project consisting of myself and three other people. We developed the basic prototype which included the graph, lenses, lens menus and some lens functions. Combining lenses was deliberately not part of this initial project, as we noticed that solving this problem was rather complex.
The next step was me working alone on the project during my master thesis, specifically to solve the question of combining lenses and their functions. Besides the stack based modification systems this also included designing interaction techniques to combine and separate lenses. The DirectX renderer was also a result of this thesis.
After this I started to improve the prototype with a colleague of mine, now as a research associate. We implemented the gesture control, improved the look of the prototype and supervised several student projects that worked on smaller functions (which all sadly did not make it into the final publication). We also implemented Mouse controls and a more traditional menu as a baseline to compare to our approach. Finally, we conducted a user study evaluating user performance and published the associated research paper.