u2vis – Information Visualization Framework for Unity

Development

Developing a generic framework is always challenging if there is only one initial use case which the framework has to satisfy, because the requirements of this project are known and help shape the framework while the requirements of future projects remain hidden. Therefore, I aimed for a flexible and highly modular design that, if not directly applicable to a new project, is at least easy to adapt and extend. The first step in this regard war to clearly separate the data from its representation using the common Model-View-Presenter approach. The next was to make sure that the dependencies between classes and Unity components are as shallow as possible. This is hindered in a sense by unity notorious limitations in working with interfaces when it comes to components and their build in serialization. However, a nifty combination of inheritance and composition solves most of this issues.

Data Handling

The data keeping itself was also a major concern, because it should also be as generic as possible without being focused on a certain kind of data. To guarantee this, the design borrows somewhat from data bases. A data set is separated in dimensions, which represent a specific aspect of a data set, like the price of an item. Each dimension has a specific data type and the current implementation supports dimensions consisting of booleans, integers, floats and strings. However, new dimensions for all kinds of data types can easily be extended. Each type of dimension can also implement an interface which indicates that this is numerical data that can be converted to floats, which is important to actually visualize them. The data itself is provided by a generic DataProvider class, which can have concrete implementations that reads data from a CSV file, streams it over the network, or reads it from a database. In theory, this should make supporting all new kinds of sources possible, because it only requires a new implementation for that particular type of source.

Preparing Data For Presentation

After a data set is available, the next step is to prepare the data for consumption by a particular visualization. This includes selecting which dimensions should be visualized, what range of data items, how the axes should be configured and so on. This is handled by DataPresenters. The goal was again to keep them as generic as possible. It turns out that for most classic types of visualizations this works surprisingly well, because they share some inherit similarities. Therefore most visualizations use the default GenericDataPresneter, which handles all of the above in a visualization agnostic fashion. It can be configured to provide an arbitrary number of data dimensions (although most visualizations take exactly two), and for each dimension a number of parameters can be specified. For each visualizations that requires a specific layout of the data which should be visualized (like a 3D bar chart, which requires data in a grid based layout), a custom presenter can be written. However it is advisable to look for common properties between groups of visualizations and target them by more generic presenters as this greatly enhances reusability.

Building Visualizations

Visualizations are created by simply adding the component of the corresponding visualization type to a game object and attaching a data presenter which tells the visualization, what data to show. The visualizations generate the meshes, axes and so on. Visual styles and visualization-specific parameters can also be configured. Each individual type of visualization inherits from a base class which does most of the heavy lifting. As a result, the implementations for, e.g., scatter plots and bar charts are rather short and focus mostly on creating the visual representation of that specific visualization, like points and bars. As a consequence, it is very easy to add additional types of visualizations to the existing framework.
Since there are very diverse preferences when it comes to the look of visualization, I emphasized the ability to configure nearly all aspects, like for example the size or thickness of data items or the shape that is used for the bars of a bar chart. Furthermore, styles can be shared between multiple visualizations and are quite easy to change using a simple drag and drop.

Interaction with Visualizations

While just looking at visualizations might suffice in some cases, it often necessary to explore a visualization further to make sense of it. Since natural interaction is becoming increasingly common on modern devices, we explicitly support touch and pen interaction besides more traditional mouse input. Visualizations can be panned and zoomed using simple touch gestures, data items can be highlighted to, e.g., display additional information or highlight them in other visualizations as well. To make this possible, brushing & linking is supported as well between visualizations that share the same data set. To avoid interaction considerations from creeping into the visualizations themselves, they are fully contained in separate components that are attached to existing visualizations. That also means that its easy to swap between mouse or touch input or even use both in unison. This also means that adding new input modalities (like mid-air interaction) is quite simple, as all they have to do is change the parameters of the visualization they are attached to.

Framework Usability

The most important point regarding usability is of course, besides the programming perspective, that of the users who want to apply the framework for the visualization of data. In that regard, nearly every component has custom editor scripts to enable their configuration directly from within the unity editor, making heavy use of drag and drop to chain the components together. Visualizations can be previewed directly in the editor scene without the need to even run the application. This makes creating new visualizations from scratch very fast and easy. In the future this will probably be even easier with the used of preconfigured prefabs for each type of visualization, that can simply be dragged into a scene and then be configured as desired.

Performance Considerations

Depending on the data that should be shown, a single visualization can become quite large and complex. To guarantee smooth performance even on less powerful hardware that, e.g., the Microsoft HoloLens, I spend some considerations how to best approach rendering each visualizations. Generally, the data items of every visualization, regardless of bar chart or line chart, consists only of a single mesh and therefore only of a single draw call. Of course this also means recreating the whole mesh whenever a single data item changes. However this proved to be less an issue than having a multitude of draw calls. Otherwise, custom shaders are used quite often as well. For example the scatter plot consists of mesh of simple point, which are converted by the shader to quads aligned with the camera that can then be texture mapped. Line charts consist of line list mesh which are converted to quads of configurable thickness as well. All this helps to improve performance considerably and to render visualizations of reasonable size fluently without problem.

Verdict

I’m quite pleased with the result of this endeavor. The proof-of-concept was to use our framework for the prototype it wanted to develop in the first place for, which combines interactive surfaces with head-mounted Augmented Reality. Since the decision was made to develop a generic framework, there was no immediate support for this setup and we had tu actually adapt the prototype so that each visualization consisted of two parts, one on the interactive surfaces and one one the AR device. This turned out to be surprisingly easy and was basically done in a single day (under a looming deadline even). I take this as a testament to the frameworks flexibility, extensibility, and also robustness.
Looking back at the development I think I spend most of my time actually struggling with the unity editor and temperament regarding what it will and will not serialize, because I wanted everything to be nicely configurable from the editor window. Developing the core of the framework went quite smoothly and swiftly after the initial question of how to structure the whole thing was solved. I enjoyed working on this framework quite a lot and have high hopes that it will be used in the future for various planned visualization projects.