Category: InfoViz

www.quadrigram.com

So for the last 8 months I have been working almost exclusively with my friends at the information visualization consulting company Bestiario on new tools to visualize information. Last year, based on our joint experience, we detected two increasing demands within innovative institutions. First the wish to think with liberty with data, outside of coding, scripting, wizard-based or blackbox solutions. Then, we perceived the necessity to diffuse the power of information visualization within organizations to reach the hands of people with knowledge and ideas of what data mean.

Our efforts have now culminated into Quadrigram, a Visual Programming Environment to gather, shape and share living data. By living data we mean data that are constantly changing and accumulating. They can come from social network, sensor feeds, human activity, surveys, or any kind of operation that produce digital information.

For Bestiario and its long track record in ‘haute couture’ interactive visualizations, Quadrigram offers ‘prêt-à-porter’ solutions for organizations, consultants, analysts, designers and programmers working routinely with these types of data. As with other services, data visualization plays a central role in the making sense and sharing of complex data.

I got the chance to work on multiple conceptual, engineering and strategic aspects of Quadrigram. In this post I summarize four most main areas I had the pleasure to shape in collaboration with Bestiario:

1) Redefining work with data

For us at Near Future Laboratory it made sense in helping Bestiario with our experience in prototyping solutions that become feedback loops where our clients can actually figure something out. Indeed, more and more results of our investigations became interfaces or objects with a means of input and control rather than only static reports. The design of Quadrigram lays on this very idea of ‘feedback loop’ and provides a WYSIWYG (What you see is what you get) interface. It is designed for iterative exploration and explanation. Each iterations or “sketches” is an opportunity to find new questions and provide answers with data. Data mutate, take different structure in order to unveil their multiple perspectives. We like to think that Quadrigram offers this unique ability to manipulate data as a living material that can be shaped in real time or as Mike Kuniavsky nicely describes in Smart Things: Ubiquitous Computing User Experience Design: “Information is an agile material that needs a medium”. And this not only concerns ‘data scientists’ but rather everybody with knowledge and ideas in a work that involves data.

With the diffusion of access to data (e.g. the open data movement), our investigation with data has become utterly multi-disciplinary. Nowadays, our projects embark different stakeholders with fast prototyped tools that promote the processing, recompilation, interpretation, and reinterpretation of insights. For instance, our experience shows that the multiple perspectives extracted from the use of exploratory data visualizations is crucial to quickly answer some basic questions and provoke many better ones. Moreover, the ability to quickly sketch an interactive system or dashboard is a way to develop a common language amongst varied and different stakeholders. It allows them to focus on tangible opportunities of product or service that are hidden within their data. I like to call this practice ‘Sketching with Data‘, others such as Matt Biddulph talks about “Prototyping with data” (see also Prototyping location apps with real data). Regardless of the verb used, we suggest a novel approach to work data in which analysis and visualizations are not the unique results, but rather the supporting elements of a co-creation process to extract value from data. In Quadrigram, the tools to sketch and prototype took the form of a Visual Programming Environment.

The teaser video summarize the vision behind Quadrigram

2) Reducing the barriers of data manipulation

Visual Programming Environments have flourished in the domain of information technologies, starting with LabVIEW in the 80s and then spreading to the emerging fields mixing data with creativity such as architecture, motion graphic and music. In these domains, they have demonstrated virtues in reducing the barrier of entry for non-experts (check the VL/HCC community for more on the topic). In the Visual Programming Environment we developed, users manipulate in an interactive way pre-programmed modules represented as graphical elements. When connected, these modules form a ‘data flow’ (also called dataflow programming) that provide a constant visual awareness the result of the program (“What You See Is What You Get”) ideal for quick “trial and error” explorations. This way the tool allows for the evaluation of multiple pathways towards the correct solution or desired result. It inspires solution-finding for non-technical professional by exposing the full flow of data.

The take a tour video presents the Visual Programming Environment that offers a transparent way of setting up a solution, that contrast with wizard-based environments and their “black boxes”.

3) Creating a coherent language

A major challenge when grouping tools to work with data within a common Visual Programming Environments has been to define basic building blocks of a language. Starting from scratch, we used an exploratory phase that led to the release of an experimental environment called Impure and its large sets (500) of diverse modules. This free solution generated a decent community of valorous 5000 users. We used Impure as testbed for our ideas and perform the necessary user studies to come up with a coherent basic language. We particularly focused on specific action verbs (what people can do see Verbs and design and verbs) that enclose the most common operations on data: sort, search, insert, merge, count, compare, replace, remove, filter, create, get, cluster, encode, decode, convert, accumulate, split, resize, set, execute, load, save. These actions are performed on Data Structures (e.g. create List, sort Table, replace String, cluster Network, compare Date, resize Rectangle, load Image, …) within specific domains (e.g. Math, Geography, Statistics, …). The language is complemented with a growing list of Visualizers categorized according to their objectives to reveal aspects about the data (e.g. compare, contextualize, relate, …). Through this structure (actions – structure – domain) user can find the appropriate module within a very dense and diverse toolset.

This exploratory analysis video shows how a unique language provides similar perspectives in the same dataset.

4) Steering the development of an environment that takes advantage of an ecosystem of great tools

Bestiario’s CEO José Aguirre always like to present Quadrigram as a sponge capable of absorbing information from many diverse sources: social networks, data bases, Internet of Things, social media tools, business analytics tools, etc. stressing that “In the wild we know that it is not the strongest who survive but rather those who best cooperate”. We brought that vision to reality with an environment based on severs ‘in the cloud’ that integrates with other sophisticated tools. Like many other platforms, Quadrigram connects to various types of data sources (databases, APIs, files, …) to load data within a workspace. But we also wanted users with detailed needs to take advantage R scripting to perform advanced statistical method or Gephi to layout large networks. The main challenge was to find and implement a protocol to communicate Quadgrigram data structure back and forth with these great tools. In other words, we wanted users to perform analysis in R as part of their data flow. Similar to the architecture of distributed systems and the used of JSON nowadays, the solution was to pass around serialized Quadrigram objects. That offers a pretty unique mechanism to store and share results of data manipulations, what we call “memories”. For instance the content of a Table stored in Quadrigram server is available publically to other tools via a URL (e.g. http://nfl.quadrigram.com/account/m/ext/memo/public/fabien/cosm/cpu store an analysis of my CPU activity)

Why do I blog this: It has been a unique opportunity to help shape a software product and bring it to market. When we created Lift Lab and now Near Future Laboratory we knew if was the kind of experience we wanted to live. This post is an attempt to keep track of the work performed to make Quadrigram a tool that we hope will open new practices around the manipulation and visualization of data. Thanks to the team at Bestiario for their talent and stimulating discussions. I will continue contributing to the project with constant technical, strategic and conceptual guidance. I have also jumped in the advisory board in company of Bernando Hernandez and Jaume Oliu.

Early last year Manuel Lima kindly invited me to contribute to his book at Princeton Architectural Press on the topic of Network Visualization. The book VisualComplexity: Mapping Patterns of Information is not available for pre-order. However my essay did not make through the last editor’s pass. My role was provide an overview of the topic “data city”, its future implications and the role of visualization in this context. I tried to give a high-level reflection on the field evaluating its present and future while keeping the text accessible with tangible examples. It was written in January 2010, it is unedited, but you still might find some relevant elements:

City and information

A city has, by default, always been about information and its diffusion. Historically, fixed settlements permitted the development of newspapers and the possibility for the exchange of information. It will continue to do so in the near future given the volume of data modern cities generate and the emerging selection of algorithms and visualizations available to us to extract information.

The digitization of information

Indeed, we are noticing a digitization of the contemporary cities with technologies embedded into its streets and buildings and carried by people and vehicles. This evolution has appended an informational membrane over the urban fabrics that afford citizens new flexibility in conducting their daily activities. Simultaneously, this membrane reports on previously invisible dynamics of a city; providing new means to the multiple actors of the urban life to reshape the spaces, the policies, the flows, the services and the many different aspects that constitute a city. For instance, the aggregated view of mobile phone traffic reveals the “pulse” of a city, detecting anomalies such as traffic congestions in real-time. Similarly, the deployment of radio frequency identification (RFID) tags connects inanimate objects into an Internet of Things. Ben Cerveny, strategic and conceptual advisor to the design studio Stamen, coined this evolutions as ”things informalize” using the following terms: “the city itself is becoming part of the Internet with a world of data moved piece by piece and collided against a open source toolchain and methodology”.

Tools and platforms to reveal the data city

The data collisions described by Ben Cerveny produce multiple layers of urban information accessible to the actors of the city for their appropriation. Mixed with the emergence of accessible cartography (e.g. Open Sreet Maps), descriptive languages (e.g. KML), data visualization platforms (e.g. GeoCommons), and data processing techniques (e.g. Geocoding), today’s representation of cities do not only depict the cityscape, they reveal conditions in the city that were previously hidden in spreadsheets and databases. As the datasets become more complex and their model of representation richer, graphically representing the city has become less a matter of convention and more a matter of invention. Indeed, traditional cartography with primitive line drawing and static images now co-exist with flexible solutions that separate row data from the map, and promote exploration with multiple-scale interactivity and reactive environments. This evolution was particularly striking with the popularity of “mash-ups”, linking information to space and mapping newly accessible urban data on top of interactive imageries.

Trends

The popularity of “mash-ups” have determined larger initiatives (e.g. “open data” and “web of data”) to free urban data from their silos and promote the public appropriation. Practically, city and government data have been also moving onto the Web making accessible the locations of infrastructures, crime reports or pollution readings. In consequence, “data scientists”, developers and designers create palettes of city data-based visualizations and application, transforming data and their visualization into a public good. In parallel, other platforms such as Pachube have contributed to the bottom-up generation and upload of city data with visualization platforms such as GeoCommons or IBM’s Many Eyes to communicate and share views. This participation offers the opportunity to change cities urban strategies, with potential innovations creating news way to look at the process of citymaking.

Other ways to share the dynamics of the city have emerged in a less obvious but nevertheless indicative unfolding. For instance, of the past years, Idealista a Spanish online real estate ad platform had been accumulating massive amount of information on the cities housing market. It is only recently they have started to offer, almost in real-time, their analysis of the evolution of the real estate market back to the public, with and API for developers to appropriate the results. This strategy offers a city the kind of insights that previously only tedious administrative survey procedures were producing.

Similarly, as the information is not always well-formatted for the analysis and visualization inquires, some initiatives had to develop “web scrapping” techniques to extract valuable data from the public web sites of local institutions and services providers. For instance, for the Oakland Crimespotting, the developers at Stamen Design parsed the web site of the Oakland City Police to produce an effective interactive visualization of crime data showing residents where crime is occurring and what types of crimes are being reported.

The roles of visualization

Exploration and engage in the discussion
The work of Stamen proved that this type of “interventionist mapping” go beyond the expository. Indeed the use of interactive visualization allow exploration and question-making; broadening the urban policy conversation. In fact, aesthetics plays a fundamental role in engaging the discussion. It is not without a reason may visualization of urban informational layers are exhibited in Museums. Indeed, the application of aesthetics to data does not only try to make citizens aware of what is happening around them, but also figuring out the most elegant ways of making the unseen felt and gather feedback. As a researcher at MIT Senseable City lab, I experienced the fundamental utility of “beautiful” visualization as part of investigation process, to attract attention of cities stakeholders, stimulate the dialogue and stretch the imagination. For instance, very early on in the Tracing the Visitor’s Eye project, we produced visualizations to acquire first-hand feedback from journalists and inhabitants of the Province of Florence. They naturally contextualize our work to the local politics, wondering whether the our results could help move the David statue to more appropriate tourist areas or whether they could better understand the impact of the implementation of low-cost airline in a near-by airport. In contrast, they also helped highlighting the “imperfect mirror to reality” we were projecting, rightfully arguing that the models and data supporting the visualization reveal only a partial perspective on visitors dynamics.

Decision making – integration into existing practices
The critics of mash-ups and raw data visualization is the necessary first step to produce knowledge. It leads to investigation, further linking the data to improve the ways professionals and authorities understand and manage cities. Indeed, architects, transportation engineers, urban planners, policy makers, community groups rely on new types of representations as communication instrument as much as means to analyze urban dynamics. In fact, the application of visualizations that combine the emerging time-space data has proven vital; particularly because language through which designer, planners and decision makers communicate plans is mainly visual.

Responsive environments
Outside the realm of professionals, the flexibility of new data processing and visualization techniques facilitate their communication to the public through multiple mediums, from projection on building facades to the transformation of physical space. Indeed, the cityscape offers plenty of interfaces to display the state of city-scale services such as energy consumption (e.g. green smoke) or road traffic. When communicated in real-time, the information creates a responsive environment capturing city dynamics, supporting the decision-making and adapting to the changing needs of the public. MIT Senseable City Lab’s seminal project WikiCity exemplifies the implementation of this feedback loop mechanism. This urban demo proposed a visualization platform for the citizen of Rome to view on large screens the city’s dynamics in real-time (e.g. presence of crowd, location of buses, awareness of events). This platform enabled people, participating to the Notte Bianca event, to become prime actors themselves, appropriating dynamically the city and the event. Besides the importunity of this type of responsive environment to improve the experience of a city, it raises challenges to design the mechanisms by which these services are provisioned and understand for which activity that citizens utilize them for?

Conclusion

The modern city is built not just upon physical infrastructure, but also upon patterns and flows of information that are growing and evolving. We are only at beginning of the development of the tools and visualizations that allow us to see these complex patterns of information over huge spans of time and space, or in any local context in real-time.
Yet, this data city face major challenges. Particularly, the collection of data and their communication involves the collaboration of multiple actors in different languages at the crossroad of urbanism, information architecture, geography and human sciences. Indeed, it is evident that the understanding of a city goes beyond logging machine states and events. Therefore, the data scientist fascination of the massive amount of data cities produces in “real-time”, should not discard the other points of view necessary to understand the city, its environment and its people. In other words, data alone does not explain and their visualizations do not stand alone.

Why do I blog this: Thanks to Manuel for the invitation. Even though the text did not pass the final cut, it was a very healthy and fun exercise to try to write about my work and domains of investigation in accessible terms.

These past weeks I had to chance to work with the alpha version of Impure, a new visual programming environment developed by my good friends at Bestiario. Impure offers a full visual language to retrieve, manipulate, process and visualize information:

Impure allows the acquisition of information from different sources, ranging from user-specific data to popular online feeds, such as from social media, real-time financial information, news or search queries. This data can then be combined in meaningful ways using built-in interactive visualizations for exploration and analysis.

Based on an event-based development structure, the software consists of 5 different modules.
1. Data Structures, which hold data coming from a data source (e.g., Number, String, List, etc.).
2. Operators, which have 1 or more receptors that enable the system to perform a specific operation (e.g., addition or subtraction).
3. Controls, which act as dynamic filters (e.g., interval selectors).
4. Visualizators, which receive data structures from operators or controls and visualize it. They usually return emitters on selected visual objects that can be used as input into another module.
5. APIs that allow real-time communication with various data sources such as Google, Twitter, Facebook, Flickr, Delicious, Ebay, etc.

The prime objective of Impure aims at bridging the gap between ‘non-programmers’ and data visualization by linking information to programmatic operators, controls and visualization methods through an easy visual and modular interface. Yet, I must admit that Impure has a lot to offer to programmers and data specialists; particularly those who necessitate to “sketch with data” as part of their practice. In other words, the type of professionals who process and visualize datasets as part of their investigation process rather than uniquely generating results.

My experience in leading investigations that aim in extracting value from network data, exploratory data visualization is crucial to quickly recognize patterns and understand complex events. But as my projects involve diverse sets of professionals (e.g. ) being able to quickly sketch an interactive dashboard is a guarantee to have a common language that helps the different actors in asking better questions, getting better feedback from them and properly focusing the investigation (some call it visual thinking or to some extend predictive analytics). Ultimately, the use of plateforms such as Impure offer that opportunity to collect insights that give a project the upper hand in decision making. Moreover, the flexibility of a visual programming environment permit to go beyond the limited possibilities for design of GIS and statistical softwares while reducing the fast-prototyping time necessary to program specific interactive visualizations (e.g. see the animations of traffic density and flows in Zaragoza, Spain, based on real-time information) produced in a few hours with Impure).

Practically, in addition to quickly share a first exploratory analysis, environments such as Impure can simplify the practice of ethno-mining particularly to co-create data with participants of the field research (see Numbers Have Qualities Too: Experiences with Ethno-Mining).

Sketching a solution for the Louvre Museum with Impure (see the complete “Sketching with data” Flickr set)

Similar approach to “sketching with data”, Stamen that has for long been leading investigative data visualization projects generally we divide their process into three distinct phases—explore, build and refine. Based on their experience, they outlined some of their common assumptions about data visualization and recommendations for how to do this kind of work; one particularly relevant to the exploration (i.e. sketching) phase:

(19) Start and End With Questions

“Traditional statistical charts can be a good first step to generate questions, especially for getting an idea about the scope of a data set. Good questions to start with include “how many things do we have”, “what do we know about each thing”, “how do the things change over time”, “how many of each category of thing do we have”, “how many things are unique” and “how many things is each thing connected to”. I don’t believe that any visualization can answer all of these questions. The best visualization will answer some questions and prompt many more.”

Apparently, they are engaged in the a similar path as Bestiario, using a Knight News Challenge grant to build a series of tools to map and visualize data that is truly Internet-native and useful. Flexible and “internet-native” environments that make easier to work with information are also emerging in the data storage end of “sketching”, for instance with the Barcelona-based FluidDB.

At Barcelona Design Week, sharing an animation of the traffic flows in Zaragoza sketched with Impure using real-time data feed from BitCarrier.