Since the early days of the web, people have wanted to visualize data to share with others. Even though the platform provides something basic for these purposes (i.e., tables, images), typically some amount of programming has been required.
Idyll by Matthew Conlen is a tool designed to make data visualization in documents easier.
Jeffrey Heer at the Interactive Data Lab at the University of Washington. Prior to grad school I worked on data visualization tools and interactive stories at FiveThirtyEight, helped the Freeman Lab build open-source tools for computational neuroscience, developed digital tools for journalists at The Huffington Post, and was the senior developer at Rhizome.I'm currently working on my Ph.D. with
Idyll is a markup language for creating and publishing interactive narratives — think posts on websites like Distill, The Upshot, FiveThirtyEight, and The Pudding. With Idyll you write markup in a text file, which is then compiled into an interactive webpage. Idyll takes a lot of inspiration (and borrows a lot of syntax) from Markdown, while trying to extend these ideas beyond static text.
In this example:
example-data.csvfile using the
Idyll ships with a set of standard components that can be invoked in the markup. Because Idyll is built on top of React, any React component can be installed from
npm and used without any additional configuration. It is also straightforward for users to write their own custom components.
Everything in Idyll is reactive, so when anything changes the document automatically updates. For example, if we wanted to have a chart toggle between a scatter and a line plot, we could add a variable that changes when a button is pressed:
[var name:"showScatter" value:true /] [Chart type:`showScatter ? "scatter" : "line"` data:exampleData /] [Button onClick:`showScatter = !showScatter`] Toggle Scatter [/Button]
As is typical with any programming language, Idyll starts with a compiler that does lexing and parsing of the input file. We rely heavily on existing open-source tools to help with this task, namely lex for the lexing and nearley to do the parsing. The compiler then spits out an abstract syntax tree (AST) that represents the hierarchy of elements that belong in the document.
I developed Idyll as a way to automate away an entire class of hardships that authors face when they want to publish documents with interactivity. The project is a synthesis of a lot of ideas and lessons learned from developing these sorts of projects at FiveThirtyEight and elsewhere.
I believe that the web is a great platform for communication and we are only now starting to see some of the potential of moving beyond static text. With Idyll I hope to make it easier for other people to express their ideas in a dynamic and engaging way and publish them online. The project also has implications beyond just blog posts and news articles, such as providing a new entry point into the authorship of interactive textbooks.
I'll continue to focus on Idyll's user friendliness and expanding the online examples that serve as a reference for new projects. I'm also interested in improving the authorship experience for interactive and data heavy websites in general, so you can expect to see continued work in this area specifically.
Regarding new features for Idyll, one big item on the roadmap is enabling custom transformations that operate on the AST. Doing this would allow new possibilities such as writing components that call out to another program at compile-time to generate new static output, for example, calling
graphviz to produce an image of a graph.
We may also add some syntactic sugar to make certain common tasks even easier. In addition to that, we've been working hard to modularize the individual components of Idyll to make it easier for others to work with Idyll in their projects.
Thanks to the persistence of the open-source community, the amount, and quality of web tooling continue to rise. I'm excited to see further development and advancing maturity of these tools that empower creation.
Focus on solving a problem that is interesting to you instead of wading through tutorials. Learn to use Google to solve problems as they come up. Use things like budo to get up and running quickly and don't bother listening to people who argue about tools online. Don't use a framework until you can articulate why you need a framework.
There's lots of interesting work being done in the WebGL / 3D graphics space. Mikola Lysenko is doing great work with regl, and Ricky Reusser has been using it to make some excellent data visualizations. Stardust.js is also an exciting project for using WebGL to visualize data.
The work on decentralization from folks working on projects like Beaker Browser and Dat is also exciting.
The Idyll folks are usually hanging out in a chatroom on gitter. It's easy to join, and we're always happy when people say "Hi" or ask questions about the project. I'd also like to call out Ben Clinkinbeard for all the hard work he has done on the development of Idyll.
Thanks for having me! ✨
Thanks for the interview Matthew! I love the fact how Idyll makes it easier for people from different domains to collaborate.
Learn more about on their site. Check out Idyll GitHub page as well.