How might we... visualize data in a clear way?
During the almost 5 years that I was at Upstatement, it was very rare for me to see the same project more than once. We'd sometimes do more work with a client we had an existing relationship with, but it was typically on different projects, and even when it wasn't, scheduling was a balancing act where it was often a question of who would even be available. PBS NewsHour is the only project that I worked on three separate times, and also the only one that I worked on over the course of all four years.
One of the things I really missed about product work since going to Upstatement is seeing how a project can evolve over time. Our position was usually creating a new foundation for a client that they'd spend years building on, and I was typically not a part of that journey. PBS NewsHour was something that really scratched that itch for me. Each year I came back to it to add more functionality, clean up some older code, and improve the UX.
The Problem
Without getting into the actual politics of it all, I think it is fair to say that the US Presidential Election in 2016 is one that drew a lot of attention and inspired people to look into politics who maybe weren't always super interested in it.
PBS NewsHour was always looking for new ways to visualize the large amount of data that the Associated Press collected, something that becomes more and more relevant every two years (especially because you can compare data over years). And it's certainly fair to say that the landscape has changed a lot in the past decade. Social media has become enormous, and more people than ever use their phones for consuming news & information.
2016 was the first election year that the very oldest members of GenZ were able to vote, with a lot more to follow in 2020, and for even more of them, 2024 will be their first election. All of this is to say, that there was a large opportunity to meet people where they were.
Our goals were not simple , but were straightforward enough. We wanted to be able to display data, in a way that was clear and easily understandable, and could easily be adapted to different platforms. But we also didn't want to just make graphics, because that's time consuming and not scalable - what we were really making was a system that could generate graphics, with some specific needs.
- Robust: The AP has a lot of data and there was a desire to be able to use it in different ways. Different graphs for different data sources, filtering, etc.
- Adaptive: It needed to be able to output these graphics in a way that worked for the web, for social media, and even for broadcast.
- Automated: When votes are coming in, updates happen quickly. It would be ideal for the website and broadcast to be constantly up to date, and for social to be able to generate images very quickly.
The graphics needed to be able to cover a wide variety of situations as well. We started with the 2018 midterms, but we wanted to be able to have it work for the 2020 primary, and then the 2020 general election. Those 3 different cases have a lot of differences to them in terms of content: senate balance of power, county-by-county results in a state, exit polling information, and so on.
The Upstatement Case Study talks about the design and Scott Batson wrote a stellar blog post about the web component aspect, but what neither really touches on is what I want to talk about: the actual dashboard, the control room for generating these visualizations.
What we Made
There was an open question for a while about whether we wanted this to support "snapshots". For instance, if an article is posted using one of these web components, should that data be locked to what it was at that time? By default, the web components continue to update as data changes. And how does that work when more fields are added or removed? If I want to go to the PBS NewsHour site and look back at results from 2016, should those be available? Tough questions to answer, because they were often outside of solving the immediate need, but PBS NewsHour's tech team put together an API that grabbed data from the Associated Press and worked to standardize it so that if there was a desire to do that kind of thing, we could. And because we started in 2018, there really wasn't much legacy data anyway - but it grows more and more every two years.
Because we started with the Midterms, there were really only a couple of graphs, but it would grow larger and larger as we moved to cover more and more conditions. The way that each one worked is that it had a series of options, and these changed based on the graph as well. Eg: If you're looking at a multi-race vs a single race, the multi-race has more options for the states that you want to include.
In the above image, you can see an example of how we built the controls for a single graph - there would be options for things like how many candidates to include, or you could choose to show the delegate count or vote count. And there were more meta settings as well - like the theme, how often the data should automatically refresh, which Race ID the data comes from, and so on.
On the right, you can see a preview that updates in real time as the settings change, using the same web component tech that now lives on the website. And that right side gives you three options for the graphic - you can download it as an image, grab the embed code to put it on a page on the site, or even open a Broadcast mode (a full-screen setting) that could be broadcast directly to the news televisions.
It also isn't the case that these were just the same image scaled to different sizes - often, the layout and amount of data is different to take better advantage of the medium.
Broadcast obviously has the most guaranteed space, but it also needs to be simple, partially because it may not be disaplayed for long, but we also don't want to give them too much to read because people will also be talking over this graphic. But we also want it to convey enough useful information to work when muted, so it more heavily calls out a winner.
The social media ones on the other hand have more limited space - and we also want those to be branded in a way that isn't necessary for the site or broadcast (since those already have branding displayed elsewhere). Mobile is simpler, eye-catching, and framed in a way to take the most advantage of the dimensions / spacing of Twitter and Instagram.
The web view specifically has the most ability to rely on the rest of the page to provide added context. If you're looking at a page of the Primary results in Pennsylvania, well, that's less information that needs to live within the component. Instead, Web offers the most interaction and ability to deep dive into more specifics, in this case viewing the voting results within each individual county.
How we made it
So, how does it all work? The Admin dashboard itself is built as a single-page app using VueJS. The controls are treated as reusable components and we use VueX so that some settings can persist between graphics. Some of the more complex visualizations (eg: the maps, exit polls), are drawn with d3, with appropriate measurement changes based on the display output.
The image downloading was achieved by simply converting the canvas element into a png, and using html-to-image where we weren't already using canvas.
It all sounds simple but that doesn't mean it started that way - one of the big obstacles that I tackled during my last round of involvement in 2022 is that we had built the midterms and general election graphs in different branches. It made some sense at the time, because you weren't likely to be using the same graphics between them - since some involved the house/senate and others involved the presidency.
But I wanted to combine them for a few reasons: I wanted the better maintainability of only having one codebase (where improvements made to one would also impact the other), I wanted the simplicity of not having to redeploy every two years, and as I mentioned earlier, I wanted the staff to still have access to those older results.
I love code refactoring, so it was a ton of fun for me to take a few days and really dig out the guts and redo everything - as part of that work, I opened up how much power different graphics had, adding mobile or broadcast support to those that didn't have it, and bringing certain graphics to races that didn't have them before.
Final Thoughts
As with every project, there are things that I would still have liked to improve on. There are a few that come to mind specifically with this.
- Presets: I'd have loved to bake in some smart presets for the graphics that were frequently used, to make it quicker to generate consistently, using standard options.
- Mapping: Utilizing the RaceIDs currently adds an extra step. There's a search right now but it's limited in results returned which means you have to go through and find the RaceID you want. I'd have loved to tie that into something like Algolia so you could just search Pennsylvania and see all of the relevant options.
- Error-Handling: Related, it's also easy to "break" a couple of these. Take the Exit Polls for instance - each one has a Question ID and a Data Source, but not every Data Source has the same questions, and not every question has the same number of result options, so you can get into some null states, and there's no real handling for that. Because of the live preview, it's obviously easy to notice and rectify, but I'd prefer it to be a situation that never occurs.
It's easy for me to look back and point out flaws or areas where I feel like more could have been done, but this was such a privilege to work on and I'm so proud of what we made. I certainly never imagined my work showing up on broadcasts viewed by millions of people . And who knows, maybe I'll still get a chance to improve some of those things ahead of the 2024 election.