Google Docs + Miso-Powered Apps: a note on collaborative workflow
Working both collaboratively and efficiently on deadline, needless to say, is important. For our fastest-paced interactives and news apps, we’ve come to use a combination of Google Spreadsheets and Dataset.js from the Miso Project in a way that we’ve come to really quite like. We figured we’d share it with you here.
Essentially, we use a Google Doc as a temporary database for our text or numeric data. When we go live, we download that data to a local CSV, change the path, and deploy.
We’ve used this workflow on the Obama Haters Book Club, the sidebar to our election coverage page (which doesn’t exist on our site anymore but which looked like this), our Meet Romney’s Bundlers interactive, a variant of it for HavingTroubleVoting.com (where we had a Google Form power a Google Spreadsheet) and the Olympic Medal Tally among others.
Note: This post is a bit more technical than usual.
The first step is getting everyone to put the information they gather into a structured spreadsheet. Here’s what the Obama bundlers spreadsheet looked like.
From there, we “Publish the spreadsheet” from the File menu — the crucial step when running an app off of a Google Spreadsheet — and import the data into our app. The Dataset.js documentation has a very simple tutorial on that step: http://misoproject.com/dataset/tutorials/googlespreadsheets
With this setup, reporters can be writing text and the photography department can be adding links to images at the same time the interactive is being built. What’s particularly great is an editor can edit the text in the spreadsheet and it will be piped into the interactive without his or her having to edit any HTML. The order of the rows can also be swapped if some elements need to appear before others on the page.
Mother Jones’s Tasneem Raja has written about this workflow as well, using Tabletop.js. Her post is definitely worth a read.
So Why Dataset over Tabletop?
I’ve come to like Dataset.js over Tabletop.js for two main reasons. The first is that powering a live news app from a Google Spreadsheet can cause a lot of problems—so we download our data from the cloud onto our servers before going live. Dataset.js makes this transition really simple.
Jeremy Singer-Vine at the Wall Street Journal has written about the dangers of running live apps from Google Docs and he has a strong argument: 1) Google can rate-limit you, killing your live app; 2) If Google changes its API in the future your app may go offline; 3) Google can add extra data to your API request that can slow down your readers; 4) Sometimes you want to make a bunch of changes all at once, like add a complete row of information — with Google, incremental changes cell-by-cell are immediately reflected in the app, which will create broken elements until you’ve finished a full row; 5) Security and data privacy — sometimes you have columns in your data that you want to use internally but shouldn’t be public facing. We ran into this with HavingTroubleVoting.com where we collected ready contact information to later verify reports but we didn’t want to make that info public to the world.
By making a local CSV whose columns we’re picking for public view, we solve a lot of these problems. We get the benefit of working collaboratively in development and can switch to something more archivable and robust for production. I have some other scripts that could be improved that will download a spreadsheet automatically and turn it into a clean CSV. If I can clean them up a bit we can post them as well.
The second reason is Dataset.js gives your data a lot of functionality, such as the ability to group, query, and turn your columns into arrays (which is particularly useful for hooking it up to libraries like HighCharts). In fact, being able to hook up a Google Spreadsheet to your app is just an extra feature of Dataset.js — as the name suggests, it’s built to work with data and it can do some fancy things regardless of whether you’re working with a local CSV or a spreadsheet in the cloud.
If you have any suggestions, I’m at @mhkeller.