NewsBeast Labs

This week, we’re super excited that our project This Is Your Rep On Guns was nominated for a Data Journalism Award in the Data-Driven Application category.

As we wrote about a couple of times on this blog, we have been tracking all 530+ representatives’ positions on gun control, updating it when news happens, and publishing their statements automatically via our Twitter robot @YourRepsOnGuns. 

It’s been a very fun project and one we hope that has been informative. We’ve received a number of emails from readers tipping us off to local articles and they’ve sent in letters from their rep explaining their position on gun control. 

If you want to help us be able to do more projects like this, there’s a People’s Choice component of the awards where you can vote for us.

If you like the project and want to give us your vote, you can find us here. We’re only 70 votes away from first place! (Sometimes the site asks you to also log in to Facebook, but nothing gets posted from your account, we’ve checked.)

Thanks for the help these past few months!

-Michael

It was the Monday morning news meeting and all we could talk about was Dennis Rodman, Kim Jong-un, and Vice Media. It was the strangest story of the week, and utterly riveting. But there was more to it than what had already been reported.

The Vice show is funded by HBO, which is owned by Time Warner, which has a boatload of shareholders. Vice Media itself has a whole range of wealthy investors, including former Viacom CEO Tom Freston and The Raine Group, a who’s who of one-percenters. Meaning: a lot of people stand to benefit from the hospitality/propaganda machine of the one of the world’s most notorious dictators.

Reporter Caitlin Dickson started looking at the various connections, using LittleSis.org, an online database that tracks the social connections among the powerful—politicians, business leaders, lobbyists, hedge funders, etc. LittleSis is a project of the nonprofit Public Accountability Initiative and a great, easily-searchable, user-friendly source for reporters. You can search people and companies, find out how they’re connected to each other and to whom they donate money.

Check out the screenshot for an example, and the story for the final product. Caitlin was on deadline pressure so was only able to scrape the surface on the Vice story, but there’s no doubt much more to mine.

-Paula Szuchman, deputy managing editor

Last month we published a package of stories marking the fortieth anniversary of the Roe v. Wade decision. It had a few moving parts but I’ll just go over some of them briefly here.

How it started

This summer you probably heard the story about the last abortion clinic in Mississippi that was threatened to close due to stricter state laws. Allison Yarrow, who sat across from me at the time, was covering the story and it got us thinking: the line “The Last Abortion Clinic in Mississippi” is attention grabbing, but it doesn’t tell the whole story. That is to say, what you really want to know is how far are people away from their nearest clinic, regardless of state boundaries. One state may have five clinics but if they’re all in the southwest corner of the state and you live in the northeast corner, and your adjoining states have multiple clinics but only at their borders farthest from you, then you’ll have a hard time getting to a clinic, even if you had many in your state. To see where this might be the case and where access to services was compounded by new restrictive provisions (over 150 nationally in the past two years) we made as close to a comprehensive database as possible of every abortion clinic. Our goal was to see what parts of the country were farthest from a clinic. From start to finish, this process took about six months. 

We got our address data from a variety of publicly available sources: Planned Parenthood, the National Abortion Federation, anti-abortion websites that keep their own lists and others. We needed to verify that the address information was correct, though, so we called over 750 clinics to confirm. We also asked them up to how many weeks they offer services. The resulting database is the only one of its kind that we know of. The Guttmacher Institute undertook an abortion provider census in 2008 but they didn’t separate clinics from hospitals from private doctors offices, which represent different levels of care that we thought was an important distinction.

What it became

We started this in July and the project evolved. We thought the election might bring the issue of abortion access to the fore but it didn’t and that gave us more time. Allison brought up the fortieth anniversary of Roe v. Wade and that let us think much bigger about the project. Because this was such a personal subject matter, we knew readers’ comments would feature prominently (from both sides of the issue) and we wanted a strong narrative component, too.

To give a human voice to the Geography of Abortion Access map, Allison flew to Wichita, Kansas, one of the areas that stood out both on our map, as a metro area far from a clinic, as well as in recent memory as the site of the 2008 murder of late-term abortion provider George Tiller. To add a broader perspective, Sam Register who runs the Newsweek Archivist tumblr went through the Newsweek archives so people could follow the topic’s coverage from the 70s through the 00s.

What we learned from reader’s stories

Over the course of the week, we shifted the question we were asking from why do you support or oppose legal abortion to a conversation about pro-life and pro-choice labels as a way to get more nuanced opinions and show the complexity of the issue. We asked readers to complete either the phrase “I’m pro-life but…” or “I’m pro-choice but…” We got more responses from our other reader-based projects but we were happy in how thoughtful and honest people were. Read our roundup of interesting responses to those questions as well as our free form “Tell us your story” prompt here.

Under the hood on the map

How to represent this dataset was tricky. We had three main issues: anonymity, unbiased geography, and context. 

Anonymity: Although we got our data from publicly available websites that anyone could find and was often information that anti-abortion groups already held, we weren’t comfortable publishing addresses, names, or exact latitudes and longitudes. We took great care to do things like scrub our final database of anything identifiable and we partially randomized each clinic’s location so they weren’t pinpoint-able from our map. On the presentation level, we added the magenta circle big enough to span multiple hexagons (our base geography layer) to let people know that an address was approximate. Even if you backtrack and find our database, you won’t get any information that would let you de-anonymize the data.

Unbiased geography: As I wrote above, we wanted to get away from the arbitrary state and county borders that most all of the research we encountered was based on. We did some initial plots using Census tracts but that presents exactly the same problem [photo]. We ended up making a hexagonal grid using the Repeating Shapes plugin for ArcMap, which lets you make a grid out of your choice of shape and size. The trick to making a hexagonal grid for the web so that the hexagons will be regular (all sides equal) no matter what degree of latitude they fall on is to make the grid in your output projection, Web Mercator EPSG: 3857. You can reproject it to do your analysis in whatever you like, but because it will eventually be displayed in Web Mercator, it will need to be created in that so as not to come out distorted in the browser. If you want a 20,000 meter in diameter hexagonal grid, here’s the one we used:  Shapefile, KML, GeoJSON.

And here’s another one that Brian Abelson, current Knight-Mozilla Fellow at the New York Times, made while he was helping out on the project. They are also 20,000 meter hex grids. This one has the state borders preserved in case you want to assign state values to each hexagon: Shapefile, KML, GeoJSON.

Context: Generating our distance map wasn’t enough to tell a story with. We added three other pieces of information that would walk people through the significance of the patterns they were seeing. The first was a map of female population aged 15-44 so that people could see the areas where women lived that were farthest away from clinics and identify significant metro areas (the pink dot density overlay). The second was the different legal restrictions that each area was subject to (areas with highlighted transparency). Again, this was an interesting way to visualize this data because we didn’t highlight every hexagon in Kansas, for example, to show that certain laws were applicable in Kansas. Instead, we highlighted hexagons whose closest clinic was in Kansas. This gave us a very realistic map so that people could see what state laws they would be subject to if their nearest clinic was across state lines. It also visually demonstrates how state laws can affect people that don’t live in that state. And third, we selected our own highlights from going through the data, such as the areas where telemedicine is banned in conjunction with mandatory in-person counseling. The combination of these laws in Arizona, for instance, means some women travel over a hundred miles and spend two days to get a prescription for the abortion bill. 

More under the hood

The map itself we built using CartoDB, which allowed us to very flexibly add the different highlighted views of the map without rebaking our tiles each time.The slider that shows clinics that only offer services up to X weeks we did by loading four tile layers on top of each other at once and show/hiding them depending on the slider value. This made the map slightly slower on initial load but it made the transitions between map states super fast — so a trade-off. 

For the highlighted states, those restyle and reload all four map layers as well. We used Leaflet.js’s ability to plot vectors to draw the line between the hexagon you’re hovering over and the closest clinic to provide some more descriptive interaction.

The heatmap was created through ArcGIS from census tract data. We filtered for just the number of women of reproductive age, 15 to 44, per tract and then used the Create Random Points function in ArcGIS to create one point for every 210 women. We came up with the 210:1 ratio by looking at a histogram of the data to see what would be an accurate dividing point. For a shameless plug, I used an online tool that I made called www.Histagram.me to generate quick, interactive histograms. Feel free to use it too.

Because the heatmap itself is done with CartoCSS layering techniques and not a statistically calculated heatmap, we made sure to compare side-by-side with a choropleth tracts map of the same data using Jenks-clustered color breaks to make sure that our heatmap told the same story as the choropleth. 

A few months ago we spoke with Andrew Hill, Senior Scientist at Vizzuality (who makes CartoDB) on some experimental ways to map the data. The line on hover came out of some of his renderings and you can see in the photos below some of the experimental line styles.

All in all it was a lot of team work, Allison, Abby, Brian, Caitlin, Lizzie, Sam and a number of other people all helped with parts of it over the course of six months. If you have any other questions about it, let me know at michael.keller@newsweekdailybeast.com

-Michael

Before we settled on the Value-by-alpha approach for showing the different state laws, some failures:

We tried outlining the different shapes and showing them in different colors:

We tried coloring the hexagon outline by the different laws that were in effect. Creating a sensical hierarchy proved difficult:

Lines instead of hexagons:

Highlighting Peurto Rico:

A value-by-alpha chart where census tracts are shaded by their percentage of women of reproductive age. Unfortunately, it’s not that intelligible and the heat map overlay is a much cleaner way of showing this relationship:

Before we made the hexagon grid, how the map looks if you use census tracts:

After the Newtown shooting in December, we had a meeting over the phone to discuss our coverage. We decided to have a two speed approach: a quick reader-driven story about why they do or don’t own guns (which we’ve written about a bit on this blog), and a deeper-dive look at the anticipated legislative issue that this and other recent shootings seemed to be bringing about, which we launched Monday as www.ThisIsYourRepOnGuns.com. The project idea grew out of the simple problem that not many people can name their representatives off the top of their head, let alone know their exact stance on gun control or how to get in touch to make their voice heard.

Eliza Shapiro, Abby Haglage and Caitlin Dickson did some awesome reporting for all 530+ representatives, digging through their voting records and previous public statements to distill their position to one of four categories: Opposes reform, Supports reform, Swing vote, or Unclear. We kept track of the sources, too, so that we could present representatives’ statements to the reader when the final thing was done. 

Brian Abelson was also around to rig together @RepsGunTweets (since renamed @YourRepsOnGuns), which served as both a tool to monitor reps’ statements to see what category they fell into, as well as an open feed for anyone interested in the topic to follow on Twitter. Read about how that was built in this blog post.

The interactive currently stacks up the number of reps in each category and lets you do a combination filter by different criteria such as chamber, party and state. You can see things like how likely legislation is to pass each chamber and where different states stand. Importantly, too, you can put in your address read information on your House representative and two Senators. Using information compiled by the Sunlight Foundation, it gives you their phone, fax (for those that prefer the fax), address, twitter, website and Facebook page so you can get in touch with them. We also pulled in each representatives NRA grade and their rating from the Brady Campaign to Prevent Gun Violence to give more context to their legislative history.

My favorite part of it though, is that we’ll be updating it as the gun debate goes on. We’ve already received emails from readers who have contacted their reps with statements that we’ll add and one person sent us a local news story from their congressperson that will move him from the Oppose reform to a Swing vote. We’ll mark these updates on the landing page so people can follow along and readers can leave their email to be notified of updates.

We also did this as its own URL similar to how we did www.HavingTroubleVoting.com. As a resource and tool that was going to hopefully have a long life, we felt an easy to remember and dedicated page showed our readers that this was something they could keep coming back to.

Under the hood

The hardest part of this was getting all of the data from multiple different sources into one nice database. We had a few different people researching, different numbers coming in from different places, and multiple editors editing. We used Google Spreadsheets and good spreadsheet etiquette to make sure people were marking the categories the same way and joined them in R. 

To make the stance information simple to update, the map copies that information from the main table on load instead of storing it separately with the map data.

The main page uses Isotope.js, which we’ve used a bunch before. But this was a little tricky because we needed to sort them into four columns. Fortunately, there’s some crazy extension for Isotope that lets you do just that. The harder part was figuring out how to get it to display top to bottom instead of bottom to top. But buried in the “Tests” documentation was a page on how to make your elements stack right-to-left for languages like Hebrew and Arabic. It includes the settings to rotate the positioning, which worked.

The only fancy mapping feature is if you click on a district, the map automatically pans and zooms to fit the founds of that district. This is done using the ST_Envelope() function in PostGIS through CartoDB. ST_Envelope() returns the bounding box of a given feature which you can sent to Leaflet.js’s fitBounds() method to pan and zoom to that box. The only problem to be aware of is ST_Envelope() will give you an array of x and y values but fitBounds() is expecting the format to be in y then x (lat, then long). As long as you reorder the elements in your coordinate array, Leaflet will be happy.

Getting the aesthetics of the map right was a little tricky. I wanted to make sure that a highlighted feature’s outline appears above the other features but below its own fill so you get a bright white border and then a subtler inner border. If you follow the symbol drawing order and compositing option rules in CartoCSS it becomes manageable.

From the failures folder

Here’s what the original mock-up looked like, which we weren’t too far off from. I reworked the top nav hierarchy into two main buttons, added more color and turned the rep detail elements into three columns instead of rows so it was more compact and graphic.

-Michael

The other month, we asked readers why they did or did not own guns. Here’s the post on the creation of that. We ended up getting over 1,500 responses, and our readers were largely thoughtful in what they wrote. Before we launched this project, we didn’t know how our audience would break down into gun owners and non-gun owners, and whether we would get the usual talking points from each side or hear new points of view. We fortunately did see a lot of interesting stories and patterns. But uncovering those trends in over a thousand responses in only a few days was a challenge. Reading through every response on deadline was unfeasible, so we had a machine do it. 

The idea of using natural language processing to make sense of a large number of reader comments came from Blair Hickman, Social Media Producer at ProPublica and we ended up using the Overview Project a natural language processing tool developed by the Associated Press (screenshotted above). You can read more about the technicals of how it works here but generally, it looks for clusters of related words and groups them together in that tree layout you see. 

Some interesting trends (Read the analysis with selected comments) that surfaced were a group of comments about how growing up with guns influenced people to both own and not own guns later in life. Many people framed their non-ownership of guns around their lack of a need and the algorithm found this cluster around phrases with “need” in them. Viewed in this light, many gun owner responses can be seen as implicitly responding to this prompt, explaining their clear, pragmatic need for gun ownership such as hunting, protection in rural areas or many others we included. Overview also brought out repeated comments that had variations on the phrase “When seconds count, police are minutes away,” a similar need for self-protection or a desire for self-reliance, depending on how you interpret it.

A number of readers discussed how their association with the military sometimes led them to be comfortable with firearms or, in many cases, the opposite. As one person put it: “As an army officer I came to realize many guns are tools created with a main purpose of killing. I don’t hunt and I don’t target shoot. No need for a gun.”

The clustering algorithm also grouped comments related to how experiences with family members committing suicide led people to not own guns, as well as religion. The posts concerning religion were especially interesting since in 2008 campaign speech, President Obama said that some people “cling to guns or religion” as a way to “explain their frustrations” with society. All of the comments submitted to us that discussed religion were from staunch non-gun owners, however. 

I think the post is worth a read if you’re interested in seeing an attempt at surfacing a conversation. On the design side, you can see we tried to get out of the way of the readers’ voices as much as possible: we wrote a short sentence intro framing each collection of comments and then let them tell their stories. Our front-end dev wiz Lynn Maharas actually pushed through that blue background quote style — which didn’t exist on our site before — especially for this so that we could string a bunch of quotes together and still be readable.

Using Overview was pretty easy, especially since it has a web interface now that you can upload your documents to. You have to remove any commas, first, however. We augmented some clusters with simple keyword searches of thematically related words that the algorithm might not be able to connect the dots between. For instance, if military was one cluster, we might want to look at Navy and Army as related words. This might not actually give any extra insight since the algorithm is looking for clusters anyway so it might pick up on those. But it could help guide a more analog approach once the machine has broken things down into potentially interesting categories. 

We clustered the spreadsheets of responses independently of one another, but one analysis would be to see if the machine would divide them automatically into two camps. Always more analysis to be done than deadlines allow. Here’s the anonymized data though, let us know at NewsBeastLabs@gmail.com if you find anything interesting.

-Michael

UPDATE: FEB 10 @RepsGunTweets has been changed to @YourRepsOnGuns. Check out www.ThisIsYourRepOnGuns.com for the ongoing project.

Brian Abelson is a data scientist who is graciously donating his time at NewsBeast Labs before he starts a full-time position as a Knight-Mozilla Open News Fellow at the New York Times in February.

For an upcoming project on the gun debate, we’ve been monitoring statements representatives have made on the topic. As President Obama prepared to unveil his proposal for gun control on Wednesday, Michael and I were curious to see the reactions of representatives to the highly publicized announcement and be able to report that in real-time. Given the degree to which breaking news is now reported (and responded to) on social media, we thought it would be useful to build a bot to log officials’ comments on certain issues and present them in real time. Such a tool could be used by news rooms to engage their readers on a continuous basis by aggregating and serving content from members of particular communities or who serve on different committees.

@RepsGunTweets was born.

We were inspired by the work of 2013 Mozilla-Knight OpenNews fellows who recently built a prototpe for an app called “if (this) then news,” a news-oriented take on IFTTT – a site for linking triggers from gmail, twitter, dropbox, and other services to actions on the web. Applying this logic to news coverage, the fellows created the shell for a tool that would monitor live data streams, detect important events, and issue notifications. As Vice President Biden took the mic, we started furiously coding up a bot that would follow the twitter accounts of US Representatives and retweet any comment that included “gun”, “assault weapon”, “firearm”, or other relvant keywords. After a couple hours of missteps and headaches, we eventually got @RepsGunTweets up and running. In the last ten days, the bot has logged 307 tweets; two-thirds of which came in the first three days. We’re still analyzing the conversation but one interesting observation is representatives who are not in favor of gun control tend to link to longer explanations of their position on their website instead of tweet a comment.

Under the hood

At its core a retweet bot is a pretty simple tool: Follow a feed, find what matters, and serve it back up under a single account. The harder part is figuring out how to accurately communicate with Twitter’s API. Using tweepy for python we were able to easily access twitter’s numerous methods. All we needed to provide it with were the the consumer key, consumer secret, access token, and access token secret for an application generated on http://dev.twitter.com/apps. The bot follows CSPAN’s member of congress list and applies a regular expression for the desired keywords and retweets any matches.For even more technical info, check out this Github page

Responsive Design - A Mobile Retrofit

A few weeks ago we launched our new mobile optimized design of thedailybeast.com, and we couldn’t be more excited! Go check it out if you haven’t already (if you’re not using a mobile device scroll to the bottom and click on “Mobile Site”). 

Our CTO Matt Stowe’s favorite thing about it is everything loads from the same exact URL as before — there’s no m.thedailybeast.com. In his words: “Why is this great? Because no matter how you share, bookmark, email, or use the link, the site serves you an optimized experience for your device.”

The challenge was to take our existing design and reorganize the content into a space less than 1/3 of the desktop size. In addition, we needed to account for a wide range of phone sizes, layouts (portrait/landscape) and resolutions (up to 300ppi). Based on the new design and requirements we decided to recreate all the markup and css for the entire site….10 weeks and 1,342 cups of coffee later we had ourselves a mobile site! Here are two features that we’re most excited about:

Responsive Design

Ethan Marcotte, who coined the term “Responsive Web Design”, describes 3 foundations as core to a responsive site:

1. Flexible Grid
2. Fluid Images
3. Media Queries

Instead of creating multiple versions of the site to fit multiple device sizes and resolutions we decided to use a responsive design approach by implementing a flexible grid system. This way our site will expand and contract to fit any width browser. For example, when your phone is turned from portrait to landscape, the site will respond by expanding to fit the screen. To produce this, we are using a 12-column flexible grid designed by Stephen Bau that is based on the great work by 960 Grid System (we love grids!). All columns widths are set as a percentage of the page rather than a fixed pixel width which allows the site to expand or contract to any size.

The easiest part of the design was making all the images “fluid” by setting their width to 100%. Now images will expand/contract to fill the size of their parent element. As for media queries, new in css3, we are currently modifying the navigation so that each element is spaced properly based on site width. In the future, we hope to implement tablet and desktop media queries for a seamless site experience across all devices.

Icon Fonts

With the new responsive site we decided to abandon the use of sprites and most icon images in our css and have opted for creating our own icon font instead. Now when you visit the mobile site, all the icons are actually a part of our custom font, not jpgs or pngs. The browser references all icon images from a small font file (~30kb). One of the benefits is that your browser will not be downloading lots of icon images, taking up device memory, bandwidth and adding to the time it takes each page to download.

An additional benefit of using a font is also so that we don’t have to recut new images if designs change and we need the image slightly bigger, smaller or a different color. Now we can change each instance of the icon with css adjusting it accordingly. Lastly, icon images will never look blurry since they are now vector data (like an .svg) instead of raster (.jpg). Here’s an example of why icon fonts are awesome. You can make your own custom fonts over at icomoon.io or download the icomoon chrome extension and work on your fonts offline.

Shout out to the entire development and design teams who created lots of really great features!

— Lynn Maharas, Front End Developer

Our last Wednesday Workshop focused, in part, on ways to get readers more involved in our stories. When news of last week’s awful shooting reached us, we wanted to open up discussion on the role of guns in America. On our Tumblr we asked readers how the shooting should be covered and many requested we steer clear of the politics and instead opt for a genuine discussion on gun control.

Gun control is a complicated issue in this country and nuanced issues can be at odds with the tools of data visualization. That is to say, data visualization and data reporting are often marked by being extremely comprehensive and boiling that comprehensiveness into one easily understandable image, graph, or layout. Doing anything comprehensive on an issue as complex as guns in our society, on deadline no less, would be tricky, and we’re not ones to put data out there that’s misleading or inconclusive.

But the other tool of digital journalism is being able to present a great deal of information in one place, which does work for a nuanced subject. We wanted to engage our readers to tell the story of guns in America in a way that showed the issue’s complexity. We posed the question as “Why do you own a gun?” or “Why don’t you own a gun?” On our site we, we set up two forms that let readers easily complete the sentence “I own a gun because…” or “I don’t own a gun because…” and displayed their responses for readers to sift through.

It’s like the digital equivalent of Man on the Street reporting, where you go and ask people on the street their opinions on an issue in the news and write up their quotes in an article. Let’s call this a Man on the Internet story, or to be gender neutral, Person on the Internet (Internet Vox Pop maybe? I’m open to suggestions).

We published the article Monday evening and less than 24 hours later we have over 900 responses — over 500 from gun owners and over 400 from non-gun owners. We have some thoughts on how the two sides explain their position but, for now, we’ll let you read through and absorb it on your own.

We’re collecting and categorizing the responses, so look for that article on the Beast later in the week.

Under the hood

We used a customized Google Form to handle the response collections. This is a nice tutorial on how to embed Google Forms into your site with custom styles and functionality. 

We’ve used custom Google Forms before, on our other shooting project actually, for a newsapp that lets readers put in their address and it finds news accounts of multiple-victim shootings near them. A Google Form then asks what they remember about the incident and collects their responses in a spreadsheet. We published some of the most moving responses that I think is worth a read.

For this project, though, I was running into trouble putting in two custom forms on one page. Since I only had a day to build this, I ended up sequestering the two forms to separate HTMl pages and iframe-ing them into my main page. This was nice because it ensured the two forms didn’t interfere with each other and since the pages were all on the same domain, I didn’t have any cross-origin issues and didn’t loose any functionaity — when you submit one form, that action bubbles up to it the parent frame and grays out the other form.

I originally wanted to do something more animated similar to this seminal piece of crowdsourced dataviz from 2008. I like how its animation gives the project energy but it comes at the cost of not being able to scroll through the responses on your own. After some thinking, I couldn’t figure out a way to have both a sit-back-and-let-the-responses-flow experience and a I-want-to-dive-into-these-responses-and-scroll-through-them-all experience. The latter is obviously the more useful for the reader, so I went with that. The election interactive is also a bit different from this since most of the emotions on each line are of the same category, so it’s not really hiding anything by not letting you scroll. For our project, each response brings its own nuance to the debate so you don’t want to hide any of them. If you have any thoughts on how to improve the presentation, I’m at @mhkeller.

Brian had the great idea that we let this conversation be medium agnostic. So in the story dek we let people know they can continue the conversation on Twitter with the hashtags #IOwnAGunBecause or #IDontOwnAGunBecause. I built some hooks into our Underscore.js templates that let us add selected tweets to our spreadsheet and display them with a Twitter icon and a link to the original tweet. That way we could pull in interesting responses from elsewhere and flag them as such. You can look at the formatHelpers object in app.js to see how it checks for content in the Twitter column and adds the image and link if it finds something.

As I’ve written, I’m a big fan of Miso’s Dataset.js, and that’s what we’re using here to pull the responses in from our Google Form Spreadsheet. Contrary to my previous post, this app does work off of a live Google Doc. I know, blasphemous. For a few workflow reasons we weren’t able to have a script download our spreadsheets and put them on a server like we did for HavingTroubleVoting.com where we had both rate-limiting and privacy issues.

That being said, we have been very closely monitoring the app to make sure it doesn’t get rate-limited and it’s been fine so far. I have a few lines commented out in the code that point to where we’d put a local CSV of the responses, so if the app went down, it would be back up in a minute or so. We also made sure not to ask for any identifying information so we had no privacy concerns. Now that we have close to a thousand responses, though, we might switch to local files so that the page loads faster. If we could have set it up to download automatically, however, that would have been our first choice.

One thing I added yesterday evening after we started getting a lot of comments was a way to filter by state. A lot of content can be overwhelming, so the more options you can give readers to drill down to a subset that might be more relevant to them, the more manageable the experience is and hopefully more engaging and memorable.

-michael keller