The other month, we asked readers why they did or did not own guns. Here’s the post on the creation of that. We ended up getting over 1,500 responses, and our readers were largely thoughtful in what they wrote. Before we launched this project, we didn’t know how our audience would break down into gun owners and non-gun owners, and whether we would get the usual talking points from each side or hear new points of view. We fortunately did see a lot of interesting stories and patterns. But uncovering those trends in over a thousand responses in only a few days was a challenge. Reading through every response on deadline was unfeasible, so we had a machine do it. 
The idea of using natural language processing to make sense of a large number of reader comments came from Blair Hickman, Social Media Producer at ProPublica and we ended up using the Overview Project a natural language processing tool developed by the Associated Press (screenshotted above). You can read more about the technicals of how it works here but generally, it looks for clusters of related words and groups them together in that tree layout you see. 
Some interesting trends (Read the analysis with selected comments) that surfaced were a group of comments about how growing up with guns influenced people to both own and not own guns later in life. Many people framed their non-ownership of guns around their lack of a need and the algorithm found this cluster around phrases with “need” in them. Viewed in this light, many gun owner responses can be seen as implicitly responding to this prompt, explaining their clear, pragmatic need for gun ownership such as hunting, protection in rural areas or many others we included. Overview also brought out repeated comments that had variations on the phrase “When seconds count, police are minutes away,” a similar need for self-protection or a desire for self-reliance, depending on how you interpret it.
A number of readers discussed how their association with the military sometimes led them to be comfortable with firearms or, in many cases, the opposite. As one person put it: “As an army officer I came to realize many guns are tools created with a main purpose of killing. I don’t hunt and I don’t target shoot. No need for a gun.”
The clustering algorithm also grouped comments related to how experiences with family members committing suicide led people to not own guns, as well as religion. The posts concerning religion were especially interesting since in 2008 campaign speech, President Obama said that some people “cling to guns or religion” as a way to “explain their frustrations” with society. All of the comments submitted to us that discussed religion were from staunch non-gun owners, however. 
I think the post is worth a read if you’re interested in seeing an attempt at surfacing a conversation. On the design side, you can see we tried to get out of the way of the readers’ voices as much as possible: we wrote a short sentence intro framing each collection of comments and then let them tell their stories. Our front-end dev wiz Lynn Maharas actually pushed through that blue background quote style — which didn’t exist on our site before — especially for this so that we could string a bunch of quotes together and still be readable.
Using Overview was pretty easy, especially since it has a web interface now that you can upload your documents to. You have to remove any commas, first, however. We augmented some clusters with simple keyword searches of thematically related words that the algorithm might not be able to connect the dots between. For instance, if military was one cluster, we might want to look at Navy and Army as related words. This might not actually give any extra insight since the algorithm is looking for clusters anyway so it might pick up on those. But it could help guide a more analog approach once the machine has broken things down into potentially interesting categories. 
We clustered the spreadsheets of responses independently of one another, but one analysis would be to see if the machine would divide them automatically into two camps. Always more analysis to be done than deadlines allow. Here’s the anonymized data though, let us know at NewsBeastLabs@gmail.com if you find anything interesting.
-Michael

The other month, we asked readers why they did or did not own guns. Here’s the post on the creation of that. We ended up getting over 1,500 responses, and our readers were largely thoughtful in what they wrote. Before we launched this project, we didn’t know how our audience would break down into gun owners and non-gun owners, and whether we would get the usual talking points from each side or hear new points of view. We fortunately did see a lot of interesting stories and patterns. But uncovering those trends in over a thousand responses in only a few days was a challenge. Reading through every response on deadline was unfeasible, so we had a machine do it. 

The idea of using natural language processing to make sense of a large number of reader comments came from Blair Hickman, Social Media Producer at ProPublica and we ended up using the Overview Project a natural language processing tool developed by the Associated Press (screenshotted above). You can read more about the technicals of how it works here but generally, it looks for clusters of related words and groups them together in that tree layout you see. 

Some interesting trends (Read the analysis with selected comments) that surfaced were a group of comments about how growing up with guns influenced people to both own and not own guns later in life. Many people framed their non-ownership of guns around their lack of a need and the algorithm found this cluster around phrases with “need” in them. Viewed in this light, many gun owner responses can be seen as implicitly responding to this prompt, explaining their clear, pragmatic need for gun ownership such as hunting, protection in rural areas or many others we included. Overview also brought out repeated comments that had variations on the phrase “When seconds count, police are minutes away,” a similar need for self-protection or a desire for self-reliance, depending on how you interpret it.

A number of readers discussed how their association with the military sometimes led them to be comfortable with firearms or, in many cases, the opposite. As one person put it: “As an army officer I came to realize many guns are tools created with a main purpose of killing. I don’t hunt and I don’t target shoot. No need for a gun.”

The clustering algorithm also grouped comments related to how experiences with family members committing suicide led people to not own guns, as well as religion. The posts concerning religion were especially interesting since in 2008 campaign speech, President Obama said that some people “cling to guns or religion” as a way to “explain their frustrations” with society. All of the comments submitted to us that discussed religion were from staunch non-gun owners, however. 

I think the post is worth a read if you’re interested in seeing an attempt at surfacing a conversation. On the design side, you can see we tried to get out of the way of the readers’ voices as much as possible: we wrote a short sentence intro framing each collection of comments and then let them tell their stories. Our front-end dev wiz Lynn Maharas actually pushed through that blue background quote style — which didn’t exist on our site before — especially for this so that we could string a bunch of quotes together and still be readable.

Using Overview was pretty easy, especially since it has a web interface now that you can upload your documents to. You have to remove any commas, first, however. We augmented some clusters with simple keyword searches of thematically related words that the algorithm might not be able to connect the dots between. For instance, if military was one cluster, we might want to look at Navy and Army as related words. This might not actually give any extra insight since the algorithm is looking for clusters anyway so it might pick up on those. But it could help guide a more analog approach once the machine has broken things down into potentially interesting categories. 

We clustered the spreadsheets of responses independently of one another, but one analysis would be to see if the machine would divide them automatically into two camps. Always more analysis to be done than deadlines allow. Here’s the anonymized data though, let us know at NewsBeastLabs@gmail.com if you find anything interesting.

-Michael

UPDATE: FEB 10 @RepsGunTweets has been changed to @YourRepsOnGuns. Check out www.ThisIsYourRepOnGuns.com for the ongoing project.

Brian Abelson is a data scientist who is graciously donating his time at NewsBeast Labs before he starts a full-time position as a Knight-Mozilla Open News Fellow at the New York Times in February.
For an upcoming project on the gun debate, we’ve been monitoring statements representatives have made on the topic. As President Obama prepared to unveil his proposal for gun control on Wednesday, Michael and I were curious to see the reactions of representatives to the highly publicized announcement and be able to report that in real-time. Given the degree to which breaking news is now reported (and responded to) on social media, we thought it would be useful to build a bot to log officials’ comments on certain issues and present them in real time. Such a tool could be used by news rooms to engage their readers on a continuous basis by aggregating and serving content from members of particular communities or who serve on different committees.
@RepsGunTweets was born.
We were inspired by the work of 2013 Mozilla-Knight OpenNews fellows who recently built a prototpe for an app called “if (this) then news,” a news-oriented take on IFTTT – a site for linking triggers from gmail, twitter, dropbox, and other services to actions on the web. Applying this logic to news coverage, the fellows created the shell for a tool that would monitor live data streams, detect important events, and issue notifications. As Vice President Biden took the mic, we started furiously coding up a bot that would follow the twitter accounts of US Representatives and retweet any comment that included “gun”, “assault weapon”, “firearm”, or other relvant keywords. After a couple hours of missteps and headaches, we eventually got @RepsGunTweets up and running. In the last ten days, the bot has logged 307 tweets; two-thirds of which came in the first three days. We’re still analyzing the conversation but one interesting observation is representatives who are not in favor of gun control tend to link to longer explanations of their position on their website instead of tweet a comment.
Under the hood
At its core a retweet bot is a pretty simple tool: Follow a feed, find what matters, and serve it back up under a single account. The harder part is figuring out how to accurately communicate with Twitter’s API. Using tweepy for python we were able to easily access twitter’s numerous methods. All we needed to provide it with were the the consumer key, consumer secret, access token, and access token secret for an application generated on http://dev.twitter.com/apps. The bot follows CSPAN’s member of congress list and applies a regular expression for the desired keywords and retweets any matches.For even more technical info, check out this Github page


UPDATE: FEB 10 @RepsGunTweets has been changed to @YourRepsOnGuns. Check out www.ThisIsYourRepOnGuns.com for the ongoing project.

Brian Abelson is a data scientist who is graciously donating his time at NewsBeast Labs before he starts a full-time position as a Knight-Mozilla Open News Fellow at the New York Times in February.

For an upcoming project on the gun debate, we’ve been monitoring statements representatives have made on the topic. As President Obama prepared to unveil his proposal for gun control on Wednesday, Michael and I were curious to see the reactions of representatives to the highly publicized announcement and be able to report that in real-time. Given the degree to which breaking news is now reported (and responded to) on social media, we thought it would be useful to build a bot to log officials’ comments on certain issues and present them in real time. Such a tool could be used by news rooms to engage their readers on a continuous basis by aggregating and serving content from members of particular communities or who serve on different committees.

@RepsGunTweets was born.

We were inspired by the work of 2013 Mozilla-Knight OpenNews fellows who recently built a prototpe for an app called “if (this) then news,” a news-oriented take on IFTTT – a site for linking triggers from gmail, twitter, dropbox, and other services to actions on the web. Applying this logic to news coverage, the fellows created the shell for a tool that would monitor live data streams, detect important events, and issue notifications. As Vice President Biden took the mic, we started furiously coding up a bot that would follow the twitter accounts of US Representatives and retweet any comment that included “gun”, “assault weapon”, “firearm”, or other relvant keywords. After a couple hours of missteps and headaches, we eventually got @RepsGunTweets up and running. In the last ten days, the bot has logged 307 tweets; two-thirds of which came in the first three days. We’re still analyzing the conversation but one interesting observation is representatives who are not in favor of gun control tend to link to longer explanations of their position on their website instead of tweet a comment.

Under the hood

At its core a retweet bot is a pretty simple tool: Follow a feed, find what matters, and serve it back up under a single account. The harder part is figuring out how to accurately communicate with Twitter’s API. Using tweepy for python we were able to easily access twitter’s numerous methods. All we needed to provide it with were the the consumer key, consumer secret, access token, and access token secret for an application generated on http://dev.twitter.com/apps. The bot follows CSPAN’s member of congress list and applies a regular expression for the desired keywords and retweets any matches.For even more technical info, check out this Github page

Responsive Design - A Mobile Retrofit
A few weeks ago we launched our new mobile optimized design of thedailybeast.com, and we couldn’t be more excited! Go check it out if you haven’t already (if you’re not using a mobile device scroll to the bottom and click on “Mobile Site”). 
Our CTO Matt Stowe’s favorite thing about it is everything loads from the same exact URL as before — there’s no m.thedailybeast.com. In his words: “Why is this great? Because no matter how you share, bookmark, email, or use the link, the site serves you an optimized experience for your device.”
The challenge was to take our existing design and reorganize the content into a space less than 1/3 of the desktop size. In addition, we needed to account for a wide range of phone sizes, layouts (portrait/landscape) and resolutions (up to 300ppi). Based on the new design and requirements we decided to recreate all the markup and css for the entire site….10 weeks and 1,342 cups of coffee later we had ourselves a mobile site! Here are two features that we’re most excited about:
Responsive Design
Ethan Marcotte, who coined the term “Responsive Web Design”, describes 3 foundations as core to a responsive site:
Flexible Grid
Fluid Images
Media Queries
Instead of creating multiple versions of the site to fit multiple device sizes and resolutions we decided to use a responsive design approach by implementing a flexible grid system. This way our site will expand and contract to fit any width browser. For example, when your phone is turned from portrait to landscape, the site will respond by expanding to fit the screen. To produce this, we are using a 12-column flexible grid designed by Stephen Bau that is based on the great work by 960 Grid System (we love grids!). All columns widths are set as a percentage of the page rather than a fixed pixel width which allows the site to expand or contract to any size.
The easiest part of the design was making all the images “fluid" by setting their width to 100%. Now images will expand/contract to fill the size of their parent element. As for media queries, new in css3, we are currently modifying the navigation so that each element is spaced properly based on site width. In the future, we hope to implement tablet and desktop media queries for a seamless site experience across all devices.
Icon Fonts
With the new responsive site we decided to abandon the use of sprites and most icon images in our css and have opted for creating our own icon font instead. Now when you visit the mobile site, all the icons are actually a part of our custom font, not jpgs or pngs. The browser references all icon images from a small font file (~30kb). One of the benefits is that your browser will not be downloading lots of icon images, taking up device memory, bandwidth and adding to the time it takes each page to download.
An additional benefit of using a font is also so that we don’t have to recut new images if designs change and we need the image slightly bigger, smaller or a different color. Now we can change each instance of the icon with css adjusting it accordingly. Lastly, icon images will never look blurry since they are now vector data (like an .svg) instead of raster (.jpg). Here’s an example of why icon fonts are awesome. You can make your own custom fonts over at icomoon.io or download the icomoon chrome extension and work on your fonts offline.
Shout out to the entire development and design teams who created lots of really great features!
— Lynn Maharas, Front End Developer

Responsive Design - A Mobile Retrofit

A few weeks ago we launched our new mobile optimized design of thedailybeast.com, and we couldn’t be more excited! Go check it out if you haven’t already (if you’re not using a mobile device scroll to the bottom and click on “Mobile Site”). 

Our CTO Matt Stowe’s favorite thing about it is everything loads from the same exact URL as before — there’s no m.thedailybeast.com. In his words: “Why is this great? Because no matter how you share, bookmark, email, or use the link, the site serves you an optimized experience for your device.”

The challenge was to take our existing design and reorganize the content into a space less than 1/3 of the desktop size. In addition, we needed to account for a wide range of phone sizes, layouts (portrait/landscape) and resolutions (up to 300ppi). Based on the new design and requirements we decided to recreate all the markup and css for the entire site….10 weeks and 1,342 cups of coffee later we had ourselves a mobile site! Here are two features that we’re most excited about:

Responsive Design

Ethan Marcotte, who coined the term “Responsive Web Design”, describes 3 foundations as core to a responsive site:

  1. Flexible Grid
  2. Fluid Images
  3. Media Queries

Instead of creating multiple versions of the site to fit multiple device sizes and resolutions we decided to use a responsive design approach by implementing a flexible grid system. This way our site will expand and contract to fit any width browser. For example, when your phone is turned from portrait to landscape, the site will respond by expanding to fit the screen. To produce this, we are using a 12-column flexible grid designed by Stephen Bau that is based on the great work by 960 Grid System (we love grids!). All columns widths are set as a percentage of the page rather than a fixed pixel width which allows the site to expand or contract to any size.

The easiest part of the design was making all the images “fluid" by setting their width to 100%. Now images will expand/contract to fill the size of their parent element. As for media queries, new in css3, we are currently modifying the navigation so that each element is spaced properly based on site width. In the future, we hope to implement tablet and desktop media queries for a seamless site experience across all devices.

Icon Fonts

With the new responsive site we decided to abandon the use of sprites and most icon images in our css and have opted for creating our own icon font instead. Now when you visit the mobile site, all the icons are actually a part of our custom font, not jpgs or pngs. The browser references all icon images from a small font file (~30kb). One of the benefits is that your browser will not be downloading lots of icon images, taking up device memory, bandwidth and adding to the time it takes each page to download.

An additional benefit of using a font is also so that we don’t have to recut new images if designs change and we need the image slightly bigger, smaller or a different color. Now we can change each instance of the icon with css adjusting it accordingly. Lastly, icon images will never look blurry since they are now vector data (like an .svg) instead of raster (.jpg). Here’s an example of why icon fonts are awesome. You can make your own custom fonts over at icomoon.io or download the icomoon chrome extension and work on your fonts offline.

Shout out to the entire development and design teams who created lots of really great features!

— Lynn Maharas, Front End Developer

Six Months in Review

NewsBeast Labs is roughly six months old and we’ve had a lot of fun. This tumblr has most of our projects for the past few months but there are a bunch from before our launch. Here’s a rough list of projects we’ve done so far.

Legal Experts Decode the Supreme Court’s Obamacare Ruling - Our very first project! We launched it the day we got DocumentCloud, which was also the morning of the Supreme Court ruling on Obamacare. We asked two law professors to make margin notes in the text of the ruling as they were reading it for the first time. Readers could follow along and read experts’ reactions as the conversation was happening.

Digital 100: Who’s Following Whom? - A network visualization of how Newsweek’s list of influential people in the digital space interact with each other on Twitter. 

Obamacare: It’s Cheaper! - I like to call these “Story Visualizations” - visual presentations of stories that could run as a list or as text, but are much more interesting visually. Matt DeLuca and I did a side-by-side on how Obamacare would affect different age groups’ healthcare spending.

2012 Olympics: The Latest Medal Tally - We had a live-updating Olympic Medal Count, (with a snazzy sortable table that I’ve written about before) that I worked on with our awesome intern Sarah Hedgecock. We also did a version of it for our right rail (sidebar).

Interactive Map: London’s Olympic Transformation - The Olympic Park rose from the rust. Sarah and I also did a satellite view before and after interactive that included a bunch of info on the star-chitect buildings.

Interactive Map: The U.S. Shooting Epidemic - Following the Aurora shooting, Brian Abelson and I made an interactive map of multiple-victim shootings since 2005 and asked readers to respond with their memories. We published a selection of the reader responses here. The full spreadsheet list is here.

As Income Inequality Widens, Rich Presidential Candidates Dominate - Lauren Streib and I worked on a chart (she did all the numbers), showing presidential income over the years. I remember this one chart taking four hours from start to finish for some reason…

Big Guns Inside the National Rifle Association Leadership - Who’s leading the NRA? I worked on a project with three colleagues Caitlin Dickson, Eliza Shapiro and Kevin Fallon on the NRA’s leadership. They dug through 990 forms and put together small profiles of the people at the top. We put it together in mosaic-style presentation. Normally this type of story would be a gallery format but since it’s not picture-based, we decided to create something more conducive to reading a lot of text.

SuperPAC App Election Ad Interactive - We partnered with the Super PAC App, an iPhone app that would identify political advertisements on TV and give you information about that group, such as how much money it was spending this election and articles about them. We made a web interface to their data to provide readers with more context for outside spending groups.

Interactive Map: Who’s Protesting Where? - When the Middle East erupted in protests in response to an anti-Muslim video uploaded to YouTube, Eliza Shapiro and I put together a visual guide with information on each protest as well as contextual information on each country. It was an interesting map to built since we had both point and polygon layers to deal with for hover states. As with all of our interactive maps, we used CartoDB.

Obama and Romney’s Bundlers - If bundlers had baseball cards, this is what they’d look like.We took a look at the biggest bundlers for each candidate. Collect ‘em all.

The Rise of the Political Non-Profit - How so-called “Dark Money” was influencing the 2012 election was one of the themes in a three-part series John Avlon and I wrote called the Super PAC Economy. This animated timeline overlays non-profit political expenditures and significant court decisions (Citizens United and lesser-known decisions) that determined what role these groups could play in politics.

The Dark Money Shuffle - Also in that series, we worked with Robert Maguire of the Center for Responsive Politics who had been compiling a database of grants that non-profits gave to each other. For the first time, we diagrammed this opaque world of money transfers that is only visible by manually going through hundreds of IRS forms. Full article

Election Right Rail - Showing the latest polls from battleground states, how those states voted historically, median income, and latest unemployment figures, our politics sidebar was full of context. It no longer lives anywhere on our site but you can see a standalone version how it looked on the eve of the election through the linked title.

Note: We did all of these projects before starting this tumblr. You’ll find write-ups for the projects that follow but if you want to know how we built any of the stuff above, send me a message at @mhkeller.

Debate Dashboard and Bingo - Brian, Sam, Vitaly Korenkov (one of our awesome developers) conceived of a great debate night dashboard. We had a livestream, a live chat with our commentators and a poll from Urtak, which is a polling platform that lets you pose simple yes/no/maybe questions to readers. It also lets readers submit questions they want other people to answer so it’s a good back and forth between questions we’re interested in and what our audience is interested in. We’re often into giving our readers a voice on the site so we liked it a lot. I came in during the last few hours before we were going to go live (a.ka. after all the hard work was done) and added a bingo card. The coolest part about it is the Bingo validation. The card checks how many you have in a row vertically, horizontally, and diagonally and tells you how many you need to win. NewsBeast Labs post.

Ground game: Obama Campaign Opens Up Big Lead in Field Offices - The airwave battle was being covered left and right, but we wanted to know what was happening on the ground. We scraped the two campaigns’ websites to map out their local HQs nationwide and found a big discrepancy between the two camps. In Ohio, for instance, Obama had a presence in so many counties where Romney didn’t that 10 percent of the state’s population lived in a county where the only volunteer center was an Obama HQ.

Technical note: We used CartoDB again for this map and it was a huge help. In the accompanying article, we ran interactive maps of Florida, Ohio, and Virginia. These separate maps required no real extra programming or map making since CartoDB builds your map by querying a database. By setting our map query to ‘SELECT * from national_map WHERE state = FL’  we had a local map in minutes that we could swap out for another state if needed, which indeed ended up happening. NewsBeast Labs post.

Interactive Hate: The Great Obama-Loathing Canon - Matt DeLuca and I teamed up again to solve the perennial problem of how do you present a lot of information to the reader in a way they can digest in bites that make sense. This time, we presented over a hundred anti-Obama books in a mosaic that you can filter down to different subject matters. NewsBeast Labs Post.

HavingTroubleVoting.com - We did an experiment on election day asking our readers, or anyone really, if they were having trouble voting, and if so, what kind of trouble. We plotted the responses on a map below and color-coded the markers based on the type of problem. We partnered with Mother Jones on it to help us go through the responses to find patterns and to contact people to tell their story. Our own reporters used the database in stories about massive lines and machine malfunctions. We’re totally honored and floored when CJR named it No. 2 in their Must-Read Interactives of 2012! More about it in our NewsBeast Labs post.

Election Night Interactive Map and Dashboard - A lot of teamwork went into our election night coverage from the development team, social, design… the list goes on. We took over our home page on election night with video commentary, a live updating tally, a live chat, article updates and more things that you could probably put a “live” prefix in front of. The map lives on in the linked title, a screenshot lives in our NewsBeast Labs post about it.

‘It Was Like a War Zone’: Hurricane-Ravaged Staten Island Reels - In the wake of the trauma caused by Hurricane Sandy, we did a map of Staten Island victims. It shows how many of the fatal tragedies were concentrated on the east side of the island.

Not-So-Super PACs: 2012’s Winners and Losers - DeLuca and I teamed up again to produce this tally of who made good investments this election cycle. There’s a long post about it, including some failed versions in our NewsBeast Labs post

Interactive Holiday Gift Guide - Lizzie Crocker, Isabel Wilkinson and I help you find out what sub-culture your friends might belong to in this gift guide flow chart.

Own a Gun? Tell Us Why? - December brought another terrible shooting and has caused much thought over the state of gun laws. We wanted to hear from rational people on both sides of the debate by lettings readers complete the sentences, “I own a gun because…” or “I don’t own a gun because…”. In three days, we had over 1,300 responses that represented very civil remarks from each group, for the most part. We analyzed the responses and did a state-by-state breakdown of the common themes. We used some interesting algorithmic clustering to find these patterns so expect a write-up soon. For now, read the post on how the project was born and how we collected the responses.

Our last Wednesday Workshop focused, in part, on ways to get readers more involved in our stories. When news of last week’s awful shooting reached us, we wanted to open up discussion on the role of guns in America. On our Tumblr we asked readers how the shooting should be covered and many requested we steer clear of the politics and instead opt for a genuine discussion on gun control.
Gun control is a complicated issue in this country and nuanced issues can be at odds with the tools of data visualization. That is to say, data visualization and data reporting are often marked by being extremely comprehensive and boiling that comprehensiveness into one easily understandable image, graph, or layout. Doing anything comprehensive on an issue as complex as guns in our society, on deadline no less, would be tricky, and we’re not ones to put data out there that’s misleading or inconclusive.
But the other tool of digital journalism is being able to present a great deal of information in one place, which does work for a nuanced subject. We wanted to engage our readers to tell the story of guns in America in a way that showed the issue’s complexity. We posed the question as “Why do you own a gun?” or “Why don’t you own a gun?” On our site we, we set up two forms that let readers easily complete the sentence “I own a gun because…” or “I don’t own a gun because…” and displayed their responses for readers to sift through.
It’s like the digital equivalent of Man on the Street reporting, where you go and ask people on the street their opinions on an issue in the news and write up their quotes in an article. Let’s call this a Man on the Internet story, or to be gender neutral, Person on the Internet (Internet Vox Pop maybe? I’m open to suggestions).
We published the article Monday evening and less than 24 hours later we have over 900 responses — over 500 from gun owners and over 400 from non-gun owners. We have some thoughts on how the two sides explain their position but, for now, we’ll let you read through and absorb it on your own.
We’re collecting and categorizing the responses, so look for that article on the Beast later in the week.
Under the hood
We used a customized Google Form to handle the response collections. This is a nice tutorial on how to embed Google Forms into your site with custom styles and functionality. 
We’ve used custom Google Forms before, on our other shooting project actually, for a newsapp that lets readers put in their address and it finds news accounts of multiple-victim shootings near them. A Google Form then asks what they remember about the incident and collects their responses in a spreadsheet. We published some of the most moving responses that I think is worth a read.
For this project, though, I was running into trouble putting in two custom forms on one page. Since I only had a day to build this, I ended up sequestering the two forms to separate HTMl pages and iframe-ing them into my main page. This was nice because it ensured the two forms didn’t interfere with each other and since the pages were all on the same domain, I didn’t have any cross-origin issues and didn’t loose any functionaity — when you submit one form, that action bubbles up to it the parent frame and grays out the other form.
I originally wanted to do something more animated similar to this seminal piece of crowdsourced dataviz from 2008. I like how its animation gives the project energy but it comes at the cost of not being able to scroll through the responses on your own. After some thinking, I couldn’t figure out a way to have both a sit-back-and-let-the-responses-flow experience and a I-want-to-dive-into-these-responses-and-scroll-through-them-all experience. The latter is obviously the more useful for the reader, so I went with that. The election interactive is also a bit different from this since most of the emotions on each line are of the same category, so it’s not really hiding anything by not letting you scroll. For our project, each response brings its own nuance to the debate so you don’t want to hide any of them. If you have any thoughts on how to improve the presentation, I’m at @mhkeller.
Brian had the great idea that we let this conversation be medium agnostic. So in the story dek we let people know they can continue the conversation on Twitter with the hashtags #IOwnAGunBecause or #IDontOwnAGunBecause. I built some hooks into our Underscore.js templates that let us add selected tweets to our spreadsheet and display them with a Twitter icon and a link to the original tweet. That way we could pull in interesting responses from elsewhere and flag them as such. You can look at the formatHelpers object in app.js to see how it checks for content in the Twitter column and adds the image and link if it finds something.
As I’ve written, I’m a big fan of Miso’s Dataset.js, and that’s what we’re using here to pull the responses in from our Google Form Spreadsheet. Contrary to my previous post, this app does work off of a live Google Doc. I know, blasphemous. For a few workflow reasons we weren’t able to have a script download our spreadsheets and put them on a server like we did for HavingTroubleVoting.com where we had both rate-limiting and privacy issues.
That being said, we have been very closely monitoring the app to make sure it doesn’t get rate-limited and it’s been fine so far. I have a few lines commented out in the code that point to where we’d put a local CSV of the responses, so if the app went down, it would be back up in a minute or so. We also made sure not to ask for any identifying information so we had no privacy concerns. Now that we have close to a thousand responses, though, we might switch to local files so that the page loads faster. If we could have set it up to download automatically, however, that would have been our first choice.
One thing I added yesterday evening after we started getting a lot of comments was a way to filter by state. A lot of content can be overwhelming, so the more options you can give readers to drill down to a subset that might be more relevant to them, the more manageable the experience is and hopefully more engaging and memorable.

-michael keller

Our last Wednesday Workshop focused, in part, on ways to get readers more involved in our stories. When news of last week’s awful shooting reached us, we wanted to open up discussion on the role of guns in America. On our Tumblr we asked readers how the shooting should be covered and many requested we steer clear of the politics and instead opt for a genuine discussion on gun control.

Gun control is a complicated issue in this country and nuanced issues can be at odds with the tools of data visualization. That is to say, data visualization and data reporting are often marked by being extremely comprehensive and boiling that comprehensiveness into one easily understandable image, graph, or layout. Doing anything comprehensive on an issue as complex as guns in our society, on deadline no less, would be tricky, and we’re not ones to put data out there that’s misleading or inconclusive.

But the other tool of digital journalism is being able to present a great deal of information in one place, which does work for a nuanced subject. We wanted to engage our readers to tell the story of guns in America in a way that showed the issue’s complexity. We posed the question as “Why do you own a gun?” or “Why don’t you own a gun?” On our site we, we set up two forms that let readers easily complete the sentence “I own a gun because…” or “I don’t own a gun because…” and displayed their responses for readers to sift through.

It’s like the digital equivalent of Man on the Street reporting, where you go and ask people on the street their opinions on an issue in the news and write up their quotes in an article. Let’s call this a Man on the Internet story, or to be gender neutral, Person on the Internet (Internet Vox Pop maybe? I’m open to suggestions).

We published the article Monday evening and less than 24 hours later we have over 900 responses — over 500 from gun owners and over 400 from non-gun owners. We have some thoughts on how the two sides explain their position but, for now, we’ll let you read through and absorb it on your own.

We’re collecting and categorizing the responses, so look for that article on the Beast later in the week.

Under the hood

We used a customized Google Form to handle the response collections. This is a nice tutorial on how to embed Google Forms into your site with custom styles and functionality. 

We’ve used custom Google Forms before, on our other shooting project actually, for a newsapp that lets readers put in their address and it finds news accounts of multiple-victim shootings near them. A Google Form then asks what they remember about the incident and collects their responses in a spreadsheet. We published some of the most moving responses that I think is worth a read.

For this project, though, I was running into trouble putting in two custom forms on one page. Since I only had a day to build this, I ended up sequestering the two forms to separate HTMl pages and iframe-ing them into my main page. This was nice because it ensured the two forms didn’t interfere with each other and since the pages were all on the same domain, I didn’t have any cross-origin issues and didn’t loose any functionaity — when you submit one form, that action bubbles up to it the parent frame and grays out the other form.

I originally wanted to do something more animated similar to this seminal piece of crowdsourced dataviz from 2008. I like how its animation gives the project energy but it comes at the cost of not being able to scroll through the responses on your own. After some thinking, I couldn’t figure out a way to have both a sit-back-and-let-the-responses-flow experience and a I-want-to-dive-into-these-responses-and-scroll-through-them-all experience. The latter is obviously the more useful for the reader, so I went with that. The election interactive is also a bit different from this since most of the emotions on each line are of the same category, so it’s not really hiding anything by not letting you scroll. For our project, each response brings its own nuance to the debate so you don’t want to hide any of them. If you have any thoughts on how to improve the presentation, I’m at @mhkeller.

Brian had the great idea that we let this conversation be medium agnostic. So in the story dek we let people know they can continue the conversation on Twitter with the hashtags #IOwnAGunBecause or #IDontOwnAGunBecause. I built some hooks into our Underscore.js templates that let us add selected tweets to our spreadsheet and display them with a Twitter icon and a link to the original tweet. That way we could pull in interesting responses from elsewhere and flag them as such. You can look at the formatHelpers object in app.js to see how it checks for content in the Twitter column and adds the image and link if it finds something.

As I’ve written, I’m a big fan of Miso’s Dataset.js, and that’s what we’re using here to pull the responses in from our Google Form Spreadsheet. Contrary to my previous post, this app does work off of a live Google Doc. I know, blasphemous. For a few workflow reasons we weren’t able to have a script download our spreadsheets and put them on a server like we did for HavingTroubleVoting.com where we had both rate-limiting and privacy issues.

That being said, we have been very closely monitoring the app to make sure it doesn’t get rate-limited and it’s been fine so far. I have a few lines commented out in the code that point to where we’d put a local CSV of the responses, so if the app went down, it would be back up in a minute or so. We also made sure not to ask for any identifying information so we had no privacy concerns. Now that we have close to a thousand responses, though, we might switch to local files so that the page loads faster. If we could have set it up to download automatically, however, that would have been our first choice.

One thing I added yesterday evening after we started getting a lot of comments was a way to filter by state. A lot of content can be overwhelming, so the more options you can give readers to drill down to a subset that might be more relevant to them, the more manageable the experience is and hopefully more engaging and memorable.

-michael keller

Oh hey! Adweek’s Charlie Warzel wrote a story about our team!

Oh hey! Adweek’s Charlie Warzel wrote a story about our team!

Google Docs + Miso-Powered Apps: a note on collaborative workflow

Working both collaboratively and efficiently on deadline, needless to say, is important. For our fastest-paced interactives and news apps, we’ve come to use a combination of Google Spreadsheets and Dataset.js from the Miso Project in a way that we’ve come to really quite like. We figured we’d share it with you here.

Essentially, we use a Google Doc as a temporary database for our text or numeric data. When we go live, we download that data to a local CSV, change the path, and deploy.

We’ve used this workflow on the Obama Haters Book Club, the sidebar to our election coverage page (which doesn’t exist on our site anymore but which looked like this), our Meet Romney’s Bundlers interactive, a variant of it for HavingTroubleVoting.com (where we had a Google Form power a Google Spreadsheet) and the Olympic Medal Tally among others. 

Note: This post is a bit more technical than usual.

The first step is getting everyone to put the information they gather into a structured spreadsheet. Here’s what the Obama bundlers spreadsheet looked like.

From there, we “Publish the spreadsheet” from the File menu — the crucial step when running an app off of a Google Spreadsheet — and import the data into our app. The Dataset.js documentation has a very simple tutorial on that step: http://misoproject.com/dataset/tutorials/googlespreadsheets

With this setup, reporters can be writing text and the photography department can be adding links to images at the same time the interactive is being built. What’s particularly great is an editor can edit the text in the spreadsheet and it will be piped into the interactive without his or her having to edit any HTML. The order of the rows can also be swapped if some elements need to appear before others on the page.

Mother Jones’s Tasneem Raja has written about this workflow as well, using Tabletop.js. Her post is definitely worth a read. 

So Why Dataset over Tabletop?

I’ve come to like Dataset.js over Tabletop.js for two main reasons. The first is that powering a live news app from a Google Spreadsheet can cause a lot of problems—so we download our data from the cloud onto our servers before going live. Dataset.js makes this transition really simple.

Jeremy Singer-Vine at the Wall Street Journal has written about the dangers of running live apps from Google Docs and he has a strong argument: 1) Google can rate-limit you, killing your live app; 2) If Google changes its API in the future your app may go offline; 3) Google can add extra data to your API request that can slow down your readers; 4) Sometimes you want to make a bunch of changes all at once, like add a complete row of information — with Google, incremental changes cell-by-cell are immediately reflected in the app, which will create broken elements until you’ve finished a full row; 5) Security and data privacy — sometimes you have columns in your data that you want to use internally but shouldn’t be public facing. We ran into this with HavingTroubleVoting.com where we collected ready contact information to later verify reports but we didn’t want to make that info public to the world. 

By making a local CSV whose columns we’re picking for public view, we solve a lot of these problems. We get the benefit of working collaboratively in development and can switch to something more archivable and robust for production. I have some other scripts that could be improved that will download a spreadsheet automatically and turn it into a clean CSV. If I can clean them up a bit we can post them as well. 

The second reason is Dataset.js gives your data a lot of functionality, such as the ability to group, query, and turn your columns into arrays (which is particularly useful for hooking it up to libraries like HighCharts). In fact, being able to hook up a Google Spreadsheet to your app is just an extra feature of Dataset.js — as the name suggests, it’s built to work with data and it can do some fancy things regardless of whether you’re working with a local CSV or a spreadsheet in the cloud. 

If you have any suggestions, I’m at @mhkeller.

-michael

For the holidays, we wanted to make the normal gift guide idea a little more interesting. The answer was a flow chart that narrows down the type of person you’re shopping for and then suggests appropriate gifts for their particular sub-culture. Lizzie Crocker and Isabel Wilkinson did a great job coming up with the categories like (Nostalgic Outdoorsman, The Closet ‘50 Shades’ Fan, Hipster Techie) and then Lizzie and I (Michael) put our heads together to make a (hopefully) witty flowchart. Our photo department headed by Marcia Allert was also a huge help.
There’s nothing fancy under the hood. The only semi-trick was our Daily Beast font family Titling isn’t always too legible at smaller sizes on the web. To fix that, we made the blue circles in Illustrator and exported them as PNGs. To do the hovers, you duplicate the image below your main image like this and make sure your image container is only tall enough to show one at a time: 

To do the hover then, your CSS is something like .img-class:hover{ background-position: 0 -102px;} The benefit to this is you don’t load a second picture when you hover so there’s no delay. This is a pretty standard way of doing this so it’s nothing revolutionary but for some reason you still see a lot of sites that have delays on their image hovers that would be better off using this technique.
Why not make the flowchart interactive too?
We decided in favor of a static image for the flowchart as opposed to something interactive since I think the wittiness of flowcharts comes across in seeing how the possibilities flow from one another and the different results in choosing one adventure over another. Also, at the sake of sounding blasphemous, interactivity can be thought of as a last resort only when you can’t fit everything onto the screen at once, or doing so would work against a focused narrative. It’s much easier for readers to scan a page full of information and see how these options unfold, in my view, than it is for them to click 10+ times and only see the options that stem from their responses. 
To continue the trend to show some shots from our failures, here’s a screenshot of the drafting process for the flowchart. We really wanted to work mud into it somewhere since it could have split off nicely with Adrenaline Junkie or Wee Ones (since kids love mud), but it didn’t quite work out. The second one is what we settled on.




-michael

For the holidays, we wanted to make the normal gift guide idea a little more interesting. The answer was a flow chart that narrows down the type of person you’re shopping for and then suggests appropriate gifts for their particular sub-culture. Lizzie Crocker and Isabel Wilkinson did a great job coming up with the categories like (Nostalgic Outdoorsman, The Closet ‘50 Shades’ Fan, Hipster Techie) and then Lizzie and I (Michael) put our heads together to make a (hopefully) witty flowchart. Our photo department headed by Marcia Allert was also a huge help.

There’s nothing fancy under the hood. The only semi-trick was our Daily Beast font family Titling isn’t always too legible at smaller sizes on the web. To fix that, we made the blue circles in Illustrator and exported them as PNGs. To do the hovers, you duplicate the image below your main image like this and make sure your image container is only tall enough to show one at a time: 

image

To do the hover then, your CSS is something like .img-class:hover{ background-position: 0 -102px;} The benefit to this is you don’t load a second picture when you hover so there’s no delay. This is a pretty standard way of doing this so it’s nothing revolutionary but for some reason you still see a lot of sites that have delays on their image hovers that would be better off using this technique.

Why not make the flowchart interactive too?

We decided in favor of a static image for the flowchart as opposed to something interactive since I think the wittiness of flowcharts comes across in seeing how the possibilities flow from one another and the different results in choosing one adventure over another. Also, at the sake of sounding blasphemous, interactivity can be thought of as a last resort only when you can’t fit everything onto the screen at once, or doing so would work against a focused narrative. It’s much easier for readers to scan a page full of information and see how these options unfold, in my view, than it is for them to click 10+ times and only see the options that stem from their responses. 

To continue the trend to show some shots from our failures, here’s a screenshot of the drafting process for the flowchart. We really wanted to work mud into it somewhere since it could have split off nicely with Adrenaline Junkie or Wee Ones (since kids love mud), but it didn’t quite work out. The second one is what we settled on.

Gift Guide Failure

Gift Guide Final

-michael

We recently gave a lesson to some editors and reporters about how to use TweetDeck, the awesome Twitter-streaming product. We thought we’d publish our step-by-step walk-through here, both so our reporters could bookmark it for future reference and to show you all what we’re up to. Obviously it’s just the basics detailed below, aimed at first-time TweetDeck users. Maybe we’ll have a more advanced class down the road.
What is TweetDeck, and why might I want to use it?
If you have a Twitter account already, you probably view and compose tweets through twitter.com. Tweetdeck is a separate Twitter product that offers a few advantages to simply using twitter.com. The most obvious advantage is that streams tweets in near-real-time, as opposed to twitter.com, which only tells you when new tweets are available to be viewed (“12 new Tweets”). The second advantage is that you can view multiple columns of tweets, rather than just your own timeline. Other columns can be lists, searches, even other users’ timelines. We’ll get to all that below. 
How to Setup TweetDeck:
First, you’ll have to decide if you want to use the TweetDeck application for the desktop or TweetDeck for the web (i.e. in-browser). Tweetdeck for the web will live in a browser tab. The desktop application will be a separate application. Ries and I both use the desktop application. It’s not a huge deal which you choose— you can switch back and forth very easily.
To setup the desktop app: Visit tweetdeck.com and click the silver download button on the right. 
To use TweetDeck in your browser: visit https://web.tweetdeck.com/
No matter which form (desktop or web browser) of TweetDeck you’ve chosen, the instructions from here on out are the same.
When you visit https://web.tweetdeck.com/ or open the application for the first time, you’ll be asked to sign in with your TweetDeck account. Note that this is different from your Twitter account. So you’ll need to create a TweetDeck account. But don’t panic. You’ll only have to log-in to Tweetdeck once and it will keep you logged in (almost) forever (click the checkbox that says “keep me logged in”). However, just this first time, you’ll need to create an account. You can use the same email and password as you do for your Twitter account.
You’ll now be asked to add your Twitter Account— here is where you log in as you would on twitter.com. Remember, Tweetdeck can handle multiple Twitter accounts, but most of you will just have your personal account. 
OK, it’s setup. Now what?
Now that Tweetdeck is open, you’ll want to make some columns. Your far left column defaults to “Home,” which you’ll probably want to keep there as it is all the accounts you follow. This is the column you see at twitter.com when you’re logged in.
A good second column is “Interactions”— which you may know as the “Connect” screen on Twitter.com, which shows your @replies, etc.. To add a column for your interactions, click the circular button with a plus sign in the middle to add a new column. Then click “interactions” and select your account.  
You may also want to add lists, like say, the Cheat Sheet source list we created. This is a bit trickier, but will become more intuitive. 
How to Add a List as a New Column in Tweetdeck:
1. Go to twitter.com in your browser. Log in to your personal Twitter account.
2. Find The Daily Beast on Twitter (https://twitter.com/thedailybeast). Click the “Lists” link on the left. Scroll down and click on “CheatSheet”. You should find yourself at: https://twitter.com/thedailybeast/cheatsheet. 
3. Click the Subscribe button on the left. 
4. Go back to Tweetdeck. Click the plus button to add a new column. Go to “Lists”. Find and click on “CheatSheet” on the left and then click the “Add column” button. 
Note: You can subscribe to as many lists as you like, and they obviously can be lists made by other users (not just @thedailybeast). You can also create your own lists. 
The third type of column you may want to have is a simple search column. Say you want to see what people are tweeting about Sandy. The best way to do this is to follow the hashtag #Sandy. Again, click the add column button, now go to search and type in #Sandy. Click add column, and you should be all set. 
Organizing Your Columns
The TweetDeck desktop application can display any number of columns from 3 to as many as you can fit on your monitor(s). To re-order your columns, click the “Columns” bar in the top middle of the application. You’ll see the title of your columns displayed in order. You can drag and drop them in to a desired order by clicking and holding on the 6 white squares on the right side of any column box. 

Again, if you need help with any of this let us know.

-Sam

We recently gave a lesson to some editors and reporters about how to use TweetDeck, the awesome Twitter-streaming product. We thought we’d publish our step-by-step walk-through here, both so our reporters could bookmark it for future reference and to show you all what we’re up to. Obviously it’s just the basics detailed below, aimed at first-time TweetDeck users. Maybe we’ll have a more advanced class down the road.

What is TweetDeck, and why might I want to use it?

If you have a Twitter account already, you probably view and compose tweets through twitter.com. Tweetdeck is a separate Twitter product that offers a few advantages to simply using twitter.com. The most obvious advantage is that streams tweets in near-real-time, as opposed to twitter.com, which only tells you when new tweets are available to be viewed (“12 new Tweets”). The second advantage is that you can view multiple columns of tweets, rather than just your own timeline. Other columns can be lists, searches, even other users’ timelines. We’ll get to all that below. 

How to Setup TweetDeck:

First, you’ll have to decide if you want to use the TweetDeck application for the desktop or TweetDeck for the web (i.e. in-browser). Tweetdeck for the web will live in a browser tab. The desktop application will be a separate application. Ries and I both use the desktop application. It’s not a huge deal which you choose— you can switch back and forth very easily.

To setup the desktop app: Visit tweetdeck.com and click the silver download button on the right. 

To use TweetDeck in your browser: visit https://web.tweetdeck.com/

No matter which form (desktop or web browser) of TweetDeck you’ve chosen, the instructions from here on out are the same.

When you visit https://web.tweetdeck.com/ or open the application for the first time, you’ll be asked to sign in with your TweetDeck account. Note that this is different from your Twitter account. So you’ll need to create a TweetDeck account. But don’t panic. You’ll only have to log-in to Tweetdeck once and it will keep you logged in (almost) forever (click the checkbox that says “keep me logged in”). However, just this first time, you’ll need to create an account. You can use the same email and password as you do for your Twitter account.

You’ll now be asked to add your Twitter Account— here is where you log in as you would on twitter.com. Remember, Tweetdeck can handle multiple Twitter accounts, but most of you will just have your personal account. 

OK, it’s setup. Now what?

Now that Tweetdeck is open, you’ll want to make some columns. Your far left column defaults to “Home,” which you’ll probably want to keep there as it is all the accounts you follow. This is the column you see at twitter.com when you’re logged in.

A good second column is “Interactions”— which you may know as the “Connect” screen on Twitter.com, which shows your @replies, etc.. To add a column for your interactions, click the circular button with a plus sign in the middle to add a new column. Then click “interactions” and select your account.  

You may also want to add lists, like say, the Cheat Sheet source list we created. This is a bit trickier, but will become more intuitive. 

How to Add a List as a New Column in Tweetdeck:

1. Go to twitter.com in your browser. Log in to your personal Twitter account.

2. Find The Daily Beast on Twitter (https://twitter.com/thedailybeast). Click the “Lists” link on the left. Scroll down and click on “CheatSheet”. You should find yourself at: https://twitter.com/thedailybeast/cheatsheet

3. Click the Subscribe button on the left. 

4. Go back to Tweetdeck. Click the plus button to add a new column. Go to “Lists”. Find and click on “CheatSheet” on the left and then click the “Add column” button. 

Note: You can subscribe to as many lists as you like, and they obviously can be lists made by other users (not just @thedailybeast). You can also create your own lists. 

The third type of column you may want to have is a simple search column. Say you want to see what people are tweeting about Sandy. The best way to do this is to follow the hashtag #Sandy. Again, click the add column button, now go to search and type in #Sandy. Click add column, and you should be all set. 

Organizing Your Columns

The TweetDeck desktop application can display any number of columns from 3 to as many as you can fit on your monitor(s). To re-order your columns, click the “Columns” bar in the top middle of the application. You’ll see the title of your columns displayed in order. You can drag and drop them in to a desired order by clicking and holding on the 6 white squares on the right side of any column box. 

image

Again, if you need help with any of this let us know.

-Sam

A defining characteristic of this election cycle was Super PACs and the hundreds of millions of dollars outside groups were spending to influence races. Now that it’s all over, we wanted to see which outside groups spent their money on succssful races and which did not. The result was our interactive Not-So-Super PACs: 2012’s Winners and Losers.
Super PACs abounded this cycle. So instead of trying to document and display all of them, we focused the narrative on how well the biggest spenders and their donors fared. To execute it, we used Center for Responsive Politics anaylsis of FEC data to find how much each PAC had spent so far in each race and then manually went through and coded each race whether the outcome was in line with or against the PAC’s interest. Then we added everything up.
Visualizing it 
This idea went through a few iterations before settling on what you see above. For a while, we’ve been wanting to use a tower graphic template - one of those vertical scroll layouts with a sticky table of contents - that I built a couple of months ago but it never seems to work out. This time, after thinking about all of the detail we wanted to display we thought bigger.
If you’re trying to visualize money flows, Sankey lines are a go-to. ProPublica did a great one showing overlapping Super PAC expenditures and you see them as flat graphics too. They show direcionality and volume = great for money.
Getting the right data
Money was flowing from donors to PACs and then to races, so we used the JSON structure that D3 lays out for its network layouts (and Sankey) visualizations. You have a list of nodes (People and PACs) and a list of node to node links (X person gave $Y to Z PAC). We were working collaboratively in Google Docs so were able to do some formulas that would print out or data structure in JSON as we were editing the document. Very handy in case you need to correct any numbers or name spellings.
Our D3 visualization was a failure.
Here’s a link to the interactive version (yes, it’s in the “failures” folder). As you can see, there were too many races to fit on the screen and the dollar amounts in some races were so high that they dwarfed everything else. So showing each race in the Sankey was out. 
This led to Sankey Idea #2.

We connected photos of the donors to the PACs, showed the percentage of succesful funds, and then put the races in a table down below. The photos were very useful because you can quickly understand that money is coming from a person and going somewhere. If we just had text, I think, without photos, it would be less clear and have less of a personality. Someone remarked that the lines almost form bodies and arms that reach out to touch Super PACs. It’s interesting to see visualized data combined with photography work out to tell a story like that.
Under the hood
We used Raphael to draw the lines, which was an improvement from D3 since we do indeed support Internet Explorer 8. We tweaked Al Shaw’s Sankey line from Tom Counsell’s Sankey library to make them span vertically instead of left to right. Here’s a jsFiddle of the code to draw the line.
The table uses Isotope.js for its animated sorting, which is snazzy but I also think does help make tabular data more understandable. Instead of clicking on a column header and everything resorts in a flash, you can see how dramatically different rows vary from view to view. It’s also nice because you can do filtering. So without much code you have a filterable, sortable table. It also saves a step of turning the object data into arrays for sorting. I’ve been wanting to add those ascending / descending arrows for a while to our tables so this was a good time for that.
This table will probably become our first stand-alone NewsBeast Labs plugin since we’ve been using it pretty frequently. That’s pretty cool because five months ago we didn’t have any interactive news code and now we’ve done enough projects that we can see what’s worked, what functionality we like and can wrap it all up into something more robust and reusable, which will make our future development that much faster. 
-michael

A defining characteristic of this election cycle was Super PACs and the hundreds of millions of dollars outside groups were spending to influence races. Now that it’s all over, we wanted to see which outside groups spent their money on succssful races and which did not. The result was our interactive Not-So-Super PACs: 2012’s Winners and Losers.

Super PACs abounded this cycle. So instead of trying to document and display all of them, we focused the narrative on how well the biggest spenders and their donors fared. To execute it, we used Center for Responsive Politics anaylsis of FEC data to find how much each PAC had spent so far in each race and then manually went through and coded each race whether the outcome was in line with or against the PAC’s interest. Then we added everything up.

Visualizing it 

This idea went through a few iterations before settling on what you see above. For a while, we’ve been wanting to use a tower graphic template - one of those vertical scroll layouts with a sticky table of contents - that I built a couple of months ago but it never seems to work out. This time, after thinking about all of the detail we wanted to display we thought bigger.

If you’re trying to visualize money flows, Sankey lines are a go-to. ProPublica did a great one showing overlapping Super PAC expenditures and you see them as flat graphics too. They show direcionality and volume = great for money.

Getting the right data

Money was flowing from donors to PACs and then to races, so we used the JSON structure that D3 lays out for its network layouts (and Sankey) visualizations. You have a list of nodes (People and PACs) and a list of node to node links (X person gave $Y to Z PAC). We were working collaboratively in Google Docs so were able to do some formulas that would print out or data structure in JSON as we were editing the document. Very handy in case you need to correct any numbers or name spellings.

Our D3 visualization was a failure.Sankey Fail

Here’s a link to the interactive version (yes, it’s in the “failures” folder). As you can see, there were too many races to fit on the screen and the dollar amounts in some races were so high that they dwarfed everything else. So showing each race in the Sankey was out. 

This led to Sankey Idea #2.

Sankey Sketch

We connected photos of the donors to the PACs, showed the percentage of succesful funds, and then put the races in a table down below. The photos were very useful because you can quickly understand that money is coming from a person and going somewhere. If we just had text, I think, without photos, it would be less clear and have less of a personality. Someone remarked that the lines almost form bodies and arms that reach out to touch Super PACs. It’s interesting to see visualized data combined with photography work out to tell a story like that.

Under the hood

We used Raphael to draw the lines, which was an improvement from D3 since we do indeed support Internet Explorer 8. We tweaked Al Shaw’s Sankey line from Tom Counsell’s Sankey library to make them span vertically instead of left to right. Here’s a jsFiddle of the code to draw the line.

The table uses Isotope.js for its animated sorting, which is snazzy but I also think does help make tabular data more understandable. Instead of clicking on a column header and everything resorts in a flash, you can see how dramatically different rows vary from view to view. It’s also nice because you can do filtering. So without much code you have a filterable, sortable table. It also saves a step of turning the object data into arrays for sorting. I’ve been wanting to add those ascending / descending arrows for a while to our tables so this was a good time for that.

This table will probably become our first stand-alone NewsBeast Labs plugin since we’ve been using it pretty frequently. That’s pretty cool because five months ago we didn’t have any interactive news code and now we’ve done enough projects that we can see what’s worked, what functionality we like and can wrap it all up into something more robust and reusable, which will make our future development that much faster. 

-michael

Notes and images from an ever-growing digital newsroom.

Newsweek & The Daily Beast

Contributors:
Brian Ries & Sam Schlinkert

Formerly:
Michael Keller, Andrew Sprouse, Lynn Maharas, & Clarisa Diaz

view archive