Posts tagged "journalism"

Six Months in Review

NewsBeast Labs is roughly six months old and we’ve had a lot of fun. This tumblr has most of our projects for the past few months but there are a bunch from before our launch. Here’s a rough list of projects we’ve done so far.

Legal Experts Decode the Supreme Court’s Obamacare Ruling - Our very first project! We launched it the day we got DocumentCloud, which was also the morning of the Supreme Court ruling on Obamacare. We asked two law professors to make margin notes in the text of the ruling as they were reading it for the first time. Readers could follow along and read experts’ reactions as the conversation was happening.

Digital 100: Who’s Following Whom? - A network visualization of how Newsweek’s list of influential people in the digital space interact with each other on Twitter. 

Obamacare: It’s Cheaper! - I like to call these “Story Visualizations” - visual presentations of stories that could run as a list or as text, but are much more interesting visually. Matt DeLuca and I did a side-by-side on how Obamacare would affect different age groups’ healthcare spending.

2012 Olympics: The Latest Medal Tally - We had a live-updating Olympic Medal Count, (with a snazzy sortable table that I’ve written about before) that I worked on with our awesome intern Sarah Hedgecock. We also did a version of it for our right rail (sidebar).

Interactive Map: London’s Olympic Transformation - The Olympic Park rose from the rust. Sarah and I also did a satellite view before and after interactive that included a bunch of info on the star-chitect buildings.

Interactive Map: The U.S. Shooting Epidemic - Following the Aurora shooting, Brian Abelson and I made an interactive map of multiple-victim shootings since 2005 and asked readers to respond with their memories. We published a selection of the reader responses here. The full spreadsheet list is here.

As Income Inequality Widens, Rich Presidential Candidates Dominate - Lauren Streib and I worked on a chart (she did all the numbers), showing presidential income over the years. I remember this one chart taking four hours from start to finish for some reason…

Big Guns Inside the National Rifle Association Leadership - Who’s leading the NRA? I worked on a project with three colleagues Caitlin Dickson, Eliza Shapiro and Kevin Fallon on the NRA’s leadership. They dug through 990 forms and put together small profiles of the people at the top. We put it together in mosaic-style presentation. Normally this type of story would be a gallery format but since it’s not picture-based, we decided to create something more conducive to reading a lot of text.

SuperPAC App Election Ad Interactive - We partnered with the Super PAC App, an iPhone app that would identify political advertisements on TV and give you information about that group, such as how much money it was spending this election and articles about them. We made a web interface to their data to provide readers with more context for outside spending groups.

Interactive Map: Who’s Protesting Where? - When the Middle East erupted in protests in response to an anti-Muslim video uploaded to YouTube, Eliza Shapiro and I put together a visual guide with information on each protest as well as contextual information on each country. It was an interesting map to built since we had both point and polygon layers to deal with for hover states. As with all of our interactive maps, we used CartoDB.

Obama and Romney’s Bundlers - If bundlers had baseball cards, this is what they’d look like.We took a look at the biggest bundlers for each candidate. Collect ‘em all.

The Rise of the Political Non-Profit - How so-called “Dark Money” was influencing the 2012 election was one of the themes in a three-part series John Avlon and I wrote called the Super PAC Economy. This animated timeline overlays non-profit political expenditures and significant court decisions (Citizens United and lesser-known decisions) that determined what role these groups could play in politics.

The Dark Money Shuffle - Also in that series, we worked with Robert Maguire of the Center for Responsive Politics who had been compiling a database of grants that non-profits gave to each other. For the first time, we diagrammed this opaque world of money transfers that is only visible by manually going through hundreds of IRS forms. Full article

Election Right Rail - Showing the latest polls from battleground states, how those states voted historically, median income, and latest unemployment figures, our politics sidebar was full of context. It no longer lives anywhere on our site but you can see a standalone version how it looked on the eve of the election through the linked title.

Note: We did all of these projects before starting this tumblr. You’ll find write-ups for the projects that follow but if you want to know how we built any of the stuff above, send me a message at @mhkeller.

Debate Dashboard and Bingo - Brian, Sam, Vitaly Korenkov (one of our awesome developers) conceived of a great debate night dashboard. We had a livestream, a live chat with our commentators and a poll from Urtak, which is a polling platform that lets you pose simple yes/no/maybe questions to readers. It also lets readers submit questions they want other people to answer so it’s a good back and forth between questions we’re interested in and what our audience is interested in. We’re often into giving our readers a voice on the site so we liked it a lot. I came in during the last few hours before we were going to go live (a.ka. after all the hard work was done) and added a bingo card. The coolest part about it is the Bingo validation. The card checks how many you have in a row vertically, horizontally, and diagonally and tells you how many you need to win. NewsBeast Labs post.

Ground game: Obama Campaign Opens Up Big Lead in Field Offices - The airwave battle was being covered left and right, but we wanted to know what was happening on the ground. We scraped the two campaigns’ websites to map out their local HQs nationwide and found a big discrepancy between the two camps. In Ohio, for instance, Obama had a presence in so many counties where Romney didn’t that 10 percent of the state’s population lived in a county where the only volunteer center was an Obama HQ.

Technical note: We used CartoDB again for this map and it was a huge help. In the accompanying article, we ran interactive maps of Florida, Ohio, and Virginia. These separate maps required no real extra programming or map making since CartoDB builds your map by querying a database. By setting our map query to ‘SELECT * from national_map WHERE state = FL’  we had a local map in minutes that we could swap out for another state if needed, which indeed ended up happening. NewsBeast Labs post.

Interactive Hate: The Great Obama-Loathing Canon - Matt DeLuca and I teamed up again to solve the perennial problem of how do you present a lot of information to the reader in a way they can digest in bites that make sense. This time, we presented over a hundred anti-Obama books in a mosaic that you can filter down to different subject matters. NewsBeast Labs Post.

HavingTroubleVoting.com - We did an experiment on election day asking our readers, or anyone really, if they were having trouble voting, and if so, what kind of trouble. We plotted the responses on a map below and color-coded the markers based on the type of problem. We partnered with Mother Jones on it to help us go through the responses to find patterns and to contact people to tell their story. Our own reporters used the database in stories about massive lines and machine malfunctions. We’re totally honored and floored when CJR named it No. 2 in their Must-Read Interactives of 2012! More about it in our NewsBeast Labs post.

Election Night Interactive Map and Dashboard - A lot of teamwork went into our election night coverage from the development team, social, design… the list goes on. We took over our home page on election night with video commentary, a live updating tally, a live chat, article updates and more things that you could probably put a “live” prefix in front of. The map lives on in the linked title, a screenshot lives in our NewsBeast Labs post about it.

‘It Was Like a War Zone’: Hurricane-Ravaged Staten Island Reels - In the wake of the trauma caused by Hurricane Sandy, we did a map of Staten Island victims. It shows how many of the fatal tragedies were concentrated on the east side of the island.

Not-So-Super PACs: 2012’s Winners and Losers - DeLuca and I teamed up again to produce this tally of who made good investments this election cycle. There’s a long post about it, including some failed versions in our NewsBeast Labs post

Interactive Holiday Gift Guide - Lizzie Crocker, Isabel Wilkinson and I help you find out what sub-culture your friends might belong to in this gift guide flow chart.

Own a Gun? Tell Us Why? - December brought another terrible shooting and has caused much thought over the state of gun laws. We wanted to hear from rational people on both sides of the debate by lettings readers complete the sentences, “I own a gun because…” or “I don’t own a gun because…”. In three days, we had over 1,300 responses that represented very civil remarks from each group, for the most part. We analyzed the responses and did a state-by-state breakdown of the common themes. We used some interesting algorithmic clustering to find these patterns so expect a write-up soon. For now, read the post on how the project was born and how we collected the responses.

Our last Wednesday Workshop focused, in part, on ways to get readers more involved in our stories. When news of last week’s awful shooting reached us, we wanted to open up discussion on the role of guns in America. On our Tumblr we asked readers how the shooting should be covered and many requested we steer clear of the politics and instead opt for a genuine discussion on gun control.
Gun control is a complicated issue in this country and nuanced issues can be at odds with the tools of data visualization. That is to say, data visualization and data reporting are often marked by being extremely comprehensive and boiling that comprehensiveness into one easily understandable image, graph, or layout. Doing anything comprehensive on an issue as complex as guns in our society, on deadline no less, would be tricky, and we’re not ones to put data out there that’s misleading or inconclusive.
But the other tool of digital journalism is being able to present a great deal of information in one place, which does work for a nuanced subject. We wanted to engage our readers to tell the story of guns in America in a way that showed the issue’s complexity. We posed the question as “Why do you own a gun?” or “Why don’t you own a gun?” On our site we, we set up two forms that let readers easily complete the sentence “I own a gun because…” or “I don’t own a gun because…” and displayed their responses for readers to sift through.
It’s like the digital equivalent of Man on the Street reporting, where you go and ask people on the street their opinions on an issue in the news and write up their quotes in an article. Let’s call this a Man on the Internet story, or to be gender neutral, Person on the Internet (Internet Vox Pop maybe? I’m open to suggestions).
We published the article Monday evening and less than 24 hours later we have over 900 responses — over 500 from gun owners and over 400 from non-gun owners. We have some thoughts on how the two sides explain their position but, for now, we’ll let you read through and absorb it on your own.
We’re collecting and categorizing the responses, so look for that article on the Beast later in the week.
Under the hood
We used a customized Google Form to handle the response collections. This is a nice tutorial on how to embed Google Forms into your site with custom styles and functionality. 
We’ve used custom Google Forms before, on our other shooting project actually, for a newsapp that lets readers put in their address and it finds news accounts of multiple-victim shootings near them. A Google Form then asks what they remember about the incident and collects their responses in a spreadsheet. We published some of the most moving responses that I think is worth a read.
For this project, though, I was running into trouble putting in two custom forms on one page. Since I only had a day to build this, I ended up sequestering the two forms to separate HTMl pages and iframe-ing them into my main page. This was nice because it ensured the two forms didn’t interfere with each other and since the pages were all on the same domain, I didn’t have any cross-origin issues and didn’t loose any functionaity — when you submit one form, that action bubbles up to it the parent frame and grays out the other form.
I originally wanted to do something more animated similar to this seminal piece of crowdsourced dataviz from 2008. I like how its animation gives the project energy but it comes at the cost of not being able to scroll through the responses on your own. After some thinking, I couldn’t figure out a way to have both a sit-back-and-let-the-responses-flow experience and a I-want-to-dive-into-these-responses-and-scroll-through-them-all experience. The latter is obviously the more useful for the reader, so I went with that. The election interactive is also a bit different from this since most of the emotions on each line are of the same category, so it’s not really hiding anything by not letting you scroll. For our project, each response brings its own nuance to the debate so you don’t want to hide any of them. If you have any thoughts on how to improve the presentation, I’m at @mhkeller.
Brian had the great idea that we let this conversation be medium agnostic. So in the story dek we let people know they can continue the conversation on Twitter with the hashtags #IOwnAGunBecause or #IDontOwnAGunBecause. I built some hooks into our Underscore.js templates that let us add selected tweets to our spreadsheet and display them with a Twitter icon and a link to the original tweet. That way we could pull in interesting responses from elsewhere and flag them as such. You can look at the formatHelpers object in app.js to see how it checks for content in the Twitter column and adds the image and link if it finds something.
As I’ve written, I’m a big fan of Miso’s Dataset.js, and that’s what we’re using here to pull the responses in from our Google Form Spreadsheet. Contrary to my previous post, this app does work off of a live Google Doc. I know, blasphemous. For a few workflow reasons we weren’t able to have a script download our spreadsheets and put them on a server like we did for HavingTroubleVoting.com where we had both rate-limiting and privacy issues.
That being said, we have been very closely monitoring the app to make sure it doesn’t get rate-limited and it’s been fine so far. I have a few lines commented out in the code that point to where we’d put a local CSV of the responses, so if the app went down, it would be back up in a minute or so. We also made sure not to ask for any identifying information so we had no privacy concerns. Now that we have close to a thousand responses, though, we might switch to local files so that the page loads faster. If we could have set it up to download automatically, however, that would have been our first choice.
One thing I added yesterday evening after we started getting a lot of comments was a way to filter by state. A lot of content can be overwhelming, so the more options you can give readers to drill down to a subset that might be more relevant to them, the more manageable the experience is and hopefully more engaging and memorable.

-michael keller

Our last Wednesday Workshop focused, in part, on ways to get readers more involved in our stories. When news of last week’s awful shooting reached us, we wanted to open up discussion on the role of guns in America. On our Tumblr we asked readers how the shooting should be covered and many requested we steer clear of the politics and instead opt for a genuine discussion on gun control.

Gun control is a complicated issue in this country and nuanced issues can be at odds with the tools of data visualization. That is to say, data visualization and data reporting are often marked by being extremely comprehensive and boiling that comprehensiveness into one easily understandable image, graph, or layout. Doing anything comprehensive on an issue as complex as guns in our society, on deadline no less, would be tricky, and we’re not ones to put data out there that’s misleading or inconclusive.

But the other tool of digital journalism is being able to present a great deal of information in one place, which does work for a nuanced subject. We wanted to engage our readers to tell the story of guns in America in a way that showed the issue’s complexity. We posed the question as “Why do you own a gun?” or “Why don’t you own a gun?” On our site we, we set up two forms that let readers easily complete the sentence “I own a gun because…” or “I don’t own a gun because…” and displayed their responses for readers to sift through.

It’s like the digital equivalent of Man on the Street reporting, where you go and ask people on the street their opinions on an issue in the news and write up their quotes in an article. Let’s call this a Man on the Internet story, or to be gender neutral, Person on the Internet (Internet Vox Pop maybe? I’m open to suggestions).

We published the article Monday evening and less than 24 hours later we have over 900 responses — over 500 from gun owners and over 400 from non-gun owners. We have some thoughts on how the two sides explain their position but, for now, we’ll let you read through and absorb it on your own.

We’re collecting and categorizing the responses, so look for that article on the Beast later in the week.

Under the hood

We used a customized Google Form to handle the response collections. This is a nice tutorial on how to embed Google Forms into your site with custom styles and functionality. 

We’ve used custom Google Forms before, on our other shooting project actually, for a newsapp that lets readers put in their address and it finds news accounts of multiple-victim shootings near them. A Google Form then asks what they remember about the incident and collects their responses in a spreadsheet. We published some of the most moving responses that I think is worth a read.

For this project, though, I was running into trouble putting in two custom forms on one page. Since I only had a day to build this, I ended up sequestering the two forms to separate HTMl pages and iframe-ing them into my main page. This was nice because it ensured the two forms didn’t interfere with each other and since the pages were all on the same domain, I didn’t have any cross-origin issues and didn’t loose any functionaity — when you submit one form, that action bubbles up to it the parent frame and grays out the other form.

I originally wanted to do something more animated similar to this seminal piece of crowdsourced dataviz from 2008. I like how its animation gives the project energy but it comes at the cost of not being able to scroll through the responses on your own. After some thinking, I couldn’t figure out a way to have both a sit-back-and-let-the-responses-flow experience and a I-want-to-dive-into-these-responses-and-scroll-through-them-all experience. The latter is obviously the more useful for the reader, so I went with that. The election interactive is also a bit different from this since most of the emotions on each line are of the same category, so it’s not really hiding anything by not letting you scroll. For our project, each response brings its own nuance to the debate so you don’t want to hide any of them. If you have any thoughts on how to improve the presentation, I’m at @mhkeller.

Brian had the great idea that we let this conversation be medium agnostic. So in the story dek we let people know they can continue the conversation on Twitter with the hashtags #IOwnAGunBecause or #IDontOwnAGunBecause. I built some hooks into our Underscore.js templates that let us add selected tweets to our spreadsheet and display them with a Twitter icon and a link to the original tweet. That way we could pull in interesting responses from elsewhere and flag them as such. You can look at the formatHelpers object in app.js to see how it checks for content in the Twitter column and adds the image and link if it finds something.

As I’ve written, I’m a big fan of Miso’s Dataset.js, and that’s what we’re using here to pull the responses in from our Google Form Spreadsheet. Contrary to my previous post, this app does work off of a live Google Doc. I know, blasphemous. For a few workflow reasons we weren’t able to have a script download our spreadsheets and put them on a server like we did for HavingTroubleVoting.com where we had both rate-limiting and privacy issues.

That being said, we have been very closely monitoring the app to make sure it doesn’t get rate-limited and it’s been fine so far. I have a few lines commented out in the code that point to where we’d put a local CSV of the responses, so if the app went down, it would be back up in a minute or so. We also made sure not to ask for any identifying information so we had no privacy concerns. Now that we have close to a thousand responses, though, we might switch to local files so that the page loads faster. If we could have set it up to download automatically, however, that would have been our first choice.

One thing I added yesterday evening after we started getting a lot of comments was a way to filter by state. A lot of content can be overwhelming, so the more options you can give readers to drill down to a subset that might be more relevant to them, the more manageable the experience is and hopefully more engaging and memorable.

-michael keller

Tracking the presidential groundgame
As the two presidential campaigns launch into their final throws, we wanted to see who had the biggest footprint of campaign headquarters across the country. The map above ran with the resulting story showing Obama with a large advantage, especially in swing states. We thought getting from idea to map / chart would be as easy as checking the candidates’ websites or calling their press offices to request a full list. Not quite.
Finding Romney’s offices
Governor Romney’s campaign didn’t get back to our requests so we dug into their website. Romney only had volunteer offices in 16 states, each of which has a web page and a map showing office locations. Here’s Ohio’s. The data that powers the map is visible if you look at the Javascript files that power it, so we copied the data sixteen states into one file and saved it. They were in JSON file format — but more on that later.
Finding Obamas’s offices
President Obama has offices in many more states than Romney so going state by state is more of a hassle and his campaign’s website doesn’t have the same convenient state-by-state maps. Instead, you input your zip code and it gives you a map of all locations within forty miles of you. 
Needless to say, getting a comprehensive list by canvassing the country in 40 mile chunks was out of the question. Digging again into the javascript that powers the website (aside: the Obama website has some slick ASCII art in their source code), their volunteer center finder works by sending out a query that looks like this http://offices.barackobama.com/postal_code/10011/ with your zip code making up the those last five digits. If you put that in your browser and hit return you’ll see the centers closest to the Daily Beast offices in Manhattan.
It automatically looks within 40 miles of you but if you add “?distance=1000” at that url, it will search within however many miles you want. 
So if you add the zip code for the rough geographic center of the US, Lebanon, Kansas and a distance of 10,000 miles you get something that looks like this:
http://offices.barackobama.com/postal_code/66952?distance=10000
which will give you a list, in JSON again, of every volunteer center in the country.
This list returns a lot of information we didn’t immediately need, though, like hours of operation and email address contacts. We wrote an R script that took just the fields the we needed for our map, mostly the latitude and longitude, city, and state and made a clean spreadsheet in CSV format.
From there we were able to do counts by state and plot them on a map. Once we had the final count, the Obama campaign confirmed that our number was in the right ballpark.
BONUS: Obama’s offices all include a Foursquare ID. If you want to play around with them, here’s the data, right click and choose “Save Links As…” if it doesn’t download automatically. Some interesting things could be finding the mayors of the obama campaign offices, which location has the most check-ins, check-ins over time in different locations etc. — let us know at newsbeastlabs@gmail.com and we’ll put up your stuff.
Here’s a spreadsheet of both candidates’ headquarters, as well.
-michael

Tracking the presidential groundgame

As the two presidential campaigns launch into their final throws, we wanted to see who had the biggest footprint of campaign headquarters across the country. The map above ran with the resulting story showing Obama with a large advantage, especially in swing states. We thought getting from idea to map / chart would be as easy as checking the candidates’ websites or calling their press offices to request a full list. Not quite.

Finding Romney’s offices

Governor Romney’s campaign didn’t get back to our requests so we dug into their website. Romney only had volunteer offices in 16 states, each of which has a web page and a map showing office locations. Here’s Ohio’s. The data that powers the map is visible if you look at the Javascript files that power it, so we copied the data sixteen states into one file and saved it. They were in JSON file format — but more on that later.

Finding Obamas’s offices

President Obama has offices in many more states than Romney so going state by state is more of a hassle and his campaign’s website doesn’t have the same convenient state-by-state maps. Instead, you input your zip code and it gives you a map of all locations within forty miles of you. 

Needless to say, getting a comprehensive list by canvassing the country in 40 mile chunks was out of the question. Digging again into the javascript that powers the website (aside: the Obama website has some slick ASCII art in their source code), their volunteer center finder works by sending out a query that looks like this http://offices.barackobama.com/postal_code/10011/ with your zip code making up the those last five digits. If you put that in your browser and hit return you’ll see the centers closest to the Daily Beast offices in Manhattan.

It automatically looks within 40 miles of you but if you add “?distance=1000” at that url, it will search within however many miles you want. 

So if you add the zip code for the rough geographic center of the US, Lebanon, Kansas and a distance of 10,000 miles you get something that looks like this:

http://offices.barackobama.com/postal_code/66952?distance=10000

which will give you a list, in JSON again, of every volunteer center in the country.

This list returns a lot of information we didn’t immediately need, though, like hours of operation and email address contacts. We wrote an R script that took just the fields the we needed for our map, mostly the latitude and longitude, city, and state and made a clean spreadsheet in CSV format.

From there we were able to do counts by state and plot them on a map. Once we had the final count, the Obama campaign confirmed that our number was in the right ballpark.

BONUS: Obama’s offices all include a Foursquare ID. If you want to play around with them, here’s the data, right click and choose “Save Links As…” if it doesn’t download automatically. Some interesting things could be finding the mayors of the obama campaign offices, which location has the most check-ins, check-ins over time in different locations etc. — let us know at newsbeastlabs@gmail.com and we’ll put up your stuff.

Here’s a spreadsheet of both candidates’ headquarters, as well.

-michael

Covering the debates

When we began thinking about how we wanted to cover the presidential debates live, we knew we needed more than just a livestream. No one just watches a live event anymore— they want real-time analysis from experts, as well as a way to voice their opinions and see the opinions of other viewers. So we built our Debate Dashboard to provide all of these services. 

We used a combination of elements to get the right mix: the livestream (of course), a live chat, an Urtak poll, and bingo.

Live chat 

We repurposed our new commenting platform, Livefyre, to host the chat. The platform also allows us to pull in tweets from our political commentators.  

Polling

We teamed up with the New York-bred Q&A application Urtak to ask our readers questions that pertained to the debate topics in real-time, which collects responses in yes/no/don’t care answers and displays them in a stream of pie charts. Through the 56 questions about policy and the candidates—the majority of which were submitted directly by readers within the app—we got nearly 13,000 responses.

Bingo

This was a last minute addition that we put together a couple hours before the debate but proved to be super popular. Our politics team contributed keywords for the cards, which then get shuffled and delt out to readers watching the debate live.

Under the hood: We used Miso’s Dataset to import the list of keywords from a Google Spreadsheet so we could work on the code while they were populating the app with terms. It’s a workflow that we’ve come to use for most of our projects. Expect a post about it soon…

-Brian, Michael & Sam

Welcome to NewsBeast Labs

NewsBeast Labs is a new staff blog that chronicles the behind the scenes of creating digital journalism at Newsweek & The Daily Beast. The main writers, for now, include in alphabetical order Senior Data Reporter Michael Keller, Senior Editor of Social Media Brian Ries, Deputy Social Media Editor & Audience Coordinator Sam Schlinkert, and Lead Developer Andrew Sprouse. 

What’s “digital journalism”? Something loosely defined as including an interesting audience engagement strategy, computer-assisted reporting, data visualization or additions to our website that it easier to use or more engaging. On this tumblr, we’ll talk about how the technology we used helped us tell the story we were after and what decisions went into displaying a story in a particular way.

We welcome feedback so drop us a line sometime, newsbeastlabs@gmail.com.

Your NB Labs team,

Andrew, Brian, Michael, & Sam

Notes and images from an ever-growing digital newsroom.

Newsweek & The Daily Beast

Contributors:
Brian Ries & Sam Schlinkert

Formerly:
Michael Keller, Andrew Sprouse, Lynn Maharas, & Clarisa Diaz

view archive