NewsBeast Labs

Oh hey! Adweek’s Charlie Warzel wrote a story about our team!

For the holidays, we wanted to make the normal gift guide idea a little more interesting. The answer was a flow chart that narrows down the type of person you’re shopping for and then suggests appropriate gifts for their particular sub-culture. Lizzie Crocker and Isabel Wilkinson did a great job coming up with the categories like (Nostalgic Outdoorsman, The Closet ‘50 Shades’ Fan, Hipster Techie) and then Lizzie and I (Michael) put our heads together to make a (hopefully) witty flowchart. Our photo department headed by Marcia Allert was also a huge help.

There’s nothing fancy under the hood. The only semi-trick was our Daily Beast font family Titling isn’t always too legible at smaller sizes on the web. To fix that, we made the blue circles in Illustrator and exported them as PNGs. To do the hovers, you duplicate the image below your main image like this and make sure your image container is only tall enough to show one at a time: 

To do the hover then, your CSS is something like .img-class:hover{ background-position: 0 -102px;} The benefit to this is you don’t load a second picture when you hover so there’s no delay. This is a pretty standard way of doing this so it’s nothing revolutionary but for some reason you still see a lot of sites that have delays on their image hovers that would be better off using this technique.

Why not make the flowchart interactive too?

We decided in favor of a static image for the flowchart as opposed to something interactive since I think the wittiness of flowcharts comes across in seeing how the possibilities flow from one another and the different results in choosing one adventure over another. Also, at the sake of sounding blasphemous, interactivity can be thought of as a last resort only when you can’t fit everything onto the screen at once, or doing so would work against a focused narrative. It’s much easier for readers to scan a page full of information and see how these options unfold, in my view, than it is for them to click 10+ times and only see the options that stem from their responses. 

To continue the trend to show some shots from our failures, here’s a screenshot of the drafting process for the flowchart. We really wanted to work mud into it somewhere since it could have split off nicely with Adrenaline Junkie or Wee Ones (since kids love mud), but it didn’t quite work out. The second one is what we settled on.

-michael

We recently gave a lesson to some editors and reporters about how to use TweetDeck, the awesome Twitter-streaming product. We thought we’d publish our step-by-step walk-through here, both so our reporters could bookmark it for future reference and to show you all what we’re up to. Obviously it’s just the basics detailed below, aimed at first-time TweetDeck users. Maybe we’ll have a more advanced class down the road.

What is TweetDeck, and why might I want to use it?

If you have a Twitter account already, you probably view and compose tweets through twitter.com. Tweetdeck is a separate Twitter product that offers a few advantages to simply using twitter.com. The most obvious advantage is that streams tweets in near-real-time, as opposed to twitter.com, which only tells you when new tweets are available to be viewed (“12 new Tweets”). The second advantage is that you can view multiple columns of tweets, rather than just your own timeline. Other columns can be lists, searches, even other users’ timelines. We’ll get to all that below. 

How to Setup TweetDeck:

First, you’ll have to decide if you want to use the TweetDeck application for the desktop or TweetDeck for the web (i.e. in-browser). Tweetdeck for the web will live in a browser tab. The desktop application will be a separate application. Ries and I both use the desktop application. It’s not a huge deal which you choose— you can switch back and forth very easily.

To setup the desktop app: Visit tweetdeck.com and click the silver download button on the right. 

To use TweetDeck in your browser: visit https://web.tweetdeck.com/

No matter which form (desktop or web browser) of TweetDeck you’ve chosen, the instructions from here on out are the same.

When you visit https://web.tweetdeck.com/ or open the application for the first time, you’ll be asked to sign in with your TweetDeck account. Note that this is different from your Twitter account. So you’ll need to create a TweetDeck account. But don’t panic. You’ll only have to log-in to Tweetdeck once and it will keep you logged in (almost) forever (click the checkbox that says “keep me logged in”). However, just this first time, you’ll need to create an account. You can use the same email and password as you do for your Twitter account.

You’ll now be asked to add your Twitter Account— here is where you log in as you would on twitter.com. Remember, Tweetdeck can handle multiple Twitter accounts, but most of you will just have your personal account. 

OK, it’s setup. Now what?

Now that Tweetdeck is open, you’ll want to make some columns. Your far left column defaults to “Home,” which you’ll probably want to keep there as it is all the accounts you follow. This is the column you see at twitter.com when you’re logged in.

A good second column is “Interactions”— which you may know as the “Connect” screen on Twitter.com, which shows your @replies, etc.. To add a column for your interactions, click the circular button with a plus sign in the middle to add a new column. Then click “interactions” and select your account.  

You may also want to add lists, like say, the Cheat Sheet source list we created. This is a bit trickier, but will become more intuitive. 

How to Add a List as a New Column in Tweetdeck:

1. Go to twitter.com in your browser. Log in to your personal Twitter account.

2. Find The Daily Beast on Twitter (https://twitter.com/thedailybeast). Click the “Lists” link on the left. Scroll down and click on “CheatSheet”. You should find yourself at: https://twitter.com/thedailybeast/cheatsheet. 

3. Click the Subscribe button on the left. 

4. Go back to Tweetdeck. Click the plus button to add a new column. Go to “Lists”. Find and click on “CheatSheet” on the left and then click the “Add column” button. 

Note: You can subscribe to as many lists as you like, and they obviously can be lists made by other users (not just @thedailybeast). You can also create your own lists. 

The third type of column you may want to have is a simple search column. Say you want to see what people are tweeting about Sandy. The best way to do this is to follow the hashtag #Sandy. Again, click the add column button, now go to search and type in #Sandy. Click add column, and you should be all set. 

Organizing Your Columns

The TweetDeck desktop application can display any number of columns from 3 to as many as you can fit on your monitor(s). To re-order your columns, click the “Columns” bar in the top middle of the application. You’ll see the title of your columns displayed in order. You can drag and drop them in to a desired order by clicking and holding on the 6 white squares on the right side of any column box. 

Again, if you need help with any of this let us know.

-Sam

A defining characteristic of this election cycle was Super PACs and the hundreds of millions of dollars outside groups were spending to influence races. Now that it’s all over, we wanted to see which outside groups spent their money on succssful races and which did not. The result was our interactive Not-So-Super PACs: 2012’s Winners and Losers.

Super PACs abounded this cycle. So instead of trying to document and display all of them, we focused the narrative on how well the biggest spenders and their donors fared. To execute it, we used Center for Responsive Politics anaylsis of FEC data to find how much each PAC had spent so far in each race and then manually went through and coded each race whether the outcome was in line with or against the PAC’s interest. Then we added everything up.

Visualizing it 

This idea went through a few iterations before settling on what you see above. For a while, we’ve been wanting to use a tower graphic template - one of those vertical scroll layouts with a sticky table of contents - that I built a couple of months ago but it never seems to work out. This time, after thinking about all of the detail we wanted to display we thought bigger.

If you’re trying to visualize money flows, Sankey lines are a go-to. ProPublica did a great one showing overlapping Super PAC expenditures and you see them as flat graphics too. They show direcionality and volume = great for money.

Getting the right data

Money was flowing from donors to PACs and then to races, so we used the JSON structure that D3 lays out for its network layouts (and Sankey) visualizations. You have a list of nodes (People and PACs) and a list of node to node links (X person gave $Y to Z PAC). We were working collaboratively in Google Docs so were able to do some formulas that would print out or data structure in JSON as we were editing the document. Very handy in case you need to correct any numbers or name spellings.

Our D3 visualization was a failure.

Here’s a link to the interactive version (yes, it’s in the “failures” folder). As you can see, there were too many races to fit on the screen and the dollar amounts in some races were so high that they dwarfed everything else. So showing each race in the Sankey was out. 

This led to Sankey Idea #2.

We connected photos of the donors to the PACs, showed the percentage of succesful funds, and then put the races in a table down below. The photos were very useful because you can quickly understand that money is coming from a person and going somewhere. If we just had text, I think, without photos, it would be less clear and have less of a personality. Someone remarked that the lines almost form bodies and arms that reach out to touch Super PACs. It’s interesting to see visualized data combined with photography work out to tell a story like that.

Under the hood

We used Raphael to draw the lines, which was an improvement from D3 since we do indeed support Internet Explorer 8. We tweaked Al Shaw’s Sankey line from Tom Counsell’s Sankey library to make them span vertically instead of left to right. Here’s a jsFiddle of the code to draw the line.

The table uses Isotope.js for its animated sorting, which is snazzy but I also think does help make tabular data more understandable. Instead of clicking on a column header and everything resorts in a flash, you can see how dramatically different rows vary from view to view. It’s also nice because you can do filtering. So without much code you have a filterable, sortable table. It also saves a step of turning the object data into arrays for sorting. I’ve been wanting to add those ascending / descending arrows for a while to our tables so this was a good time for that.

This table will probably become our first stand-alone NewsBeast Labs plugin since we’ve been using it pretty frequently. That’s pretty cool because five months ago we didn’t have any interactive news code and now we’ve done enough projects that we can see what’s worked, what functionality we like and can wrap it all up into something more robust and reusable, which will make our future development that much faster. 

-michael

latimes:

That was fun.

News GIFs! We like this.

A couple of weeks ago we had a small project that uses a stack we’ve come to like. The project was an interactive collage displaying just under a hundred books that take a somewhat negative view of the president, to put it lightly. With so many books we wanted a way for people to filter the list to see an amount that was more easily digestible. Matthew DeLuca separated them out into helpful categories like “Economy” and “Dangerous Radical”. You can hover over them to see an excerpt if you’re so inclined. We hope you like it.

Under the hood

We used the Isotope library, which we’ve used before in both similar layouts and with sortable tables, to handle the animation and filtering. We used the Miso Dataset library (more on that in a forthcoming post) to assemble the data in a spreadsheet so multiple people could work on it simultaneously and editors could easily access the copy. When everything was ready we downloaded it to a local csv and went from there. It’s a workflow that has worked well on projects like these.

-michael

HavingTroubleVoting.com is one of our latest projects to come out of our election coverage. It was born when fellow reporter, Allison Yarrow, whom I (Michael) sit across from, was talking about how we should do something to make voting issues fascinatingly interesting and digital. A Google Form, some reading about issues facing voters, and some CSS media calls later, we had a mobile-friendly submission form tied to a Leaflet map. 

We partnered with Mother Jones who had a similar project since the task of verifying so many reports (at last count ~700) is quite a process. As you can see, we took a different approach with this one: the site has a separate URL and we built it completely outside of our CMS. We took inspiration from the question-URL-based Guardian US project IsBarackObamaThePresident.com. I’m a fan because it lends itself to be more creative and have a longer shelf-life than a story that goes into a regular CMS, has a timestamp and all that. It also gives itself a catchy url that’s easy to remember. 

Under the hood

We ran into some issues being rate-limited using the results of our  Google Form as a data source so I wrote in some middleware with R that pulls down the form, does the geocoding for new entries (to avoid geocoding rate-limits) and let me put it on S3 easily.

Originally we were geo-locating the submissions based on browser location (since the main thinking was for mobile) but the percentage of successful tags was lower than expected. We also decided we would rather geocode based on the address people wrote in as opposed to their browser. I used a version of this geocoder to grab new entries every hour or so and make a new file to upload. 

We’re still going through the results and collecting more, if you have any, let us know! We’re pretty much always online at newsbeastlabs@gmail.com.

-michael

For our election coverage we wanted to show a mix of commentary and live information on the night of. Live commentary means video as well as a chat component but the hardest part was delivering live data results without having the page to refresh, which would interrupt the
video and be entirely too slow.

To solve that problem, we’re using CartoDB’s real-time election map which lets us make changes to our data on the backend and have the results fade in as results are pushed to our readers. We added a few details such as number tallies that tick up one by one so readers can see changes when a lot of states are called at once. We also made the numbers of recently called states to green so that you can see at a glance which states were newly called this round.

As a contingency we also have some views in place that will kick in automatically in case a state is marked as “Disputed” or if it ends in a tie.

After tonight, the map will live on here. We hope you like it!

-michael

Tracking the presidential groundgame

As the two presidential campaigns launch into their final throws, we wanted to see who had the biggest footprint of campaign headquarters across the country. The map above ran with the resulting story showing Obama with a large advantage, especially in swing states. We thought getting from idea to map / chart would be as easy as checking the candidates’ websites or calling their press offices to request a full list. Not quite.

Finding Romney’s offices

Governor Romney’s campaign didn’t get back to our requests so we dug into their website. Romney only had volunteer offices in 16 states, each of which has a web page and a map showing office locations. Here’s Ohio’s. The data that powers the map is visible if you look at the Javascript files that power it, so we copied the data sixteen states into one file and saved it. They were in JSON file format — but more on that later.

Finding Obamas’s offices

President Obama has offices in many more states than Romney so going state by state is more of a hassle and his campaign’s website doesn’t have the same convenient state-by-state maps. Instead, you input your zip code and it gives you a map of all locations within forty miles of you. 

Needless to say, getting a comprehensive list by canvassing the country in 40 mile chunks was out of the question. Digging again into the javascript that powers the website (aside: the Obama website has some slick ASCII art in their source code), their volunteer center finder works by sending out a query that looks like this http://offices.barackobama.com/postal_code/10011/ with your zip code making up the those last five digits. If you put that in your browser and hit return you’ll see the centers closest to the Daily Beast offices in Manhattan.

It automatically looks within 40 miles of you but if you add “?distance=1000” at that url, it will search within however many miles you want. 

So if you add the zip code for the rough geographic center of the US, Lebanon, Kansas and a distance of 10,000 miles you get something that looks like this:

http://offices.barackobama.com/postal_code/66952?distance=10000

which will give you a list, in JSON again, of every volunteer center in the country.

This list returns a lot of information we didn’t immediately need, though, like hours of operation and email address contacts. We wrote an R script that took just the fields the we needed for our map, mostly the latitude and longitude, city, and state and made a clean spreadsheet in CSV format.

From there we were able to do counts by state and plot them on a map. Once we had the final count, the Obama campaign confirmed that our number was in the right ballpark.

BONUS: Obama’s offices all include a Foursquare ID. If you want to play around with them, here’s the data, right click and choose “Save Links As…” if it doesn’t download automatically. Some interesting things could be finding the mayors of the obama campaign offices, which location has the most check-ins, check-ins over time in different locations etc. — let us know at newsbeastlabs@gmail.com and we’ll put up your stuff.

Here’s a spreadsheet of both candidates’ headquarters, as well.

-michael