Breakdown of: Mining the Dispatch

What is Mining the Dispatch?

Mining the Dispatch is a project that uses computer software to analyze the Richmond Daily Dispatch, a Richmond newspaper, from Abraham Lincoln’s election to the day the city was evacuated during the Civil War. They did this due to the paper’s archive, which has now gone fully digital, and has over 100,000 articles published during those five years alone. After feeding all of these articles into a program that found how many times a certain topic was mentioned, they used these data points to better understand what life was like in Richmond during this time period.

But Why?

This project, overseen by faculty at the University of Richmond, seeks to show some of the political and cultural changes and consistencies in Richmond during the Civil War, as it was the capital city of the Confederacy. In their introduction page, they have a quote from George Alfred Townsend, a northern journalist, that was written right after the town fell to union forces: “This town is the Rebellion …. Its history is the epitome of the whole contest.” But for all of this historical importance, it is not often mentioned in history classrooms or textbooks according to the people who put together this project. If that is true, this source would be a valuable font of information for both historians and social scientists looking to understand what life was like in the Confederacy’s capital at the height of the Civil War.

How/Methods? 

Mining the Dispatch uses something called Topic Modeling to discover patterns in the data that their software churns out. A topic is a cluster of words that more likely than not will appear together in any given document, and when their statistical software found a cluster, it could be assigned to a given category. For example, if the computer finds words that are descriptors of someone, and most likely with some term for African American and a sum of money, it would assign it to the “Runaway Slave” category. They ended up with forty topics, the resulting graphs for a large number of them are all available on their website.

What Do The Results Look Like?

At first glance, something like this. Which, admittedly, is confusing. I thought the same too on my first look through, but when you click on a link, this happens:

After clicking on a link it brings up a page that looks like this, with the above graph showing the proportion of space taken up by any given topic, and below are some specific articles about whatever topic one has clicked on. It is an easy site to use, with just a few clicks needed to see the full breadth of all of the database’s information. While there is a lot of information to be seen and analyzed here, it is a very text-based project, with minimalist graphs being the only sources of color or imaging. I do not know how this could be improved on, but with some additional visuals or fewer long blocks of texts, it could not only be interesting for the scholar, but the populace in general.

Geo. Alfred Townsend, Campaigns of a Non-Combatant, and his Romaunt Abroad during the War (New York: Blelock and Company, 1866)