The project I chose to analyze was the Mining the Dispatch Project, which is a project analyzing the frequency of various topics that appeared in articles from the Civil War era in Richmond, Virginia. The articles are all from Richmond’s Daily Dispatch newspaper from November 1860 to April 1865, and the 40 topics this project covers includes fugitive slave ads, secession, casualties, legislature, and more. The site, isn’t the most interactive site, but contains plenty of information:
Breaking Down “The Black Box”
In understanding the “black box” of this digital project, it’s important to understand the sources, processes, and presentation. It appears that the author of this project, Robert K. Nelson, Director of the Digital Scholarship Lab at the University of Richmond, used 2 main sources to gather all of the data. This includes the Daily Dispatch, a digital archive of the aforementioned newspaper, and the Mallet software package, which has the ability to analyze large amounts of text. Additionally, the project benefitted from the ideas, insights, and comments of the Digital Scholarship Lab staff, Scott Nesbit, Nate Ayers, and Nathan Altice, as well as Edward L. Ayers and Chris Kemp.
For the process of this project, Nelson utilized topic modeling to pull all of the articles and sort the data into topics. Topic modeling uses statistical techniques to categorize individual texts and to discover categories, topics, and patterns that we might not be aware of in those texts. A topic modeling program generates however many topics the programmer wants, and then the program is able to generate the topics based on patterns in the statistical algorithm. The program is also able to generate proportions of what topics appear in each and every document, since each document likely has many different topics it falls under.
The main presentation of the website is different graphs for each topic that show the popularity of the topic over the course of the Civil War. This website is not as dynamic as others, but users can choose popularity based on the count of how many articles contain the topic, or the proportion of print space the topic takes up. Users can click on each topic, as well as a link to a transcript of articles that fall under the selected topic. The “topics” page looks like this:
Additional Questions
Regarding additional questions that relate to this website, it’s clear that history is an academic field in which this project is in conversation with, since historians could use this data to better understand what Richmond was like during the Civil War. Statistics could also apply here, since we could analyze the correlation between different topics, or see if any events had a statistically significant effect on a topic. Adding on to this, the site does not make an argument on it’s own; interpretation is left up to the viewer. This website aims to provide an unbiased look on what life was like in Richmond, and let users explore the data themselves.
After breaking this project down, I’m now left to wonder: how do other important Civil War cities compare to Richmond? While it’s possible that the data is not as full in other cities, it would be interesting to compare the prevalence of various topics in cities across the US to see if topics differed within different locations of the country.
1 thought on “Analyzing the Mining the Dispatch Project”
Leave a Reply
This site uses Akismet to reduce spam. Learn how your comment data is processed.
I love how much you went into details about statistical methods used to generate topics of interest for the programmer. Also, with how extensive Mining the Dispatch is, you did a really good job explaining. One thing I think that I would change about your analysis is the pictures in the beginning are a little small and I have trouble figuring out what they are showing. Maybe have a link to where you took the screen shots. Otherwise great job I loved reading your analysis oof this project!