To begin our project, we manually downloaded a representative sample of PDFs of The Carletonian front pages since 1893. However, we needed a streamlined way to extract the text from the PDFs for use in textual analysis software like Voyant. We reached out to reference librarian Sarah Calhoun and she initially suggested the low-tech method of copying and pasting text from each PDF but also referred us to an expert named David Bliss who might have a better solution.
During class on Thursday, we met with David, Carleton’s Digital Archivist. He was a great resource given his intimate knowledge of the Carletonian Collection in the archives, having worked with them more than nearly anyone else on campus. Over email, he told us about two powerful Python scripts that could be useful for us: one that would combine all of our PDFs in a given year into one file and another script that would extract the text layer of the PDFs into a plaintext file in the same location. Under his guidance, our group member Maddy deployed the Python scripts and we were able to begin analyzing the text files.
We used Voyant to upload our data and find the most common words in our text files. Next, we downloaded a CSV file of that data so that we could have a subset of data to work with that was more meaningful to the overall point of our project. Finally, we uploaded that CSV file to Flourish to create a prettier and more organized visualization of our data.
The data visualizations present in this post are more exploratory in nature. We were interested in showing trends in word usage over the years (1893-2019) as well as seeking out patterns that will help us develop questions and conclusions for the final product.
We needed to make several changes to our CSV file in Excel in order to utilize Flourish to visualize our data. One change that we made was multiplying all of our values by 1000 to better showcase the differences in frequencies per year. Additionally, this prevented Flourish from rounding some of our values to zero, leading to increased clarity in the graphs.
Ultimately, we’re looking forward to exploring other data visualization methods as well as putting together our final website. Below are some additional line graphs we created using Flourish using the words committee, class, campus, and president.