Who Said What at Last Week's Presidential Debate?

-





The final presidential debate took place last Thursday October 22nd 2020 between Donald Trump and Joe Biden. The transcript for the debate can be found here:

Using a web-scraping tool, I was able to access the digital transcript and isolate each candidates' speaking portions in order to run some exploratory text analysis. Disclaimer- this post will not include any political commentary. Rather, I am interested in uncovering any patterns that occurred in the transcript. 

 First off, how many times did each candidate speak? 

Donald Trump, despite the first ever introduction of a mute button during a presidenti, was credited for speaking 62 more times than his opponent. Next, what words were most frequently used by each candidate? With minimal processing, below are the results for the candidates:




Visualized a different way, we can clearly see some significant overlap between the two:



These top word breakdowns get a little more interesting when we add some of these frequently occurring top words to our 'stop' list (words that are removed from consideration due to not being informative). For example, if we remove 'going', 'said', and 'people' from the analysis, we get the updated top word breakdowns here:


I'll let the reader come to their own conclusions given these updated top words form each candidate. I also ran this type of text processing for the moderator, and her top word by a long shot was "President". 

Another common text analysis technique is word count. Both overall total number of words and number of unique words:



The first concept that most people associate with natural language processing is sentiment analysis. Using the sentiment calculator in the TextBlob package, the sentiment of each candidate's responses as well as the calculated subjectivity score can be visualized below:




From these figures, it would appear that Joe Biden had a more uplifting message in his communications that evening. However, these sentiment scores should always be taken with a grain of salt because a "positive" sentiment in this sense (a political debate) may be much different than what is considered positive in the corpus on which this sentiment model was trained. 

Now, with some exploratory data analysis done, I would like the build a classifier. This classifier's inputs would be a quote from the debate, and the output would be that model's prediction of who said that quote. 

In other words, Can we Build a Model that predicts which Candidate said a specific quote? To create this model, we will use a count vectorizer to build a data frame of word frequencies. I'll spare readers the details of how exactly this process works, unless you'd like to check out the code here. 

I used the count vectorizer technique to fit and assess 4 different models. Their respective accuracy scores are plotted below:



The accuracy performance of each of these 4 models were pretty comparable to each other ranging from 76% to 79%. However, these were pretty basic models, and if it is worth doing, it is worth overdoing! I then built a much more sophisticated random forest model with the intent of 
    1. Improving accuracy and 
    2. finding out which words were most important in deciding who said what. 

With a relatively small sample size I was unable to drastically improve accuracy, but I was able to grab which words were most important in this beefed up model:


 
Most of these words may be circumstantial, and therefore not very informative if we use this model on more future text, but the one word that stuck out above the rest was "true". It appears that if the word 'true' appears in the quote, the model has a better idea of who said it. This, paired with Joe Biden's top word (after processing) being "fact" is pretty interesting! Thanks for reading. 


Comments

Post a Comment

Popular posts from this blog

Using a Neural Network to Predict Pneumonia From X-Ray Images

-
Image
According to the American Lung Association , ' Pneumonia is an infection of the lungs that may be caused by bacteria, viruses, or fungi. The infection causes the lungs' air sacs (alveoli) to become inflamed and fill up with fluid or pus. That can make it hard for the oxygen you breathe in to get into your bloodstream. The symptoms of pneumonia can range from mild to severe, and include cough, fever, chills, and trouble breathing.' Pneumonia Infographic Patients with pneumonia could have the following symptoms: Cough that produces phlegm or sometimes blood Fever Shortness of breath or difficulty breathing Chills or shaking Fatigue Sweating Chest or muscle pain If a person is suspected of having pneumonia, a doctor will begin by asking the patient about their medical history and symptoms. Next will be a physical exam, so the doctor can listen to the patient's lungs. In checking for pneumonia, a doctor will listen for abnormal sounds like crackling, rumbling or wheezing. I
Read more

Using a Neural Network to Classify Lego Figures

-
Image
 The efficacy of image classification has grown leaps and bounds in recent years with the advent of neural networks. Without going into too much detail, an artificial neural network consists of a series of layers through which inputs flow becoming output predictions. Each layer is composed of a predetermined number of neurons, and each neuron feeds information to the next layer of neurons via weighted connections. These weights are real numbers that flow from layer to layer eventually reaching an output layer, producing a numerical prediction.  For the purposes of image classification, the inputs are pixels. These pixels are converted to numeric inputs by gray-scale figures or color codes (RBG, hex, etc). The output is the neural network's prediction of what that image contains. The neural network learns using a dataset of labeled images; recording correct predictions and attempting to grasp what elements of a picture correspond to which labels.  This abstract concept can be intui
Read more

Artwork Locator

-
Image
 Have you ever wondered where in the world famous artists' work currently resides? I was, so I built this interactive dashboard  (follow the link for the interactive version, below is just a snapshot) to help answer that question. Additionally, the world's largest museums typically only display roughly 5% of their collection at any given time, so I included a graphic outlining the proportion of that artist's work on display vs in storage.  Disclaimer: The data included is a global snapshot gathered in 2016, so changes likely have been made to the locations of certain pieces included in this dataset.
Read more