Twitter Sentiment and The Debates

I’d like to enlist your help in a project we have been creating at the USC Annenberg Innovation Lab . We have built an analytics tool to track sentiment around the two candidates using Twitter as the data feed. So we take in every tweet about the candidates and our language-ware reads the tweets and creates a score from 1-100, positive or negative. In the Presidential Debate this Wednesday we will probably be reading and analyzing up to 1000 tweets per second. Ideally our real time sentiment analysis will act like a million person focus group on the debate.This work is similar to the analyses we’ve done over the past 18 months on pop culture events like the Oscars, Summer Blockbusters, Fashion Week trends, and sporting events such as the World Series and Super Bowl.

So here is where we need help. Computers, as you might imagine, are not great at detecting sarcasm. Though advances in software and cognitive computing are helping us make great progress.When we first started this project I remember a tweet, “I’m so happy Michelle Bachmann has thrown her tin foil hat in the ring”. The computer thought that was a positive sentiment towards Bachmann, until a human student corrected it. We have built a human annotation page into the dashboard where you can correct any tweet you think has been mis-scored. From the main page of the site, click “Go to Tweets/annotation page”. There you will see a sampling of the latest tweets and their score. Click on any given tweet and you will be taken to an annotation page where you can manually correct the computer. Obviously the more people who do this, the better the outcome.

One last note. Obviously as you read some of the most negative tweets you will be appalled at what passes for political dialogue in the age of Twitter. We surmise that because many people tweet anonymously, they feel free to speak in the most hateful language. As you will see, most people tweet against a candidate rather than for a candidate. It says something about the psychology of Twitter, but I’m not sure just what it means for an digital democracy.

We’re learning quite a bit through this project and others about how analytics technologies can be applied to Big Data to understand and predict trends. Thanks in advance for your help.

This entry was posted in Futurism, Politics and tagged , , , , . Bookmark the permalink.

19 Responses to Twitter Sentiment and The Debates

  1. len says:

    Time permitting.

    It says something about the psychology of Twitter, but I’m not sure just what it means for an digital democracy.

    It says groundlings haven’t changed much.

  2. Roman says:


    Before Wed’s exercise, it would interesting to review the demographics of ‘tweeting’, specifically, who tweets and why?

    Also, do ‘tweeters’ regard tweeting as another form of dialogue (i.e. multidimensional – exchange of ideas, emotions etc.), or is it something else entirely?

    It has always impressed me as more ‘expression’ (i.e. one dimensional – rant, rage, vent etc.) than dialogue. Correct?

  3. len says:

    Comments where comments are to conversation what pieces of meat are to a trout line: baited hooks. If this were Gellish, we’d be filling in the optional “intention” column. Or that’s my take on it.

  4. Alex Bowles says:

    @Roman Can’t speak to how others use it, but I find that it’s a good information filter, but one that takes a bit of time and effort to tune properly.

    It works by realizing that any subject is going to have a collection of commentators surrounding it, and that some are going to be better than others. Step one is following a bunch of people, then weeding out the ones that are second rate. I’ve been following a lot of science writers recently, especially ones focused on energy. I’m also following a lot of mapping and cartography people, and some robotics people, along with some wonks who write about politics and economics, a few architecture, art, and design types, and some people covering the business of media and tech.

    Many of these people write elsewhere. Some of what they tweet is self-referential, but most of it has to do with what they’re reading. Of course, anything that really gets people excited generates a lot of buzz, and that’s really what you scan for. It’s also interesting to see how some events (the Mars landing, for instance) cut across all categories. It’s the only thing anyone was tweeting about, regardless of their regular interests. And then there’s the real-time hive-mind aspect, which can be entertaining (“omfg the Chair!).

    Occasionally I look at the trending topics section, and realize that huge numbers of people are using it very differently. Clicking through some of the tweets on the Annenberg site reconfirms this. I mean…wow.

    But normally I’d never come across any of that. For me it is very much about an exchange of ideas. Links are essential. Tweets are just the wrappers, but ones that give you a sense of the sharers personality. Over time, you begin to recognize which sources are more reliable, and ones with bad signal to noise ratios. When I tweet things, it’s because I think they’re especially good, funny, insightful, etc. It’s an ecosystem, after all. I get a lot from it, and in return I enjoy providing solid material for others.

  5. Alex Bowles says:

    Related: The rapidly changing media landscape and what it means for politics — in 1 chart.

  6. len says:

    @Alex Bowles

    I wonder what the effects of alternative reality tweeting would be on a sentiment analysis system. It’s improv + real world situations or stimuli. Fun to play on Facebook. The musicians, actors and anyone who has to be “good on their feet” know how to do it. The groundlings, mostly not. They tend to want to be tweetTrolls and break up the flow.

  7. Andres says:

    This is fascinating. I wonder if the knee jerk towards complaint is solely an American quality; to find fault first and then solutions after… no judgement here, just an observation. I’m in Jon.

  8. len says:


    After a few decades of participating in the online lists from the original email lists to social media, the speed and size of the boxes have changed, but not the games. And no, it is not an American quality only. There are social cultural biases toward particular behaviors but the mammals are pretty much different makes of the same models.

    Beware the zeitgeist. Depending on the analysis model (say latent semantic analysis or vector models and how deeply features are analyzed), with skill and practice, it is easy to game these systems. Willingness to do that may be something that varies by culture which is why background profiling of personna is important to rating the predictive power of the model and the make.

    Never underestimate the willingness and desire of the humans to make monkeys of each other and machines. Gaming is the most fun there is. Alternative reality games are one example. For the serious minded in this industry, the abillity of clever types to introduce small amplifying effects (wasping) on the big data inputs is in itself a means to change the outcomes as they are fedback through the cultural controls. Done covertly and with some skill, your sentiment analyses can be turned to their advantage. How well you can show you can defend against that determines your own rating.

  9. ʇʇǝuɹnq ǝuoq ʇ says:

    ˙ןǝʌou uosqıb ɯɐıןןıʍ ɐ ɟo ʇno sı ʇı ǝʞıן spɐǝɹ pɐǝɹɥʇ sıɥʇ

  10. len says:

    :) Meanie!

    The marvel of using humans is I can read that without calling a different parser or a mirror.

    Something interesting to ponder about limits and non-limits to systems that have to make predictions. Not an easy read but…

    Controls emerge out of competition for resources. For example, as Lowery noted, a new Internet lobbying group was created to ensure some resources remain cheap/free to industries that have been rerouting energy/wealth to their own systems. Some call this “disruptive innvation”. Others call it stealing. We look to the larger control systems for judgement but it is far to easy to corrupt those.

    On the other hand, natural systems tend to find balance. In this case, over time other systems emerge to create their own ecosystems. Among these will be the digital forensics industries that need their own standards, lobbying and other professional associations and they will tend to defend their customers business models. It takes time but in my experience, the locals (say artists in this case) will begin to notice other natural allies emerging as they realize their own business models. It is sort of a free market model on the surface, but really, it is natural selection at work.

  11. len says:

    PS: Someone asked me at work where all of this sentiment analysis and feedback control, behaviior driving was going, or “what’s next”. I told him we are turning into Disneyland SteamPunk, which as you surmise, is a lot like Gibson. But a theme park has all those little secret exits and entrances from which park employees magically appear to put people in the right lines or….

    I don’t like theme parks myself but I’m a dinosaur who doesn’t like bluegrass festivals because they all sound the same and play the same songs and if a sax player shows us, there are all these secret exist and entrances from which bluegrass guards magically appear to out people in the right lines. The pioneers have all the fun and most of the bad times, then the doctors and lawyers imitate them and call it a day.

    So much for Americana.

  12. John says:

    What bothers me most about twitter sentiment analysis is that it’s predicated on the idea that humans are the only source of tweets. We saw that this was wrong during the Russian protests last year( This will only get worse. The medium is the message.

  13. Alex Bowles says:

    It’s not all robots.

    @johnmoe As record producers go, Flank Steak Burnett is not as good as his brother.

  14. John says:

    Of course it’s not all robots, but eventually the noise from all the robots poisons the well. The problem is that incentivizing tweet generation fuels an arms race that can’t be won.

  15. len says:

    John :Of course it’s not all robots, but eventually the noise from all the robots poisons the well. The problem is that incentivizing tweet generation fuels an arms race that can’t be won.

    Not necessarily. There are means to detect bots, trolls, etc., and to weight away from the influence. It is not precisely an arms race because although you can use bot swarms to overwhelm internet connections, you can’t completely disguise directed content attacks. Digital forensics are advancing fast because it is a growing dues paying market. Techniques to authenticate and test the legitimacy of messages and other content are getting better and the incentives to improve them are increasing.

    It is a cost race because the means to alter a message to remove those traces gets more expensive and disincentivizes the opportunity costs. I brought up alternative reality humor because it is one of the cheaper and more difficult attack strategies given a well-practiced ensemble. On the other hand, they also need incentives and this comes down to the goals of the event being influenced. A vote on American Idol and a solicitation of opinions on a camera lens are different kinds of events.

    To influence an event, you want to know as much as you can about the type of event, the terms that occur in events of that type (frequency and amplitude/intensity), personna profiles if you can get them, and the design/features of the analyst/program. Then you can set up the scenarios for the “conversations” that you will use if possible to induce non-linear effects and reduce the possibility that the “scam” is detectable.

    You can cheat an honest man by taking advantage of honesty. You can’t necessarily turn him. You can change the environment/zeitgeist to disfavor honesty. This is somewhat the case for download pirates. See Clockwork Orange.

  16. JTMcPhee says:

    Can’t wait to hear all the different takes on the Great Debate, which added weight to my notion that Democrats neither want to rule nor how to. Decision to Romney on this one, more’s the frippin’ pity. Obama just can’t grab them levers of power when it’s mano a mano. “Of course, all Romney had to do for weeks and weeks was to get primed for the debate, and Obama with the weight of the Free World dangling from his scrawny neck had more important things to do, like select drone targets and who gets renditioned this week and Double Secret Findings to read and review and sign off on. Give the Man a break, okay?”

    Please, Spirit of the Universe, don’t let this be a Sign of Things To Come.

  17. len says:

    It isn’t that bad. To win that Romney had to commit to a set of issues his base hates and the fact that he isn’t being eviscerated is they hate Obama more than they love their issues.

    Romney told you what he was going to do in his 47% speech. That is exactly what he did with a knowing smirk. OTOH, every time he flip flopped to carve off those few extra undecideds, there was a tell.

    Winning on style is like getting a woman to marry you by wearing an expensive suit. The divorce is more expensive. Let’s see what Biden, a man who does not refrain from throwing a punch, does with NellyBush-Ryan.

Leave a Reply