CI’s tonality algorithm performs linguistic and statistical analysis on each post to determine its overall sentiment
June 24th, 2008
Matt Dickman at Techno//Marketer has recently been posting about how sentiment is determined in online conversations while pointing out the difficulty posed with language and mentions Collective Intellect.
“Part of their analysis is of language within conversations and the sentiment that is displayed there. The sentiment is then tracked over time and can be a key metric in the success of a campaign. Their formula for extracting the sentiment is not publicly accessible so I am not sure how they calculate it.”
His question prompted our own response to explain how we determine sentiment. What follows is our current working approach hot from those that know.
Sentiment = The Collective Intellect tonality algorithm performs linguistic and statistical analysis on each post to determine its overall sentiment (positive, negative, or neutral). The CI algorithm is designed to operate effectively across a broad range of domain areas. The algorithm has been tested on several standard sentiment datasets (such as movie reviews) and consistently performs at a level close to human inter-rater accuracy.
And if you are wondering what human inter-rater accuracy is, here is that definition.
Human inter-rater accuracy (really correlation) is a measure of how well a set of independent raters correlate on rating a test set. For example, for a particular test set, if the sentiment ratings of a set of 4 human raters only correlate at 70% (on average) then that is the best accuracy level any automated system could hope to attain on that test set.
Clearly, there is a lot more to discover here, but that is a good reason why you should stay in touch with the social media strategy and activation experts at Collective Intellect.
Tags: Blog Analysis, Social Media Analytics, Social Media Organization // 1 Comment »
Twitter Micro Markup Spec
April 25th, 2008

Just like the semantic web I think Twitter would benefit tremendously from introducing a micro markup language that monitoring tools could use to pick up data more accurately. It’s a fairly hard problem to parse blogs and tag them with semantic and qualitative data. Now think about shrinking that text down to 140 characters and you really don’t have a hell of a lot to work with. If we introduce an abbreviated name-value system inside our tweets we can convey a ton of rich information that will be easier to get the message across to the companies and services that anger and overjoy us, events we’re attending, observations we make, etc. It’s time to take the next step with Twitter.
I think it would be simplest to put an abbreviated 2 letter type code at the start of the message with a sentiment symbol next (+ = good, - = bad, / = neutral), and the subject that you will be talking about. For instance I just got back from a trip to Minneapolis. My flight was delayed again (probably my fourth flight in a row on United that was delayed).
I twittered the following: “RV-UAL another trip, another flight delay on United”. In the first 6 characters I know that this is a review of United Airlines and it is negative. Simple, brief, easy for both humans and machines to parse and interpret.
The codes I’m thinking of are:
OB = observation
EV = Event
RV = Review
If it’s an event, perhaps the standard could be to create the event in facebook and snurl it for the event subject. what are some others? Let me know what you think!
Tags: Social Media Best Practice, Social Media Organization // 1 Comment »
Future of the next generation agency
January 23rd, 2008
Around the office, we talk alot about the future of the next generation agency. As a technology solution that is helping companies and agencies measure and engage with social media, we have a lot of skin in the game, so to speak, on the future of agencies and agency holding companies.
These two links from MediaPost, I think, do a good job of laying out the issues that face agencies today:
- A New Era For Agencies, posted by Dave Morgan of AOL.
- Is This the Year Of The Agency?, posted by
Cory Treffiletti of Catalyst SF
Tags: Brand Management, Social Media Organization // Add Comment »