Stock Price Extraction: Improving Sentiment Measures
Sentiment has been used in stock analysis and trading for some time. Historically, surveys have been used, which tend to work well but are also expensive, time consuming, and limited to a small number of assets. The advent of social media broadened the amount of data as well as the number of assets that could be covered. However, traditional sentiment approaches focus on analyzing word choice which can often create other problems – not least of which misinterpreting slang and sarcasm.
Additionally, many analysts will attempt to create the ‘perfect’ sentiment system by optimizing the results and in the process ‘overfitting’. In such a case, the results look spectacular but are not based on a true interpretation of sentiment but on some other variable such as financial return of stock trading based off of signals for that particular dataset. The problem then arises later when conditions change and the sentiment system ceases to produce such wonderful results.
ZettaCap reimagined a better sentiment approach – one that not only got around many of the weaknesses of other systems but also better quantified the opinion of the commentator towards that asset. Stock Price Extraction (SPE) identifies stock, or asset, prices within social media and compares this price to the then-current price. We further drill down on if that price is meant to be a target price, an assumed support level, an assumed resistance level, a conditional buy level, etc.
For instance, if a commentator says “The Dow Jones is going to 30,000!”, a traditional sentiment measure would not likely pick up anything as this statement does not contain obvious word-based sentiment. But, if you knew that the Dow was then trading at 20,000 you would know exactly what the commentator was saying and you could even quantify just how bullish he / she was at that time.
Another advantage, outside of pure sentiment, of the SPE technique includes the identification of important price levels. Prices mentioned tend to cluster around certain levels. Investors, but especially traders, understand the importance of price level identification and SPE clarifies important price levels unlike other sentiment indicators that simply identify direction.
SPE is an original and innovative approach to analyzing social media for investment analysis. It is the only known approach that focuses on prices mentioned. Its machine learning based algorithm was trained on thousands of tweets and applied successfully to stocks, currencies, and commodities. The image here shows a basic diagram of the extraction, storage, and utilization of the data.
Important because:
- Unique sentiment system that gets around issues of ‘overfitting’ data and problems associated with misinterpreting slang and sarcasm,
- Only system that uses prices mentioned to measure sentiment,
- Prices mentioned also serve to highlight important price levels for traders, only system to do so.