Context and objective
I proposed this solution to a financial company specializing in cryptocurrency investments. The idea came from the need to better understand public consensus about Bitcoin through social media analysis. I collected millions of tweets containing Bitcoin references to conduct an in-depth analysis.
My project’s main goal was to analyze the sentiment expressed in these tweets and create a graph showing daily sentiment evolution. I also wanted to answer specific questions like:
- Do negative tweets receive more likes than positive ones?
- Do negative tweets generate more interactions (replies) than positive ones?
As an additional objective, I tried to verify if sentiment variations were associated with Bitcoin value changes, using historical BTC/USD exchange rate data.
Implemented solution
I developed the project using Python, with particular emphasis on Pandas libraries for data manipulation and TextBlob for sentiment analysis. I pre-processed tweet data to remove non-textual elements and normalize text. Then, I applied sentiment analysis to classify tweets as positive, negative, or neutral. I aggregated results daily to track sentiment evolution over time.
To answer specific questions, I conducted statistical analyses comparing the average number of likes and replies between positive and negative tweets. Finally, I integrated Bitcoin historical value data to examine correlation between sentiment variations and price fluctuations.
Here’s an example of code I used for sentiment analysis:
from textblob import TextBlob
def analyze_sentiment(text):
analysis = TextBlob(text)
if analysis.sentiment.polarity > 0:
return 'positive'
elif analysis.sentiment.polarity < 0:
return 'negative'
else:
return 'neutral'This script defines a function that classifies the sentiment of a text based on the polarity value calculated by TextBlob.
Results and benefits
My analysis revealed that sentiment in Bitcoin-related tweets varies significantly over time, with peaks corresponding to specific cryptocurrency market events. I found that negative tweets tend to receive more replies than positive ones, while the number of likes showed no significant difference between the two tweet types.
Integrating Bitcoin price data allowed me to highlight some correlations between sentiment and market value changes. However, establishing a direct causal link wasn’t possible.
Challenges and how I overcame them
One of the main challenges I faced was managing the large amount of unstructured textual data. To solve this, I used advanced pre-processing techniques, including:
- Stopword removal and text tokenization
- Stemming and lemmatization to improve sentiment analysis accuracy
- Filtering duplicate or spam tweets to reduce data noise
Another challenge was the need to process data within reasonable timeframes. To optimize performance, I implemented tools like multiprocessing and query optimization for data retrieval.
Conclusion and impact
My project demonstrated the value of sentiment analysis as a tool for monitoring public opinions and identifying cryptocurrency market trends. While I couldn’t establish a direct correlation between sentiment and Bitcoin value, my analysis provided useful insights for the company’s investors and market researchers.
For future developments, I’m considering integrating more advanced machine learning models to improve sentiment classification accuracy and using data from multiple sources (forums, news, blogs) to provide a more comprehensive market view.