Machine learning (ML) is revolutionizing data analysis and predictive modeling. In this article, I’ll guide you through creating a simple sentiment classifier to predict whether a movie review is positive or negative. This beginner-friendly project uses Python and scikit-learn, and by the end, you’ll have a functional model ready for real-world applications. Let’s explore the five steps I took to build this project from scratch!
Step 1: Understanding Model Training Basics
Model training involves teaching an ML model to identify patterns in data. For this project, we aim to classify movie reviews as “positive” or “negative” based on their text. The process includes:
- Data Preparation: Collecting and cleaning data.
- Choosing a Model: Selecting an algorithm (e.g., Logistic Regression).
- Training: Feeding data to the model to learn.
- Evaluation: Testing the model’s performance.
- Deployment: Integrating the model into a project.
What I Did:
- Installed Python (3.8+) and required libraries: scikit-learn, pandas, and numpy.
- Ran this command to set up my environment:
pip install scikit-learn pandas numpy - Chose to build a sentiment classifier for movie reviews, a classic ML task.
This step established the foundation, ensuring I had the tools and a clear objective.
Step 2: Preparing the Data
Data is the backbone of any ML model. For simplicity, I used a small sample dataset of movie reviews, but you can scale up with datasets like IMDb from Kaggle.
What I Did:
- Created a sample dataset with five reviews and their sentiments.
- Cleaned the text by converting it to lowercase and removing punctuation.
- Converted text into numerical features using a bag-of-words model with
CountVectorizer.
Here’s the code:
import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer
import re
# Sample dataset
data = {
'review': [
'This movie was amazing and I loved it!',
'Terrible film, really boring.',
'Great acting and wonderful story.',
'Awful, I hated this movie.',
'Fantastic experience, highly recommend!'
],
'sentiment': ['positive', 'negative', 'positive', 'negative', 'positive']
}
# Create a DataFrame
df = pd.DataFrame(data)
# Clean text data
def clean_text(text):
text = text.lower() # Convert to lowercase
text = re.sub(r'[^\w\s]', '', text) # Remove punctuation
return text
df['review'] = df['review'].apply(clean_text)
# Convert text to numerical features
vectorizer = CountVectorizer()
X = vectorizer.fit_transform(df['review']) # Features
y = df['sentiment'] # Labels
print("Feature matrix shape:", X.shape)
print("Labels:", y)
This code transformed raw text into a numerical format, creating a feature matrix (X) and labels (y) for the model.
Step 3: Choosing and Training the Model
I selected Logistic Regression, a robust algorithm for binary classification (positive vs. negative).
What I Did:
- Split the data into training (80%) and testing (20%) sets for later evaluation.
- Trained the Logistic Regression model on the training data.
Here’s the code:
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Initialize and train the model
model = LogisticRegression()
model.fit(X_train, y_train)
print("Model trained successfully!")
print("Training data shape:", X_train.shape)
print("Testing data shape:", X_test.shape)
The model learned patterns from the training data, preparing it for predictions.
Step 4: Evaluating the Model
To assess performance, I tested the model on the unseen test set and calculated metrics like accuracy.
What I Did:
- Made predictions on the test set.
- Computed accuracy and generated a classification report for detailed metrics.
Here’s the code:
from sklearn.metrics import accuracy_score, classification_report
# Make predictions
y_pred = model.predict(X_test)
# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)
# Print results
print("Test Accuracy:", accuracy)
print("\nClassification Report:")
print(classification_report(y_test, y_pred))
Note: With only five reviews, the test set had one sample, making metrics less reliable. For robust results, use a larger dataset like IMDb with thousands of reviews.
Step 5: Deploying the Model in a Project
I built a simple application to predict sentiments for new reviews, showcasing real-world usability.
What I Did:
- Saved the model and vectorizer for reuse.
- Created a function to preprocess and predict sentiment for new reviews.
- Developed an interactive script for user inputs.
Here’s the code:
import joblib
# Save the model and vectorizer
joblib.dump(model, 'sentiment_model.pkl')
joblib.dump(vectorizer, 'vectorizer.pkl')
# Function to predict sentiment
def predict_sentiment(review, vectorizer, model):
review = clean_text(review)
review_vector = vectorizer.transform([review])
prediction = model.predict(review_vector)
return prediction[0]
# Interactive script
print("Sentiment Classifier: Enter a movie review to predict its sentiment.")
while True:
user_review = input("Enter your review (or type 'exit' to quit): ")
if user_review.lower() == 'exit':
break
sentiment = predict_sentiment(user_review, vectorizer, model)
print(f"Predicted Sentiment: {sentiment}\n")
# Example usage
sample_reviews = [
"This movie was fantastic and thrilling!",
"I didn’t enjoy the plot, it was confusing."
]
for review in sample_reviews:
sentiment = predict_sentiment(review, vectorizer, model)
print(f"Review: {review}\nPredicted Sentiment: {sentiment}\n")
This script enables users to input reviews and receive instant sentiment predictions, demonstrating practical deployment.
What’s Next?
This project is a great starting point! Here are ways to enhance it:
- Improve the Model: Experiment with algorithms like Naive Bayes or use larger datasets.
- Deploy as a Web App: Use Flask or Streamlit for a user-friendly interface.
- Explore New Domains: Apply the workflow to predict stock trends, customer feedback, or spam.
Building this sentiment classifier was an exciting way to dive into the ML workflow. Whether you’re new to ML or advancing your skills, I hope this inspires your next project!
Try It Yourself: Grab the code, install the libraries, and experiment with your dataset. Share your projects or questions on LinkedIn—let’s connect and learn together!
#MachineLearning #Python #DataScience #SentimentAnalysis #ScikitLearn
Leave a Reply