Linear Regression
Linear Regression
Linear Regression is a supervised learning algorithm that models the relationship between a dependent variable () and independent variables () by fitting a linear equation to observed data. It optimizes the slope coefficients by minimizing the Mean Squared Error (MSE) using Ordinary Least Squares (OLS) or Gradient Descent.
Complexity Profile
| Case | Complexity |
|---|---|
| Best Case | O(P^2 * N) |
| Average Case | O(P^2 * N) |
| Worst Case | O(P^2 * N) |
| Space Complexity | O(P) |
Code Implementation
import numpy as np
class LinearRegressionGD:
def __init__(self, lr=0.01, epochs=1000):
self.lr = lr
self.epochs = epochs
self.weights = None
self.bias = None
def fit(self, X, y):
n_samples, n_features = X.shape
self.weights = np.zeros(n_features)
self.bias = 0
for _ in range(self.epochs):
y_pred = np.dot(X, self.weights) + self.bias
# Compute gradients
dw = (1 / n_samples) * np.dot(X.T, (y_pred - y))
db = (1 / n_samples) * np.sum(y_pred - y)
# Update weights
self.weights -= self.lr * dw
self.bias -= self.lr * db
def predict(self, X):
return np.dot(X, self.weights) + self.biasReal-World Applications
- Economic forecasting (predicting house prices or retail sales volumes).
- Trend lines analysis for scientific models.
- Risk assessment tools in financial banking software.
K-Means
K-Means is a centroid-based clustering algorithm. It partitions $N$ observations into $K$ distinct clusters where each data point belongs to the cluster with the nearest mean (centroid). It repeats two steps: assigning points to the closest centroid, and recalculating centroids based on the average of all cluster members.
Naive Bayes
Naive Bayes is a probabilistic classifier based on Bayes' Theorem. It makes the 'naive' assumption that features are conditionally independent of each other given the class label, which simplifies joint probability calculation and enables fast training speeds on large datasets.