Recommender Systems I: Content-Based and Collaborative Filtering

Recommender systems are arguably the most economically influential machine-learning systems ever built. They decide which videos you see next, which products appear first, and which courses a learning platform suggests. Today we learn the two classical families of recommenders and their core difficulties.

The problem#

We have $m$ users, $n$ items and a sparse interaction matrix $\mathbf{R} \in \mathbb{R}^{m \times n}$, where $r_{ui}$ is user $u$'s rating of item $i$ (explicit feedback) or an indicator of a click, purchase or view (implicit feedback). Most entries are unknown — typical matrices are more than 99% empty. The task is to predict missing entries, or more usefully, to rank unseen items for each user.

Content-based filtering#

Describe each item by features (genre, keywords, text embedding, price) and build a profile for each user from the items they liked. Recommend items similar to the profile.

• Strengths: works for new items with features; recommendations are explainable ("because you liked X"); no need for other users' data.
• Weaknesses: limited by the quality of item features; tends to recommend "more of the same" (low serendipity); new users have no profile.

Collaborative filtering (CF)#

CF ignores item content and relies on the wisdom of other users: people who agreed in the past tend to agree in the future.

User-based CF#

Predict $u$'s rating of item $i$ from similar users who rated $i$:

Subtracting each user's mean rating $\bar{r}$ removes individual rating habits (generous vs strict raters).

Item-based CF#

Predict from items similar to $i$ that $u$ has rated. Item–item similarities are more stable than user–user similarities (items change less than people's tastes) and can be precomputed. Amazon's classic "customers who bought this also bought" system was item-based.

Similarity measures#

• Cosine similarity between rating vectors.
• Pearson correlation (cosine on mean-centred ratings) — adjusts for rating scales.
• Jaccard similarity for binary implicit data.
• Shrinkage $\text{sim} \times \frac{n_{\text{common}}}{n_{\text{common}} + \beta}$ — down-weights similarities computed from few co-rated items.

import numpy as np

R = np.array([            # rows: users, cols: items, 0 = unknown
    [5, 4, 0, 1, 0],
    [4, 5, 1, 0, 1],
    [1, 0, 5, 4, 5],
    [0, 1, 4, 5, 4],
    [5, 5, 0, 0, 1],
], dtype=float)
mask = R > 0

def item_similarity(R, mask):
    means = np.where(mask, R, np.nan)
    C = np.where(mask, R - np.nanmean(means, axis=0, keepdims=True), 0)   # item-centred
    norms = np.linalg.norm(C, axis=0) + 1e-9
    return (C.T @ C) / np.outer(norms, norms)

S = item_similarity(R, mask)

def predict(u, i, k=2):
    rated = np.where(mask[u])[0]
    nbrs = rated[np.argsort(-S[i, rated])[:k]]
    w = S[i, nbrs]
    return float((w @ R[u, nbrs]) / (np.abs(w).sum() + 1e-9))

print("user 0, item 2:", round(predict(0, 2), 2))   # low: user 0 likes items 0-1, dislikes 3
print("user 3, item 0:", round(predict(3, 0), 2))

Implicit feedback#

Most real data is implicit: clicks, views, purchases, listening time. It has no negative signal — an unclicked item may be disliked or simply unseen. Implicit methods treat interactions as positive confidence and sample unobserved items as weak negatives. Evaluation uses ranking metrics:

• Precision@k / Recall@k — relevant items in the top $k$;
• NDCG@k — rewards relevant items ranked higher;
• MAP and hit rate.

Evaluate by holding out each user's most recent interactions (temporal split), not random ones, to mimic real use.

The hard problems#

1. Cold start — new users and new items have no interactions. Remedies: content features, popularity-based defaults, onboarding questions, hybrid models.
2. Sparsity — few co-rated items make similarities unreliable.
3. Popularity bias — popular items get recommended more, get more interactions, and become even more popular — a feedback loop that starves niche items.
4. Filter bubbles — optimising short-term engagement can narrow what users see.
5. Scalability — millions of users × items; neighbourhood methods need approximate nearest-neighbour search.

Hybrid systems#

Modern recommenders combine collaborative signals, content features and context (time, device, location) — often in two stages: a fast candidate generation step retrieves hundreds of items (e.g. via embedding nearest neighbours), and a heavier ranking model (gradient boosting or a neural network) orders them. The next lecture covers matrix factorisation, the method that learns the embeddings.