I remember sitting in a windowless lab during my PhD, staring at a dendrogram that looked less like a meaningful data structure and more like a tangled mess of Christmas lights. I had been taught that hierarchical clustering was this elegant, mathematical inevitability, but in practice, I was struggling to figure out why my clusters were merging in ways that made zero sense for the actual system I was building. Most textbooks treat the algorithm as a black box that just works, skipping over the messy reality that your choice of linkage criteria can completely warp your results. It’s frustrating because people often treat it as a “set it and forget it” tool, when in reality, the way you define distance is the difference between discovering a pattern and just generating noise.

I am not here to give you a sanitized lecture or a list of definitions you could find in a Wikipedia entry. Instead, I want to walk through the actual mechanics of how these trees are built, from the initial proximity matrices to the final merges. We are going to look at where the math holds up and where it breaks down when you scale up. My goal is to ensure that when you implement hierarchical clustering, you actually understand the structural trade-offs you are making.

Agglomerative vs Divisive Clustering the Direction of Growth

When we talk about how these trees are actually built, we’re essentially choosing between two opposing philosophies: bottom-up or top-down. Most of the unsupervised machine learning algorithms you’ll encounter in the wild are agglomerative, meaning they start with every single data point acting as its own tiny, lonely cluster. The algorithm then iteratively finds the closest neighbors and merges them. It is a greedy process; once two points are joined, they are locked into that relationship forever, regardless of what happens later in the hierarchy.

Divisive clustering takes the exact opposite approach. You start with one giant, monolithic cluster containing everything and systematically hack it into smaller pieces. While this sounds more intuitive, it is computationally much more expensive because you have to decide how to split the data at every single step. In practice, most researchers stick to the agglomerative route because it’s more efficient, even if it means you have to be careful about how you choose your distance metrics in clustering to avoid merging groups that shouldn’t be together.

Distance Metrics in Clustering Defining Proximity and Error

If you want to understand how these clusters actually form, you have to stop thinking about “similarity” as a vague concept and start thinking about it as a mathematical distance. In unsupervised machine learning algorithms, the entire structure of your tree depends on how you define the gap between two points. If you use Euclidean distance, you are measuring the straight-line “as the crow flies” distance between coordinates, which works fine for physical space but can fail miserably if your data lives in high-dimensional spaces where everything starts to feel equally far apart.

The real complexity arises when we decide how to measure the distance between two groups rather than just two points. This is where the choice between single linkage vs complete linkage becomes the pivot point for your entire analysis. Single linkage looks at the closest neighbors between clusters, which often leads to “chaining”—a phenomenon where clusters grow into long, thin strands that don’t actually represent distinct groups. Complete linkage, conversely, looks at the most distant members, forcing the clusters to be more compact and spherical. It’s a trade-off: you can have a sensitive, sprawling tree or a rigid, tightly-packed one, and there is no single “correct” metric that works for every dataset.

Practical Realities: How to Avoid Getting Lost in the Tree

• Don’t let the dendrogram fool you into thinking the clusters are “correct.” A dendrogram is just a visualization of how the algorithm made its decisions based on your distance metric; it doesn’t actually know if the resulting groups make sense in the real world. You still have to validate the clusters against your actual domain knowledge.
• Watch your memory usage like a hawk. Unlike K-means, which only needs to keep track of cluster centroids, most agglomerative implementations require a distance matrix. If you have 100,000 data points, that matrix is going to be massive, and you’ll likely run out of RAM long before the algorithm finishes.
• Be extremely careful when combining different distance metrics with different linkage methods. Using Euclidean distance with single-linkage clustering is a recipe for “chaining,” where your clusters become long, thin strings of points rather than cohesive blobs. If you want tight, spherical clusters, you’re better off pairing Euclidean distance with Ward’s method.
• Remember that hierarchical clustering is sensitive to outliers. Because the algorithm builds connections step-by-step, a single point sitting far away from everything else can either act as a bridge that incorrectly merges two distinct groups or remain as a lonely singleton for almost the entire process, skewing your perception of the hierarchy.
• Use the hierarchy to your advantage by looking at the “gap” between merges. When you see a large vertical jump in the dendrogram, that’s the algorithm struggling to find a reason to connect two groups. That jump is often a much more reliable indicator of the “natural” number of clusters than simply picking a number and hoping for the best.

What to carry away from this

Hierarchical clustering isn’t a single algorithm but a family of approaches; you have to decide upfront whether you are building up from individuals (agglomerative) or breaking down a whole (divisive), as the computational cost and the resulting structure will differ significantly.

The “closeness” of your data is entirely dependent on your choice of distance metric, and picking the wrong one—like using Euclidean distance for high-dimensional sparse data—will result in clusters that look mathematically sound but are physically meaningless.

The dendrogram is a beautiful map of relationships, but it is also a rigid one; unlike K-means, you can’t easily “re-cluster” once the hierarchy is built, so your initial linkage decisions dictate the entire fate of the model.

The Trade-offs of Nested Structure

We have looked at how hierarchical clustering moves from the bottom up through agglomeration or the top down through division, and how the entire process hinges on your choice of distance metric. There is no “correct” way to measure proximity; a Euclidean distance might work for spatial coordinates, but it will fail you if your data lives in a high-dimensional feature space where the concept of “closeness” becomes muddy. You have to remember that the dendrogram is not a ground truth—it is a mathematical representation of your assumptions. If your linkage criterion is too aggressive, you’ll end up with artificial clusters that don’t actually reflect the underlying distribution of your data.

Ultimately, hierarchical clustering is less about finding a single “right” answer and more about exploring the topology of your data. It gives you the luxury of seeing how groups relate to one another across different scales, something a flat K-means approach simply cannot do. As you move forward with your own implementations, don’t just look for the most aesthetically pleasing tree. Instead, look for the points where the structure feels forced. Understanding where the algorithm struggles is often more instructive than seeing where it succeeds.

About Dr. Ingrid Falk-Weller

I write for the person who wants to understand the mechanism, not memorise the conclusion. If a claim has a caveat, the caveat goes in the paragraph, not a footnote.