BIRCH-AE: A Hierarchical Ensemble Framework for Scalable E-Commerce User Segmentation with Autoencoder-Enhanced Feature Learning

The rapid expansion of e-commerce platforms has intensified demand for scalable, high-quality user segmentation systems capable of efficiently processing millions of behavioral records. This paper presents BIRCH-AE, a hierarchical ensemble clustering framework that integrates the Balanced Iterative Reducing and Clustering using Hierarchies (BIRCH) algorithm with autoencoder-based feature learning for large-scale e-commerce analytics. The autoencoder compresses high-dimensional behavioral data into compact latent representations, mitigating the curse of dimensionality and improving cluster separability. Multiple BIRCH configurations are combined through four ensemble strategies: Majority Voting, Weighted Voting, Advanced Affinity-based Spectral Clustering (AASC), and the proposed BIRCH-Optimized Hierarchical Consensus (BOHC). Dynamic selection based on multi-criteria evaluation automatically identifies the best-performing strategy per dataset setting, emphasizing that no single consensus method is universally optimal. Experiments on two large-scale datasets (Retail Rocket with 1.4M users and E-Commerce Behavior with 4.5M users) show improved clustering quality and practical scalability. BOHC achieves up to 23% silhouette improvement over single BIRCH for transaction-focused data with clearer hierarchical structure, while multi-domain data favor strong base models. Autoencoder feature learning improves clustering quality by 23–53% over raw features. The full 4.5M-user experiment was executed as a BOHC scalability run, completed in approximately 5 minutes, while framework-level comparative analyses were conducted through repeated stratified 30% subset trials. These findings support BIRCH-AE as a practical and adaptive segmentation framework for enterprise-scale e-commerce analytics.

Text 126891.pdf - Published Version Available under License Creative Commons Attribution. Download (3MB) Official URL or Download Paper: https://ieeexplore.ieee.org/document/11540050/

Actions (login required)

View Item