Asynchronous Distributed ADMM for Learning with Large-Scale and High-Dimensional Sparse Data Set

Wang, Dongxia and Lei, Yongmei (2019) Asynchronous Distributed ADMM for Learning with Large-Scale and High-Dimensional Sparse Data Set. In: Advanced Hybrid Information Processing. Third EAI International Conference, ADHIP 2019, Nanjing, China, September 21–22, 2019, Proceedings, Part II.

[thumbnail of 74974.pdf] PDF
74974.pdf

Download (409kB)

Abstract

The distributed alternating direction method of multipliers is an effective method to solve large-scale machine learning. At present, most distributed ADMM algorithms need to transfer the entire model parameter in the communication, which leads to high communication cost, especially when the features of model parameter is very large. In this paper, an asynchronous distributed ADMM algorithm (GA-ADMM) based on general form consensus is proposed. First, the GA-ADMM algorithm filters the information transmitted between nodes by analyzing the characteristics of high-dimensional sparse data set: only associated features, rather than all features of the model, need to be transmitted between workers and the master, thus greatly reducing the communication cost. Second, the bounded asynchronous communication protocol is used to further improve the performance of the algorithm. The convergence of the algorithm is also analyzed theoretically when the objective function is non-convex. Finally, the algorithm is tested on the cluster supercomputer “Ziqiang 4000”. The experiments show that the GA-ADMM algorithm converges when appropriate parameters are selected, the GA-ADMM algorithm requires less system time to reach convergence than the AD-ADMM algorithm, and the accuracy of these two algorithms is approximate.

Item Type: Conference or Workshop Item (UNSPECIFIED)
Date Deposited: 04 Mar 2026 19:13
Last Modified: 20 Nov 2026 11:39
URI: https://eprints.eai.eu/id/eprint/55630

Actions (login required)

View Item
View Item