Decentralized Baseband Processing for Massive MU-MIMO Systems

Transcription

1 1 Deentralized Baseband Proessing for Massive MU-MIMO Systems Kaipeng Li, Rishi Sharan, Yujun Chen, Tom Goldstein, Joseph R. Cavallaro, and Christoph Studer arxiv: v1 [s.it] 15 Feb 017 Abstrat Ahieving high spetral effiieny in realisti massive multi-user (MU) multiple-input multiple-output (MIMO) wireless systems requires omputationally-omplex algorithms for data detetion in the uplink (users transmit to base station) and beamforming in the downlink (base station transmits to users). Most existing algorithms are designed to be exeuted on entralized omputing hardware at the base station (BS), whih both results in prohibitive omplexity for systems with hundreds or thousands of antennas and generates raw baseband data rates that exeed the limits of urrent interonnet tehnology and hip I/O interfaes. This paper proposes a novel deentralized baseband proessing arhiteture that alleviates these bottleneks by partitioning the BS antenna array into lusters, eah assoiated with independent radio-frequeny hains, analog and digital modulation iruitry, and omputing hardware. For this arhiteture, we develop novel deentralized data detetion and beamforming algorithms that only aess loal hannel-state information and require low ommuniation bandwidth among the lusters. We study the assoiated trade-offs between error-rate performane, omputational omplexity, and interonnet bandwidth, and we demonstrate the salability of our solutions for massive MU- MIMO systems with thousands of BS antennas using referene implementations on a graphi proessing unit (GPU) luster. Index Terms Alternating diretion method of multipliers (ADMM), onjugate gradient, beamforming, data detetion, equalization, general-purpose omputing on graphis proessing unit (GPGPU), massive MU-MIMO. I. INTRODUCTION MASSIVE multi-user (MU) multiple-input multiple-output (MIMO) is among the most promising tehnologies for realizing high spetral effiieny and improved link reliability in fifth-generation (5G) wireless systems [3], [4]. The main idea behind massive MU-MIMO is to equip the base-station K. Li, Y. Chen, and J. R. Cavallaro are with the Department of Eletrial and Computer Engineering, Rie University, Houston, TX ( kl33@rie.edu; yujun.hen@rie.edu; avallar@rie.edu). R. Sharan was with Cornell University, Ithaa, NY, and is now at MITRE, MLean, VA ( rrs7@ornell.edu) T. Goldstein is with the Department of Computer Siene, University of Maryland, College Park, MD ( tomg@s.umd.edu). C. Studer is with the Shool of Eletrial and Computer Engineering, Cornell University, Ithaa, NY ( studer@ornell.edu). The work of K. Li, Y. Chen, and J. R. Cavallaro was supported by the US NSF under grants CNS-16533, ECCS-1374, ECCS The work of R. Sharan and C. Studer was supported by the US NSF under grants ECCS and CCF , and by Xilinx In. The work of T. Goldstein was supported in part by the US NSF under grant CCF and by the US Offie of Naval Researh under grant N Parts of this paper have been presented at the 016 GlobalSIP Conferene [1] and the Asilomar Conferene on Signals, Systems, and Computers []. The present paper ontains a new ADMM-based data detetion algorithm and a generalized ADMM-based beamforming algorithm, as well as orresponding implementations on a GPU luster for the uplink and downlink. (BS) with hundreds or thousands of antenna elements, whih inreases the spatial resolution and provides an energy-effiient way to serve a large number of users in the same timefrequeny resoure. Despite all the advantages of this emerging tehnology, the presene of a large number of BS antenna elements results in a variety of implementation hallenges. One of the most ritial hallenges is the exessively high amount of raw baseband data that must be transferred from the baseband proessing unit to the radio-frequeny () antenna units at the BS (or in the opposite diretion). Consider, for example, a 18 BS-antenna massive MU-MIMO system with 40 MHz bandwidth and 10-bit analog-to-digital onverters (ADCs). For suh a system, the raw baseband data rates from and to the units easily exeed 00 Gbit/s. Suh high data rates not only pose severe implementation hallenges for the omputing hardware to arry out the neessary baseband proessing tasks, but the resulting raw baseband data stream also exeeds the bandwidth of existing high-speed interonnets, suh as the ommon publi radio interfae (CPRI) [5]. A. Challenges of Centralized Baseband Proessing Reent testbed implementations for massive MU-MIMO, suh as the Argos testbed [6], [7], the LuMaMi testbed [8], and the BigStation [9], reveal that the entralized baseband proessing required for data detetion in the uplink (users ommuniate to BS) and downlink (BS ommuniates to users using preoding) is extremely hallenging with urrent interonnet tehnology. In fat, all of the proposed data detetion or beamforming algorithms that realize the full benefits of massive MU-MIMO in systems with realisti (finite) antenna onfigurations, suh as zero-foring (ZF) or minimum mean-square error (MMSE) equalization or beamforming [10], rely on entralized baseband proessing. This approah requires that full hannel state information (CSI) and all reeive/transmit data streams are available at a entralized node, whih proesses and generates the baseband signals that are reeived from and transmitted to the radio-frequeny () hains. To avoid suh a traditional, entralized baseband proessing approah, existing testbeds, suh as the Arogs testbed [6], rely on maximum-ratio ombining (MRC), whih enables fully deentralized hannel estimation, data detetion, and beamforming diretly at the antenna elements. Unfortunately, MRC signifiantly redues the spetral effiieny for realisti antenna onfigurations ompared to that of ZF or MMSE-based methods [10], whih prevents the use of high-rate modulation and oding shemes that fully exploit the advantages of massive MU-MIMO.

2 FEC FEC users base station (BS) base station (BS) users map. map. uplink hannel CHEST detetor F detetor CHEST deoder FEC CHEST beamformer F beamformer CHEST downlink hannel Det. Det. De. De. Fig. 1. Overview of the proposed deentralized baseband proessing (DBP) arhiteture. Left: Massive MU-MIMO uplink: U single-antenna users ommuniate to the base station (BS). The B BS antenna elements are divided into C lusters, whih independently perform hannel estimation (CHEST) and deentralized data detetion. Right: Massive MU-MIMO downlink: The BS performs deentralized beamforming; eah of the C lusters only uses loal hannel state information. In both senarios, only a minimum amount of onsensus information is exhanged among the lusters (indiated by the dashed green lines). B. Deentralized Baseband Proessing (DBP) In this paper, we propose a deentralized baseband proessing (DBP) arhiteture as illustrated in Figure 1, whih alleviates the bottleneks of massive MU-MIMO aused by extremely high raw baseband data rates and implementation omplexity of entralized proessing. We partition the B BS antennas into C independent lusters, eah having B antennas for the th luster so that B = C =1 B. For simpliity, we will assume lusters of equal size and set S = B whih implies SC = B. Eah luster is assoiated with loal omputing hardware, a so-alled proessing element (PE), that arries out the neessary baseband proessing tasks in a deentralized fashion. A entral fusion node ( F in Figure 1) proesses a small amount of onsensus information that is exhanged among the lusters and required by our deentralized baseband algorithms (the dashed green lines in Figure 1). Throughput the paper, we fous on time-division duplexing (TDD), i.e., we alternate between uplink and downlink ommuniation within the same frequeny band. In the uplink phase, U users ommuniate with the BS. First, CSI is aquired via pilots at the BS and stored loally at eah luster. Then, data is transmitted by the users and deoded at the BS. In the downlink phase, the BS transmits data to the U users. By exploiting hannel reiproity, the BS performs deentralized beamforming (or preoding) to mitigate MU interferene (MUI) and to fous transmit power towards the users. As for the uplink, the C deentralized beamforming units only aess loal CSI. The key features of the proposed DBP arhiteture an be summarized as follows: (i) DBP redues the raw baseband data rates between eah luster and the assoiated hains. In addition, the I/O bandwidth of eah PE an be redued signifiantly as only raw baseband data from a (potentially small) subset of antennas must be transferred on and off hip. (ii) DBP lowers the omputational omplexity per PE by distributing and parallelizing the key signal-proessing tasks. In addition to deentralizing hannel estimation (CHEST), data detetion, and beamforming, DBP enables frequeny-domain proessing (e.g., fast Fourier transforms for orthogonal frequeny-division multiplexing) as well as impairment ompensation (e.g., for arrier frequeny and sampling rate offsets, phase noise, or I/Q imbalane) loally at eah luster. (iii) DPB enables modular and salable BS designs; adding or removing antenna elements simply amounts to adding or removing omputing lusters and the assoiated elements, respetively. (iv) DPB allows one to distribute the antenna array and the assoiated omputing hardware over multiple buildings an idea that was put forward reently in the massive MU-MIMO ontext [11]. C. Relevant Prior Art The literature desribes mainly three methods that are related to DBP: oordinated multipoint (CoMP), loud radio aess networks (C-RAN), and testbeds that perform distributed baseband proessing aross frequenies. The following paragraphs disuss these results. 1) Coordinated multipoint (CoMP): Coordinated multipoint (CoMP) is a distributed ommuniation tehnology to eliminate inter-ell interferene, improve the data rate, and inrease the spetrum effiieny for ell-edge users [1]. CoMP distributes multiple BSs aross ells, whih ooperate via bakhaul interonnet to perform distributed uplink reeption and downlink transmission. CoMP has been studied for ooperative transmission and reeption [13] [15] in 3GPP LTE-A, and is widely believed to play an important role in 5G networks [16] along with other tehnologies, suh as massive MU-MIMO [17]. Several algorithms for distributed beamforming with CoMP have been proposed in [18] [0]. The paper [18] proposes a distributed preoding algorithm for multi-ell MIMO downlink systems using a dirty-paper oding. The papers [19], [0] propose distributed beamforming algorithms based on Gauss- Seidel and alternating diretion method of multipliers (ADMM). These methods assume that the BSs in different ells have aess to loal CSI and oordinate with eah other with limited bakhaul information exhange. While these results are, in spirit, similar to the proposed DBP approah, our arhiteture (i) onsiders a deentralized arhiteture in whih the omputing hardware is olloated to support low-lateny onsensus information exhange, (ii) takes expliit advantage of massive MU-MIMO (the other results in [18] [0] are for traditional, small-sale MIMO systems), and (iii) proposes a pratial way to partition baseband proessing that is omplementary to

3 3 CoMP. In fat, one ould integrate DBP together with CoMP to deal with both intra-ell multi-user interferene and interell transmission interferene more effetively, and to realize deentralized PHY layer proessing using our DBP and higher layer (MAC layer, network layer, et.) resoure alloation and oordination with CoMP shemes. In addition, we propose more sophistiated algorithms that enable superior error-rate performane ompared to the methods in [19], [0]. ) Cloud radio aess networks (C-RAN): The idea behind C-RAN is to separate the BS into two modules, a remote radio head (RRH) and a baseband unit (BBU), whih are onneted via high-bandwidth interonnet. The RRHs are plaed near the mobile users within the ell, the BBUs are grouped into a BBU pool for entralized proessing and loated remotely from RRH [1] [3]. C-RAN and CoMP both oordinate data transmission among multiple ells but with different physial realizations. CoMP integrates eah pair of radio heads with assoiated BBU together and allows low-lateny data transfer between eah radio head and its orresponding BBU. Different BBUs are separately plaed aross multiple ells, entailing long lateny on oordination among BBUs. C-RAN, in ontrast, shifts the BBU oordination lateny in CoMP to the data transfer lateny between RRHs and BBUs, sine BBUs are now grouped in a pool and an oordinate effiiently. Therefore, whether CoMP or C-RAN is more appropriate depends on whether BBU oordination or RRH-BBU data transfer is more effiient in a real-world deployment. Analogously to CoMP, we ould integrate DBP together with C-RAN to exploit the benefits of both tehnologies. For example, eah RRH now an be a large-sale antenna array (requiring higher RRH-BBU interonnetion bandwidth). The assoiated BBU itself may rely on DBP and perform our algorithms to resolve intra-ell multi-user interferene, while oordinating with other BBUs for inter-ell interferene mitigation. 3) Distributed proessing aross frequenies: Existing testbeds, suh as the LuMaMi testbed [8], [4] and the BigStation [9], distribute the baseband proessing aross frequenies. The idea is to divide the total frequeny band into lusters of subarriers in orthogonal frequeny-division multiplexing (OFDM) systems where eah frequeny luster is proessed onurrently, enabling high degrees of parallelism [8], [9], [4]. Unfortunately, eah frequeny luster still needs aess to all BS antennas, whih may result in high interonnet bandwidth. Furthermore, the frequeny band must somehow be divided either using analog or digital iruitry, and frequeny deentralization prevents a straightforward use of other waveform andidates, suh as single-arrier frequeny-division multiple aess (SC-FDMA), filter bank multi-arrier (FBMC), and generalized frequeny division multiplexing (GFDM) [5]. In ontrast, our DBP arhiteture performs deentralization aross antennas, whih is ompatible to most waveforms and requires data transmission only between a subset of antennas and the lusters. We emphasize, however, that DBP an be used together with frequeny deentralization in fat, our referene GPU implementation results shown in Setion VI exploit spatial deentralization and frequeny parallelism. D. Contributions We propose DBP to redue the raw baseband and hip I/O bandwidths, as well as the signal-proessing bottleneks of massive MU-MIMO systems that perform entralized baseband proessing. Our main ontributions are as follows: We propose DBP, an novel arhiteture for salable, FDDbased massive MU-MIMO BS designs, whih distributes omputation aross lusters of antennas. We develop two deentralized algorithms for near-optimal data detetion in the massive MU-MIMO uplink; both algorithms trade off error-rate performane vs. omplexity. We develop a deentralized beamforming algorithm for the massive MU-MIMO downlink. We perform a simulation-based tradeoff analysis between error-rate performane, onsensus data rate, and omputational omplexity for the proposed deentralized data detetion and beamforming algorithms. We present implementation results for data detetion and beamforming on a GPU luster that showase the effiay and salability of the proposed DBP approah. Our results demonstrate that DBP enables modular and salable BS designs for massive MU-MIMO with thousands of antenna elements while avoiding exessively high baseband and I/O data rates and signifiantly reduing the high omputational omplexity of onventional entralized algorithms. E. Notation Lowerase and upperase boldfae letters designate olumn vetors and matries, respetively. For a matrix A, we indiate its transpose and onjugate transpose by A T and A H respetively. The M M identity matrix is denoted by I M and the M N all-zeros matrix by 0 M N. Sets are denoted by upperase alligraphi letters; the ardinality of the A is denoted by A. The real and imaginary parts of a omplex salar a are R{a} and I{a}, respetively. The Kroneker produt is and E[ ] denotes expetation. F. Paper Outline The rest of the paper is organized as follows. Setion II details the DBP arhiteture and introdues the assoiated uplink and downlink system models. Setion III proposes two deentralized data detetion algorithms. Setion IV proposes the deentralized beamforming algorithm. Setion V provides performane and omplexity results. Setion VI summarizes our GPU luster implementation results. We onlude in Setion VII. All proofs are relegated to Appendix A. II. DBP: DECENTRALIZED BASEBAND PROCESSING We now detail the DBP arhiteture illustrated in Figure 1 and the system models for the uplink and downlink. We onsider a TDD massive MU-MIMO system and we assume a suffiiently long oherene time, i.e., the hannel remains onstant during both the uplink and downlink phases. In what follows, we fous on narrowband ommuniation; a generalization to wideband ommuniation is straightforward.

4 4 A. Uplink System Model and Arhiteture 1) Uplink system model: In the uplink phase, U singleantenna 1 user terminals ommuniate with a BS having B U antenna elements. Eah user enodes its own information bit stream using a forward error orretion (FEC) ode and maps the resulting oded bit stream to onstellation points in the set O (e.g., 16-QAM) using a predefined mapping rule (e.g., Gray mappings). At eah user, the resulting onstellation symbols are then modulated and transmitted over the wireless hannel (subsumed in the blok in Figure 1). The transmit symbols s u, u = 1,, U, of all U users are subsumed in the uplink transmit vetor s u O U. The baseband-equivalent inputoutput relation of the (narrowband) wireless uplink hannel is modeled as y u = H u s u + n u, where y u C B is the reeived uplink vetor, H u C B U is the (tall and skinny) uplink hannel matrix, and n u C B is i.i.d. irularly-symmetri omplex Gaussian noise with variane N 0 per omplex entry. The goal of the BS is to estimate the transmitted ode bits given (approximate) knowledge of H u and the reeived uplink vetor y u. This information is then passed to the deoder, whih omputes estimates for the data bits of eah user. ) Deentralized arhiteture: Consider the left-hand side (LHS) of Figure 1. The proposed DBP arhiteture partitions the reeive vetor y into C lusters so that (y u ) T = [(y u 1 ) T,, (y u C )T ] with y u C B and B = C =1 B. As mentioned in Setion I-B, we assume lusters of equal size and set S = B. By partitioning the uplink hannel matrix (H u ) T = [(H u 1) T,, (H u ) T ] row-wise into bloks of dimension H u C B U, = 1,, C, and, analogously, the noise vetor as (n u ) T = [(n u 1) T,, (n u C )T ], we an rewrite the uplink input-output relation at eah luster as follows: y u = H u s u + n u, = 1,, C. (1) The goal of DBP in the uplink is to ompute an estimate for s u in a deentralized manner: eah luster only has aess to y u, H u, and onsensus information (see Setion III). As shown in LHS of Figure 1, eah antenna element is assoiated to loal proessing iruitry; this inludes analog and digital filtering, amplifiation, mixing, modulation, et. As a onsequene, all required digital proessing tasks (e.g., used for OFDM proessing) are also arried out in a deentralized manner. Even though we onsider perfet synhronization and impairment-free transmission (suh as arrier frequeny and sampling rate offsets, phase noise, or I/Q imbalane), we note that eah luster and the assoiated proessing iruitry would be able to separately ompensate for suh hardware non-idealities with well-established methods [6]. This key property signifiantly alleviates the hallenges of perfetly synhronizing the loks and osillators among the lusters. 3) Channel estimation: During the training phase, eah luster must aquire loal CSI, i.e., ompute an estimate of H u. To this end, U orthogonal pilots are transmitted from the users prior to the data transmission phase. Sine eah luster 1 A generalization to multi-antenna user terminals is straightforward but omitted for the sake of simpliity of exposition. Other partitioning shemes may be possible. A study of alternative partitioning shemes is left for future work. has aess to y u, it follows from (1) that the assoiated loal hannel matrix H u an be estimated per luster. The estimate for the hannel matrix (as well as y u ) is then stored loally at eah luster and not made aessible to the other lusters; this prevents a bandwidth-intensive broadast of CSI (and reeive vetor data) to all lusters during the training phase. 4) Data detetion: During the data transmission phase, deentralized data detetion uses the reeive vetor y u, the assoiated CSI H u, and onsensus information to generate an estimate of the transmitted data vetor s u. This estimate is then passed to the deoder whih omputes estimates for the information bits of eah user in a entralized manner; suitable data detetion algorithms are proposed in Setion III. B. Downlink System Model and Arhiteture 1) Downlink system model: In the downlink phase, the B BS antennas ommuniate with the U B single-antenna user terminals. The information bits for eah user are enoded separately using a FEC. The BS then maps the resulting (independent) oded bit streams to onstellation points in the alphabet O to form the vetor s d O U. To mitigate MUI, the BS performs beamforming (BF), i.e., omputes a BF vetor x d C B that is transmitted over the downlink hannel. Beamforming requires knowledge of the (short and wide) downlink hannel matrix H d C U B and the transmit vetor s d O U to ompute a BF vetor that satisfies s d = H d x d (see Setion IV for the details). By assuming hannel reiproity, we have the property H d = (H u ) T [3], [4], whih implies that the hannel matrix estimated in the uplink an be used in the downlink. The baseband-equivalent input-output relation of the (narrowband) wireless downlink hannel is modeled as y d = H d x d + n d, where y d C U is the reeive vetor at all users and n d C U is i.i.d. irularly-symmetri omplex Gaussian noise with variane N 0 per omplex entry. By transmitting x d over the wireless hannel, the equivalent inputoutput relation is given by y d = s d + n d and ontains no MUI. Eah of the users then estimates the transmitted ode bits from yu, d u = 1,, U. This information is passed to the deoder, whih omputes estimates for the user s data bits. ) Deentralized arhiteture: Consider the right-hand side (RHS) of Figure 1. Sine the partitioning of the BS antennas was fixed for the uplink (f. Setion II-A), the BF vetor x must be partitioned into C lusters so that (x d ) T = [(x d 1) T,, (x d C )T ] with x d C B. By using reiproity and the given antenna partitioning, eah luster has aess to only H d = (H u ) T. With this partitioning, we an rewrite the downlink input-output relation as follows: y d = C =1 Hd x d + n d () The goal of DBP in the downlink is to ompute all loal BF vetors x d, = 1,, C in a deentralized manner: eah luster has aess to only s, H d, and onsensus information (see Setion IV for more details). As shown in the RHS of Figure 1, eah antenna element is assoiated to loal proessing iruitry. Analogously to the uplink, the required analog and digital signal proessing tasks (e.g., used for OFDM modulation or impairment ompensation)

5 5 an be arried out in a deentralized manner, whih alleviates the hallenges of perfetly synhronizing the lusters. 3) Beamforming: In the downlink phase, deentralized BF uses the transmit vetor s, deentralized CSI H d, and onsensus information in order to generate BF vetors x d that satisfy s = C =1 Hd x u. This ensures that transmission of the vetors x d removes MUI; a suitable algorithm is detailed in Setion IV. III. DECENTRALIZED UPLINK: DATA DETECTION We now propose two deentralized data detetion algorithms for the massive MU-MIMO uplink. We start by disussing the general equalization problem and then, detail our novel ADMM and CG-based data detetion algorithms. To simplify notation, we omit the uplink supersript u in this setion. A. Equalization-Based Data Detetion In order to arrive at omputationally effiient algorithms for deentralized data detetion, we fous on equalization-based methods. Suh methods ontain an equalization stage and a detetion stage. For the equalizations stage, we are interested in solving the following equalization problem (E0) ˆx = arg min s C U g(s) + 1 y Hs in a deentralized manner. Here, the funtion g(s) is a onvex (but not neessarily smooth or bounded) regularizer, whih will be disussed in detail below. For the detetion stage, the result ˆx of the equalization problem (E0) an either be slied entry-wise to the nearest onstellation point in O to perform hard-output data detetion or used to ompute approximate soft-output values e.g., log-likelihood ratio (LLR) values [7]. For zero-foring (ZF) and minimum mean-squared error (MMSE) data detetion, we set the regularizer to g(s) = 0 and g(s) = N 0 /E s s, respetively, where E s = E[ s ] is the expeted per-user transmit energy. 3 The generality of the equalization problem (E0) also enompasses more powerful data detetion algorithms. In partiular, we an set g(s) = χ(s C), where χ(s C) is the harateristi funtion that is zero if s is in some onvex set C and infinity otherwise. Speifially, to design data-detetion algorithms that outperform ZF or MMSE data detetion, we an use the onvex polytope around the onstellation set O, whih is given by { O C = i=1 α is i (α i 0, i) } O i=1 α i = 1. For QPSK with O = {±1 ± i}, the onvex set C is simply a box with radius 1 (i.e., side length of ) entered at the origin. In this ase, (E0) orresponds to the so-alled box-onstrained equalizer [8] whih was shown reently to (often signifiantly) outperform ZF or MMSE data detetion [9]. In addition, boxonstrained equalization does not require knowledge of the noise variane N 0, whih is in stark ontrast to the traditional MMSE equalizer. The deentralized equalization algorithm proposed next enables the use of suh powerful regularizers. 3 For the sake of simpliity, we assume an equal transmit power at eah user. An extension to the general ase is straightforward. B. Deentralized Equalization via ADMM To solve the equalization problem (E0) in a deentralized fashion, we make use of the ADMM framework [30]. We first introdue C auxiliary variables z = s, = 1,, C, whih allow us to rewrite (E0) in the equivalent form (E0 ) ˆx = arg min g(s) + C =1 1 y H z. s C U, z =s, =1,...,C Note that the added onstraints in (E0 ) enfore that eah loal vetor z agrees with the global value of s. As detailed in [30], these onstraints an be enfored by introduing Lagrange multipliers {λ } C =1 for eah luster, and then omputing a saddle point (where the augmented Lagrangian is minimal for s and z, and maximal for λ) of the so-alled saled augmented Lagrangian funtion, whih is defined as L(s, z, λ) = g(s) + C =1 { 1 y H z + ρ s z λ for some fixed penalty parameter ρ > 0. Here, we stak all C auxiliary variables into the vetor z T = [z T 1 z T C ] and stak all C saled Lagrange multiplier vetors into the vetor λ T = [λ T 1 λ T C ], where z, λ C U. The saddle-point formulation of (E0 ) is an example of a global variable onsensus problem [30, Se. 7.1] and an be solved using ADMM. We initialize s (1) = 0 U 1 and λ (1) = 0 U 1 for = 1,, C, and arry out the following iterative steps: (E1) z (t+1) 1 = arg min y H z + ρ s (t) z λ (t) z C U (E) s (t+1) = arg min g(s) + C =1 1 (t+1) s z λ (t) s C U (E3) λ (t+1) = λ (t) γ ( s (t+1) z (t+1) for the iterations t = 1,, until onvergene or a maximum number T max of iterations has been reahed. The parameter ρ > 0 ontrols the step size and γ = 1 is a typial hoie that guarantees onvergene. See [31] for a more detailed disussion on the onvergene of ADMM. Steps (E1) and (E) an be arried out in a deentralized manner, i.e., eah luster = 1,, C only requires aess to loal variables and loal hannel state information, as well as the onsensus vetors s (t) and s (t+1). Step (E) updates the onsensus vetor. While the vetors {z (t+1) } and {λ (t) } for every luster appear in (E), it is known that this an be omputed using only the global average of these vetors, whih is easily stored on a fusion node [30]. The arhiteture proposed in Setion II an ompute these averages and perform this update in an effiient manner. We next disuss the key details of the proposed deentralized data detetion algorithm. C. ADMM Algorithm Details and Deentralization 1) Step (E1): This step orresponds to a least-squares (LS) problem that an be solved in losed form and independently on eah luster. For a given luster, we an rewrite the ) }

6 6 minimization in Step (E1) in more ompat form as [ ] [ ] z (t+1) = arg min y H z C ρ(s (t) λ (t) z U ) ρ IU whih has the following losed-form solution: Here, y reg z (t+1) = y reg + ρb 1 (s (t) λ (t) ). (3) = B 1 H H y is the regularized estimate with B 1 = (H H H + ρi U ) 1. To redue the amount of reurrent omputations, we an preompute B 1 and reuse the result in eah iteration. For situations where the luster size S is smaller than the number of users U, we an use the Woodbury matrix identity [3] to derive the following equivalent update: z (t+1) = y reg + (I U H H A 1 H )(s (t) λ (t) ). (4) Here, y reg = H H A 1 y is a regularized estimate of the transmit vetor with A 1 = (H H H + ρi S ) 1. This requires the inversion of an S S matrix, whih is more easily omputed than the U U inverse required by (3). We note that whether (3) or (4) leads to lower overall omputational omplexity depends on U, S, and the number of ADMM iterations (see Setion V-A). ) Step (E): This step requires gathering of loal omputation results, averaging the sum in a entralized manner, and distributing the averaged onsensus information. To redue the data that must be exhanged, eah luster only ommuniates the intermediate variable w (t) = z (t+1) + λ (t), and only the average of these vetors is used on the fusion node. This simplifiation is aomplished using the following lemma; a proof is given in Appendix A-A. Lemma 1. The problem in Step (E) simplifies to s (t+1) = arg min g(s) + C s v (t) (5) s C U with v (t) = 1 C w(t) = 1 C C =1 w(t) and w (t) = z (t+1) + λ (t). Computation of (5) requires two parts. The first part orresponds to a simple averaging proedure to obtain v (t), whih an be arried out via sum redution in a tree-like fashion followed by entralized averaging. The seond part is the minimization in (5) that is known as the proximal operator for the funtion g(s) [33]. For ZF, MMSE, and box-onstrained equalization with QAM alphabets, the proximal operator has the following simple losed-form expressions: (E-ZF) s (t+1) = v (t) (E-MMSE) s (t+1) = CEs N 0+CE s v (t) (E-BOX) s (t+1) u = sgn(r{v u (t) }) min{ R{v u (t) }, r} + i sgn(i{v u (t) }) min{ I{v u (t) }, r} for u = 1,, U. Here, (E-BOX) is the orthogonal projetion of the vetor v (t) onto the hyperube with radius r that overs the QAM onstellation. For BPSK, the proximal operator orresponds to the orthogonal projetion onto the line [ r, +r] given by s (t+1) u = sgn(r{v u (t) }) min{ R{v u (t) }, r}, u = 1,, U. After omputation of (5), the onsensus vetor s (t+1) needs, Algorithm 1 Deentralized ADMM-based Data Detetion 1: Input: y, H, = 1,,, C, ρ, γ, N 0, and E s : Preproessing: 3: if S U then 4: A 1 = (H H H + ρ 1 I S) 1 5: y reg 6: else = H H A 1 y 7: B 1 = (H H H + ρ 1 I U ) 1 8: y reg = B 1 H H y 9: end if 10: ADMM iterations: 11: Init: λ (1) = 0, z (1) = y reg, s (1) = ( N 0 E s 1: for t =, 3,, T max do 13: λ (t) = λ (t 1) + γ(z (t 1) s (t 1) ) 14: if S U then 15: z (t) = y reg 16: else 17: z (t) 18: end if 19: w (t) = y reg = z (t) + (s (t 1) λ (t) + ρb 1 (s (t 1) λ (t) ) + λ (t) 0: w (t) = C =1 w(t) 1: s (t) = (N 0/E s + C) 1 w (t) : end for 3: Output: ˆx = s (Tmax) + C) 1 ( C =1 z(1) ) ) H H A 1 H (s (t 1) λ (t) // Consensus to be distributed to all C lusters. In pratie, we distribute w (t) as soon as it is available, and the saling steps to get s (t+1) from w (t) are omputed loally on eah luster after it reeives w (t). This way no luster waits for the omputation of s (t+1) on a entral/master worker (fusion node) before ADMM iterations proeed. 3) Step (E3): This step an be arried out independently in eah luster after s (t+1) has been alulated. We summarize the resulting deentralized ADMM proedure for MMSE equalization in Algorithm 1. The equalization output is simply the onsensus vetor s (Tmax). Note that Algorithm 1 slightly deviates from the proedure outlined in Setion III-B. We will analyze the algorithm omplexity 4 in Setion V-A; a GPU luster implementation will be disussed in Setion VI. D. Deentralized Equalization via Conjugate Gradients If the regularization funtion g(s) of (E0) is quadrati, as in the ase for MMSE equalization where g(s) = N 0 /E s s, then we an solve (E0) with an effiient deentralized onjugate gradient (CG) method [34] [36]. Our method builds on the CG algorithm used in [34] for entralized equalization-based data detetion in massive MU-MIMO systems. Our idea is to break all entralized omputations that rely on global CSI and reeive data (i.e., H and y) into smaller, independent problems that only require loal CSI and reeive data (H and y ). The entralized CG-based detetor in [34] involves two stages: a preproessing stage for alulating the MRC output y MRC and a CG iteration stage to estimate ˆx. 4 The operataion H H A 1 H on line 15 of Algorithm 1 ould be omputed one in a preproessing stage to avoid reurrent omputations during the iterations. Instead, in Algorithm 1 we diretly ompute H H A 1 H (s (t 1) λ (t) ) in eah iteration beause this approah requires only three matrix-vetor multipliations per iteration; preomputing H H A 1 H requires two ostly matrix-matrix multipliations. Hene, our omplexity analysis in Setion V-A refers to the proedure detailed in Algorithm 1. )

7 7 Algorithm Deentralized CG-based Data Detetion 1: Input: H, = 1,, C, and y, and ρ : Preproessing: 3: y MRC = H H y // Deentralized 4: y MRC = C =1 ymrc // Centralized 5: CG iterations: 6: Init: r (0) = y MRC, p (0) = r (0), x (0) = 0 7: for t = 1,, T max do 8: Deentralized (eah luster performs the same operation): 9: w (t) = H H H p (t 1) 10: Centralized (onsensus on a entralized proessing unit): 11: w (t) = =1 w(t) // Consensus 1: Deentralized (eah luster performs the same operations): 13: e (t) = ρp (t 1) + w (t) 14: α = r (t 1) /((p (t 1) ) H e (t) ) 15: x (t) = x (t 1) + αp (t 1) 16: r (t) = r (t 1) αe (t 1) 17: β = r (t) / r (t 1) 18: p (t) = r (t) + βp (t 1) 19: end for 0: Output: ˆx = x (Tmax) In the preproessing stage, we rewrite the MRC vetor y MRC = H H y as y MRC = C =1 HH y, whih deentralizes the preproessing stage. Speifially, eah luster omputes H H y ; the results of eah luster are then summed up in a entralized manner to obtain the MRC output y MRC. For the CG iteration stage, we need to update the estimated transmit vetor and a number of intermediate vetors required by the CG algorithm (see [34] for the details). While most operations are not diretly dependent on global CSI H but on intermediate results, the update of the following vetor e (t) = ( ρi + H H H ) p (t 1), (6) requires diret aess to the global hannel matrix H and thus, must be deentralized. Here, ρ = N 0 /E s for MMSE equalization and ρ = 0 for zero-foring equalization. It is key to realize that the Gram matrix an be written as H H H = C =1 HH H. Hene, we an reformulate (6) as e (t) = ρp (t 1) + C =1 HH H p (t 1). (7) Put simply, by loally omputing w (t) = H H H p (t 1) at eah antenna luster, we an obtain the result in (7) by performing the following entralized omputations that do not require global CSI: w (t) = C =1 w(t) and e (t) = ρp (t 1) + w (t). The deentralized CG-based data detetion algorithm is summarized in Algorithm. The omputations of e (t), x (t), r (t), and p (t) do not require aess to the (global) hannel matrix H and an be arried out in a entralized proessing unit. We must, however, broadast the vetor p (t) to eah antenna luster before the deentralized update of w (t+1) in the next iteration an take plae. Alternatively, we an diretly broadast the onsensus vetor w (t), so that eah antenna luster an simultaneously ompute their own opy of e (t), x (t), r (t), and p (t) in a deentralized manner to ensure the loal existene of p (t) for updating w (t+1). With this alternative approah, we an ompletely shift the omplexity from the entralized proessing unit to the loal proessing units, leaving the alulation of w (t) as the only entralized omputation in a CG iteration. This approah also enables the onatenation of data gathering and broadasting, whih an be implemented using a single message-passing funtion (Setion VI). 5 IV. DECENTRALIZED DOWNLINK: BEAMFORMING We now develop a deentralized beamforming algorithm for the massive MU-MIMO downlink. We start by disussing the general beamforming (or preoding) problem, and then detail our ADMM-based beamforming algorithm. To simplify notation, we omit the downlink supersript d. A. Beamforming Problem We solve the following beamforming problem (P0) ˆx = arg min x subjet to s Hx ε. x C B whih aims at minimizing the instantaneous transmit energy while satisfying the preoding onstraint s Hx ε. By defining the residual interferene as e = s Hˆx, we see that transmission of the solution vetor ˆx of (P0) leads to the input-output relation y = s + e + n with e ε. Hene, eah user only sees their dediated signal ontaminated with Gaussian noise n and residual interferene e, whose energy an be ontrolled by the parameter ε 0. By setting ε = 0, this problem has a well-known losed-form solution and orresponds to the so-alled zero-foring (ZF) beamformer, whih is given by ˆx = H H (HH H ) 1 s assuming that U B and H is full rank. Our goal is to develop an algorithm that omputes the solution of (P0) in a deentralized fashion. B. Deentralized Beamforming via ADMM By introduing C auxiliary variables z = H x, = 1,, C, we an rewrite (P0) in the following equivalent form: (P0 ) ˆx = arg min x subjet to C s =1 z ε x C B and z = H x, = 1,, C. Here, H is the downlink hannel matrix at luster. The solution to the onstrained preoding problem (P0 ) orresponds to a saddle point of the saled augmented Lagrangian funtion: L(s, z, λ) = 1 x + C ρ =1 H x z λ + X (z), where X (z) is the harateristi funtion for the onvex onstraint of the preoding problem (P0), i.e., X (z) = 0 if s C =1 z ε and X (z) = otherwise. The problem (P0) orresponds to a sharing onsensus problem with regularization [30, Se. 7.3]. In order to arrive at a deentralized preoding algorithm, we now use the ADMM framework to find a solution to (P0 ). We initialize 6 z (1) = max{u/b, 1/C}s. We then perform the 5 The Gram matrix G = H H H an be preomputed to avoid reurrent omputations (line 9 in Algorithm ). However, pratial systems only need a small number of CG iterations, and H H H p (t 1) at line 9 is omputed using two matrix-vetor multipliations, whih avoids the expensive matrix-matrix multipliation needed to form G. 6 This initializer is a properly-saled version of the MRC beamforming vetor and exhibits exellent error-rate performane in pratie.

8 8 following three-step proedure until onvergene or a maximum number of iterations has been reahed: (P1) x (t+1) 1 =arg min x + ρ H x z (t) λ (t) x C S C (P) z (t+1) ρ =arg min H x (t+1) z λ (t) +X (z) z C U =1 (P3) λ (t+1) =λ (t) γ ( H x (t+1) z (t+1) ). Here, z is the loal beamforming output and z is the onsensus solution of (P). The parameter ρ > 0 affets the step size and γ = 1 ensures onvergene of the algorithm. While both the Steps (P1) and (P3) an effiiently be omputed in a deentralized manner, it is not obvious how Step (P) an be deentralized. We next show the details to transform Step (P) into a form that requires simple global averaging. C. ADMM Algorithm Details and Deentralization 1) Step (P1): Analogous to Step (E1), this step orresponds to a LS problem that an be solved in losed form and independently in every luster. For a given luster = 1,, C, we an rewrite the minimization in (P1) as x (t+1) = arg min x C S [ ρ(z (t) 0 S 1 + λ (t) ) ] whih has the following losed-form solution: x (t+1) = A 1 H H (z (t) + λ (t) ). [ ] ρh I S x, Here, A 1 = (H H H + ρ 1 I S ) 1 requires the omputation of an S S matrix inverse. If the luster size S is larger than the number of users U, then we an use the Woodbury matrix identity [3] to derive the following equivalent update: x (t+1) = H H B 1 (z (t) + λ (t) ). Here, B 1 = (H H H + ρ 1 I U ) 1 requires the omputation of an U U matrix inverse. We note that U, S, and the number of iterations an determine whih of the two x (t+1) variations leads to lower overall omputational omplexity. ) Step (P): The presene of the indiator funtion X (z) makes it non-obvious whether this step indeed an be arried out in a deentralized fashion. The next results shows that a simple averaging proedure analogously to that used in Step (E1) for deentralized data detetion an be arried out to perform Step (E); the proof is given in Appendix A-B. Lemma. The minimization in Step (P) simplifies to { } z (t+1) = w (t) ε ( + max 0, 1 1 s v (t) C s v(t)) (8) with v (t) = 1 C w(t) = 1 C C =1 w(t) ; w (t) = H x (t+1) λ (t). For ε = 0, we get an even more ompat expression z (t+1) = w (t) + 1 C s v(t), = 1,, C Evidently (8) only requires a simple averaging proedure, whih an be arried out by gathering loal omputation results from and broadasting the averaged onsensus bak to eah luster. Algorithm 3 Deentralized ADMM-based Beamforming 1: Input: s, H, = 1,,, C, ρ, and γ : Preproessing: 3: if S U then 4: A 1 = (H H H + ρ 1 I S) 1 5: else 6: B 1 = (H H H + ρ 1 I U ) 1 7: end if 8: ADMM iterations: 9: Init: z (1) = max{u/b, 1/C}s, λ (1) = A 1 H H z (1) = 0 10: x (1) (S U) or H H B 1 z (1) (S > U) 11: for t =, 3,, T max do 1: m (t 1) = H x (t 1) 13: w (t 1) = m (t 1) λ (t 1) // Consensus 15: z (t) + C 1 (s w (t 1) ) 16: λ (t) = λ (t 1) γ(m (t 1) z (t) ) 17: if S U then 18: x (t) = A 1 H H (z (t) + λ (t) ) 19: else 0: x (t) = H H B 1 (z (t) + λ (t) ) 14: w (t 1) = C =1 w(t 1) = w (t 1) 1: end if : end for 3: Output: ˆx = [x (Tmax) 1 ; x (Tmax) ; ; x (Tmax) C ] 3) Step (P3): This step an be performed independently in eah luster after distributing w (t) and getting loal z (t+1). The resulting ADMM-based deentralized beamforming proedure is summarized in Algorithm 3, where we assume ε = 0. To failitate implementation of the deentralized beamforming algorithm, we initialize z (1), λ (1), x (1) and then update the variables in the order of z (t), λ (t), x (t) realizing that the final output of the loal beamformer is simply x (Tmax). Note that Algorithm 3 slightly deviates from the step-by-step preoding proedure in Setion IV-B. We will analyze the algorithm omplexity 7 in Setion V-A and show the referene implementation of Algorithm 3 in Setion VI. Remark 1. Although we propose a deentralized sheme using CG for uplink data detetion in Setion III-D, a similar deentralization method of CG is not appliable in the downlink. Sine we partition the uplink hannel matrix H row-wise into C bloks, we should similarly partition the downlink hannel matrix olumn-wise into bloks due to the hannel reiproity; this prevents an expansion analogously to (7). Consequently, we fous exlusively on ADMM-based beamforming. V. RESULTS We now analyze the omputational omplexity and onsensus bandwidth of our proposed algorithms. We also show error-rate simulation results in LTE-like massive MU-MIMO uplink and downlink systems. We investigate the performane/omplexity trade-offs and show pratial operating points of our deentralized methods under various antenna onfigurations, providing design guidelines for deentralized massive MU-MIMO BSs. 7 The matries P = A 1 H H or P = HH B 1 ould be preomputed to avoid reurrent omputations within the ADMM iterations (at line 18 or 0 in Algorithm 3). Instead, we diretly ompute A 1 H H (z (t) + λ (t) ) or H H B 1 (z (t) + λ (t) ), whih only requires two matrix-vetor multipliations; preomputing P requires ostly matrix-matrix multipliations. Hene, our omplexity analysis in Setion VI refers to Algorithm 3.

9 9 A. Computational Complexity In Table I, we list the number of real-valued multipliations 8 of our deentralized ADMM-based downlink beamforming (ADMM-DL), ADMM-based uplink detetion (ADMM-UL) and CG-based uplink detetion (CG-UL) algorithms. We also ompare the omplexity to that of onventional, entralized ZF downlink preoding (ZF-DL) and MMSE uplink detetion (MMSE-UL). For all deentralized algorithms and modes, for example, the S S mode when S U and the U U mode when S > U, we show the timing (TM) omplexity and arithmeti (AR) omplexity. We assume that the entralized omputations take plae on a entralized PE while deentralized omputations are arried out on multiple deentralized PEs. For the entralized omputations, both the TM and AR omplexities ount the number of real-valued multipliations on the entralized PE. For the deentralized operations, the TM omplexity only ounts operations that take plae on a single loal proessing unit where all deentralized loal proessors perform their own omputations in parallel at the same time, thus refleting the lateny of algorithm exeution; in ontrast, the AR omplexity ounts the total omplexity aumulated from all loal proessing units, thus refleting the total hardware osts. The omplexity of our methods depends on the number of lusters C, the number of users U, the number of BS antennas S per antenna luster, and the number of iterations T max to ahieve satisfatory errorrate performane. We also divide the omplexity ounts into three parts: preproessing before ADMM or CG iterations, first iteration, and subsequent iterations. The omplexity in the first iteration is typially lower as many vetors are zero. Table I reveals that preproessing for ADMM exhibits relatively high omplexity, whereas CG-based detetion is omputationally effiient. The per-iteration omplexity of ADMM is, however, extremely effiient (depending on the operation mode). Overall, CG-based data detetion is more effiient than the ADMM-based ounterpart, whereas the latter enables more powerful regularizers. Centralized ZF or MMSE beamforming or detetion, respetively, require high omplexity, i.e., saling with U 3, but generally ahieve exellent errorrate performane [4]. We will analyze the trade-offs between omplexity and error-rate performane in Setion V-D. B. Consensus Bandwidth The amount of data passed between the entralized proessing unit and the deentralized loal units during ADMM or CG iterations sales with the dimension of the onsensus vetor w (t). For a single subarrier, w (t) is a vetor with U omplexvalued entries. If we perform detetion or preoding for a total of N CR subarriers, then in eah ADMM or CG iteration, we need to gather U N CR omplex-valued entries from eah loal proessing unit for onsensus vetors orresponding to N CR subarriers, and broadast all N CR onsensus vetors to eah loal proessor afterwards. Suh a small amount of data exhange relaxes the requirement on interonnetion bandwidth 8 We ignore data-dependenies or other operations, suh as additions, division, et. While this omplexity measure is rather rude, it enables fundamental insights into the pros and ons of deentralized baseband proessing. among deentralized PEs, and avoids the large data transfer between the entire BS antenna array and BS proessor in a onventional entralized BS. However, as we will show with our GPU luster implementation in Setion VI, the interonnet lateny of the network ritially effets the throughput of DBP. C. Error-Rate Performane We simulate our deentralized data detetion and beamforming algorithms in an LTE-based large-sale MIMO system. For both the uplink and downlink simulations, we onsider OFDM transmission with 048 subarriers in a 0 MHz hannel, and inorporate our algorithms with other neessary baseband proessing bloks, inluding 16-QAM modulation with Gray mapping, FFT/IFFT for subarrier mapping, rate-5/6 onvolutional enoding with random interleaving and Viterbibased hannel deoding. We generate the hannel matries using the Winner-II hannel model [37] and onsider hannel estimation errors, i.e., we assume a single orthogonal training sequene per user and ative subarrier. In Figure, we show the oded bit error-rate (BER) performane against average SNR per reeive antenna for deentralized ADMM detetion (Figure (a)), for deentralized CG detetion (Figure (b)) in the uplink, and for deentralized ADMM beamforming (Figure ()) in the downlink. We onsider various antenna onfigurations. We fix the number of users U = 16, and set S = 8 (for S U ase) or S = 3 (for S > U ase), and sale the total BS antenna number B = S C from 64 to 51 by hoosing C = 8 and C = 16. We see that for all the onsidered antenna and luster onfigurations, only -to-3 ADMM or CG iterations are suffiient to approah the performane of the linear MMSE equalizer. For the S > U ase, even a single ADMM iteration enables exellent BER performane for detetion and beamforming without resulting in an error floor. We note that the amount of onsensus information that must be exhanged during eah ADMM or CG iteration is rather small. Hene, our deentralized data detetion and beamforming algorithms are able to ahieve the BER of entralized solutions without resulting in prohibitive interonnet or I/O bandwidth this approah enables highly salable and modular BS designs with hundreds or thousands of antenna elements. D. Performane/Complexity Trade-off Analysis Figure 3 illustrates the trade-off between error-rate performane and omputational omplexity of our proposed methods. As a performane metri, we onsider the minimum required SNR to ahieve 1% BER; the omplexity is haraterized by the TM omplexity and depends on the number of ADMM or CG iterations (the numbers next to the urves). As a referene, we also inlude the BER performane of entralized MMSE data detetion and ZF beamforming (dashed vertial lines). For the uplink, Figures 3(a) and 3(b) show the trade-offs for ADMM-based and CG-based data detetion, respetively. We see that only a few CG iterations are suffiient to ahieve near- MMSE performane whereas ADMM requires a higher number of iterations to ahieve the same performane. CG-based data detetion exhibits the better trade-off here, and is the preferred