IN massive MIMO (MaMi) an unconventionally high number. The World s First Real-Time Testbed for Massive MIMO: Design, Implementation, and Validation

Transcription

1 he World s First Rel-ime estbed for Mssive MIMO: Design, Implementtion, nd Vlidtion Steffen Mlkowsky, Joo Vieir, Ling Liu, Pul Hrris 2, Krl Niemn 3, Nikhil Kundrgi 3, In Wong 3, Fredrik ufvesson, Viktor Öwll, nd Ove Edfors Dept. of Electricl nd Informtion echnology, Lund University, Sweden 2 ommuniction Systems nd Networks Group, University of Bristol, UK 3 Ntionl Instruments, Austin, exs, USA firstnme.lstnme@{eit.lth.se, bristol.c.uk, ni.com} rxiv:70.06v [cs.i] 20 Dec 206 Abstrct his pper sets up frmework for designing mssive multiple-input multiple-output (MIMO) testbed by investigting hrdwre (HW) nd system-level requirements such s processing complexity, duplexing mode nd frme structure. king these into ccount, generic system nd processing prtitioning is proposed which llows flexible scling nd processing distribution onto multitude of physiclly seprted devices. Bsed on the given HW constrints such s mximum number of links nd mximum throughput for peer-to-peer interconnections combined with processing cpbilities, the frmework llows to evlute vilble off-the-shelf HW components. o verify our design pproch, we present the LuMMi (Lund University Mssive MIMO) testbed which constitutes the first reconfigurble rel-time HW pltform for prototyping mssive MIMO. Utilizing up to 00 bse sttion ntenns nd more thn 50 Field Progrmmble Gte Arrys, up to 2 user equipments re served on the sme time/frequency resource using n LElike Orthogonl Frequency Division Multiplexing time-division duplex-bsed trnsmission scheme. Field trils with this system show tht mssive MIMO cn sptilly seprte multitude of users in sttic indoor nd sttic/dynmic outdoor environment. Index erms 5G, system design, testbed, outdoor mesurement, indoor mesurement, softwre-defined rdio, DD I. INRODUION IN mssive MIMO (MMi) n unconventionlly high number of bse sttion (BS) ntenns (hundreds or even higher) is employed to serve e.g., fctor of ten less user equipments (UEs). Due to the excess number of BS ntenns, liner signl processing my be used to sptilly focus energy with high precision, llowing to seprte multitude of UEs in the sptil domin while using the sme time/frequency resource []. MMi theory promises vriety of gins, e.g., increse in spectrl nd energy efficiencies [2], thereby tckling the key chllenges defined for 5G. Although MMi is promising theoreticl concept, further development requires prototype systems for proof-of-concept nd performnce evlution under rel-world conditions to identify ny further chllenges in prctice. Becuse of its importnce, both industry nd cdemi re mking efforts in building MMi testbeds, including the Argos testbed with 96-ntenns [3], Eurocom s 64-ntenn long-term evolution (LE) comptible testbed, Smsung s Full-Dimension (FD) MIMO testbed nd Fcebook s Project Aries. Nevertheless, publictions systemticlly describing the design considertion nd methodology of MMi testbed re missing nd reltime rel-scenrio performnce evlution of MMi systems using testbeds hve not been reported yet. At Lund University, the first rel-time MMi testbed, the Lund University MMi (LuMMi) testbed, showing successful MMi trnsmission on the up-link (UL), ws built [4]. Ever since, mny testbeds hve been constructed using similr rchitecture nd hrdwre, e.g., the MMi testbeds t the University of Bristol [5], Norwegin University of Science nd echnology in rondheim nd University of Leuven in Belgium. he LuMMi testbed provides fully reconfigurble pltform for testing MMi under rellife conditions. o build rel-time MMi testbed mny chllenges hve to be coped with. For exmple, shuffling dt from 00 or more ntenns, processing lrge-scle mtrices nd synchronizing huge number of physiclly seprted devices. All this hs to be mnged while still ensuring n overll reconfigurbility of the system llowing experimentl hrdwre nd softwre solutions to be tested rpidly. his pper discusses how implementtion chllenges re ddressed by first evluting high-level hrdwre (HW) nd system requirements, nd then setting up generic frmework to distribute the dt shuffling nd processing complexity in MMi system bsed on the given HW constrints for interconnection network nd processing cpbilities. Bsed on the frmework nd requirements, suitble off-the-shelf HW pltform is selected nd evluted. herefter, thorough description of the LuMMi testbed is provided. his includes system prmeters, bse-bnd processing fetures, synchroniztion scheme nd other detils. he LuMMi testbed constitutes flexible pltform tht supports prototyping of up to 00- ntenn 20 MHz bndwidth MMi, simultneously serving 2 UEs in rel-time using Orthogonl Frequency Division Multiplexing (OFDM) modultion in time-division duplex (DD) trnsmission mode. Bit Error Rtes (BERs) nd constelltions for rel-time UL nd down-link (DL) uncoded trnsmission in sttic indoor nd sttic/dynmic outdoor scenrio re presented. Our first rel-life mesurement cmpigns show, tht MMi is cpble of sptilly seprting up to 2 UEs in the sme time/frequency resource even for high user density per unit re, while cmpign with mobile UEs proved proper functionlity for speeds of up to 50 km/h. he gthered

2 2 where f eq ( ) constructs n pproprite equliztion mtrix. B. Down-link On the DL, ech UE receives its corresponding symbol û k which re collected in vector û (û,..., û K ), representing the symbols received by ll UEs. With this nottion, the received signl becomes û = Hx + w (3) Fig.. A MMi system model. Ech ntenn t the BS (left side) trnsmits liner combintion of K user-intended dt symbols u K kk=. After propgtion through the DL wireless chnnel B, ech user ntenn receives liner combintion of the signls trnsmitted by the M BS ntenns. Finlly, ech of the K users, sy user k, produces n estimte of its own intended dt symbol, i.e., u k. Similr opertion is employed for UL dt trnsmission. Here, reciprocity for the propgtion chnnel is ssumed, i.e., B = B. ABLE I LINEAR PREODING/DEEION SHEMES MR/MR ZF RZF DL G H G H (GG H ) G H (GG H + β regpre I K ) UL G H (G H G) G H (G H G + β regdec I K ) G H results suggest n increse in spectrl efficiency of t lest one order of mgnitude compred to current stndrds like LE. By building the LuMMi testbed we now hve tool which supports ccelerted design of lgorithms [6] nd their vlidtion bsed on rel mesurement dt, with the dditionl benefit of rel-world verifiction of digitl bse-bnd solutions. II. MASSIVE MIMO BASIS A simplified model of MMi BS using M ntenns while simultneously serving K single ntenn UEs in DD opertion in propgtion chnnel B is shown in Fig.. o simplify nottion, this discussion ssumes bse-bnd equivlent chnnel nd expressions re given per subcrrier, with subcrrier indexing suppressed throughout. A. Up-link he UL power levels used by the K UEs during trnsmission build the K K digonl mtrix P ul. By collecting the trnsmitted UE symbols in vector z (z,..., z K ), the received signls r (r,..., r M ) t the BS re described s r = G P ul z + w () where G is the UL chnnel, nd w N (0, I M ) is independent nd identiclly distributed (iid) circulrly-symmetric zeromen complex Gussin noise. he estimted user symbols ẑ (ẑ,..., zˆ K ) from the K UEs re obtined by liner filtering of the received vector r s ẑ = f eq (G)r, (2) G is the up-link rdio chnnel cpturing both, the propgtion chnnel B nd the up-link hrdwre trnsfer functions. where the K M mtrix H is the DL rdio chnnel 2, w N (0, I K ) is n iid circulrly-symmetric zero-men complex Gussin receive noise vector with covrince mtrix I K, nd x (x,..., x M ) is the trnsmit vector. As explicit DL chnnel estimtion is very resource consuming, it is not considered prcticl in MMi setup []. king into ccount tht the propgtion chnnel B is generlly greed on to be reciprocl [6], the estimted UL chnnel mtrix G cn be utilized to trnsmit on the DL. However, differences due to nlog circuitry in the UL nd DL chnnels, G nd H, need to be compensted. hus, possible construction for x is of the form x = f cl (f pre (G))u, (4) where u (u,..., u K ) is vector contining the symbols intended for the K UEs, f pre ( ) is some precoding function, nd f cl ( ) is reciprocity clibrtion function to be discussed next.. Reciprocity librtion In most prcticl systems, the UL nd DL chnnels re not reciprocl, i.e. G H. his is esily seen by fctorizing G nd H s G = R B B U, nd H = R U B B, (5) where digonl mtrices R B nd R U model the non-reciprocl hrdwre responses of BS nd UE receivers (RXs), respectively, nd digonl mtrices B nd U similrly model hrdwre responses of their trnsmitters (Xs). hus, in order to construct precoder bsed on the UL chnnel estimtes, the non-reciprocl components of the chnnel hve to be clibrted. Previous clibrtion work showed tht this is possible by using f pre (G) = f cl (f pre (G)), (6) where = R B B is the, so-clled, clibrtion mtrix which cn be estimted internlly t the BS [6]. Such clibrtion is sufficient to cncel inter-user interference stemming from nonreciprocity [7]. D. Liner detection & precoding schemes ble I shows selection of weighting mtrices used in liner precoding nd detection schemes, with non-reciprocity compenstion included in the form of the M M digonl mtrix s defined bove. he mximum rtio trnsmission 2 H is the down-link rdio chnnel cpturing both, the propgtion chnnel B nd the down-link hrdwre trnsfer functions.

3 3 ABLE II HIGH-LEVEL SYSEM PARAMEERS Prmeter Vrible Vlue Bndwidth W 20 MHz Smpling Rte F s MS/s FF Size N FF 2048 # Used subcrriers N used 200 yclic prefix N cp 44 smples OFDM symbol length t OFDM 7.4 µs (MR) precoder nd the mximum rtio combining (MR) decoder mximize rry gin without ctive suppression of interference mong the UEs []. he zero-forcing (ZF) precoder nd ZF combiner employ the pseudo-inverse, which provides inter-user interference suppression with the penlty of lowering the chievble rry gin. A scheme tht llows trde-off between rry gin nd interference suppression is the regulrized ZF (RZF) precoder nd RZF combiner. his is chieved by properly selecting the regulriztion constnts β regpre nd β regdec. If β regpre nd β regdec re selected to minimize men-squre error (MSE) E u ρ û 2, where ρ is scling constnt, we obtin the minimum MSE (MMSE) precoder/detector [8]. III. SYSEM DESIGN ASPES Hving discussed the MMi bsics, we move on to system design spects. hese include modultion scheme, frme structure nd hrdwre requirements. A. Modultion Scheme While mny different modultion schemes cn be used with MMi, this pper focuses on OFDM, employed in mny modern wireless communiction systems. Properly designed OFDM renders frequency-flt nrrowbnd subcrriers, fcilitting the single chnnel equliztion strtegy used here. For ese of comprison nd simplicity, LE-like OFDM prmeters, s shown in ble II, re used throughout this discussion. he more common prmeters with LE, the esier it is to evlute how MMi s n dd-on would influence current cellulr systems. B. DD versus FDD urrent cellulr systems either operte in frequency-division duplex (FDD) or DD mode. FDD is, however, considered imprcticl for MMi due to excessive resources needed for DL pilots nd chnnel stte informtion (SI) feedbck. DD opertion relying on reciprocity only requires orthogonl pilots in the UL from the K UEs, mking it the fesible choice [9]. For this reson, we focus entirely on DD below.. Reciprocity o llow opertion in DD mode, differences in the X nd RX trnsfer functions on both the BS nd UEs hve to be clibrted s discussed in Sec. II-. Drifts over time re minly cused by HW temperture nd voltge chnges, nd thus, the clibrtion intervl depends on the operting environment of the BS. D. Frme Structure he frme structure defines mong other things, the pilot rte. he ltter determines how well chnnel vritions cn be trcked nd, indirectly, the lrgest supported UE speed. ) Mobility: he mximum supportble mobility, e.g., the mximum speed of the UEs is defined by the UL pilot trnsmission intervl. Assuming 2D wide-sense sttionry chnnel with uncorrelted isotropic scttering nd tht the time correltion between consecutive chnnel estimtes shll not drop below 0.9 for proper functionlity, the mximum supported Doppler frequency, ν mx, cn be found by solving J 0 (2πν mx p ) = 0.9, (7) for ν mx, where J 0 ( ) is the zeroth-order Bessel function, of the first kind nd p the distnce between pilots in time. Hence, the mximum supportble speed of ny UE my be evluted using v mx = cν mx f c, (8) once specific frme structure is provided. In (8) v mx is the mximum supported speed of UE, c the speed of light nd f c the chosen crrier frequency. 2) Processing ltency: he frme structure lso limits the processing ltency in the criticl precoding turnround time, i.e., the time between reception of the lst UL pilot nd the trnsmission of DL dt. hus, DD trnsmission poses tight ltency requirements for the bse-bnd processing, especilly for high mobility scenrios. he DD precoder turnround time my be formulted s: = rf,x + rf,rx + OFDM + SI + precode + rout, (9) where rf,x nd rf,rx re the nlog front-end delys for X nd RX, respectively, OFDM the processing ltency for OFDM modultion/demodultion (including cyclic prefix (P) nd gurd bnd opertion), SI the time for SI cquisition, nd precode the processing ltency for precoding, including reciprocity compenstion. Additionl sources of ltency include overhed in dt routing, pcking, nd unpcking, i.e., rout. Depending on the specific rrngement of the OFDM symbols nd the pilot repetition pttern in the frme structure, bse-bnd processing solutions, i.e., SI nd precode, hve to be optimized to ensure no constrint violtions. 3) Pilot pttern: o cquire SI t the BS, the K UEs trnsmit orthogonl pilots on the UL. More specificlly, ech UE sends pilots on orthogonl subcrriers, i.e., ech UE uses every K-th subcrrier with the first UE strting t subcrrier 0, the second t subcrrier etc., overll utilizing full OFDM symbol. hnnel estimtes between pilot positions cn be obtined through interpoltion. Fig. 2 shows generic frme structure cpturing the forementioned spects in hierrchicl mnner. At the beginning of ech BS reciprocity cycle, reciprocity clibrtion t the BS is performed nd within these certin number of DL pilot cycles re encpsulted where precoded DL pilot symbols re trnsmitted. he length of the BS reciprocity cycle is determined by the stbility of the trnsceiver chins in the BS. As the reciprocity clibrtion t the BS side only compenstes

4 4 BS Rec. librtion BS Reciprocity ycle ABLE III PROESSING REQUIREMENS IN A MAMI SYSEM UL pilots required DL Pilots ontrol Lyer Dt Dt Dt ime Slots UL Pilot DL Pilot ycle ontrol ycle Function Generl Specific Gops/s Gops/s FF/IFF 4M log 2 (N FF )N FF /t OFDM 26 Detection 4MKN used /t OFDM 80 Precoding 4MKN used /t OFDM 80 Recip. l. 4MKN( used /t OFDM 80 Pseudo-inv. 4N used 2MK 2 + K 3) / (2t OFDM) 080 UL Dt OFDM OFDM OFDM DL Pilot DL Dt Switch Gurd ABLE IV DAA SHUFFLING REQUIREMENS IN A MAMI SYSEM Fig. 2. Generic frme structure of LE like DD-bsed MMi system. Within one BS reciprocity cycle the BS opertes using the sme reciprocity clibrtion coefficients. A certin number of DL pilot cycles re integrted s UEs suffer from fster chnging environments. Ech control cycle contins control lyer to perform, for exmple over-the-ir synchroniztion nd within these the dt trnsmission slots re encpsulted. for BS trnsceivers, DL pilots re necessry to compenste for trnsceiver differences t the UE side. heir frequency depends on the stbility t the UE side nd cn be considered significntly smller thn for the BS s UEs re subject to fster chnges in their opertionl environment, e.g., therml differences when hving the UE in pocket or using it indoors or outdoors. o be ble to send precoded pilots on the DL, trnsmission of UL pilots is required beforehnd. Severl control cycles re embedded inside ech DL pilot cycle crrying certin number of dt time slots. ime slots contin five different OFDM symbol types for physicl lyer implementtion. hese re (i) UL Pilot where the UEs trnsmit orthogonl pilots to the BS, (ii) UL Dt where ll UEs simultneously send dt to the BS, (iii) DL Pilot where the BS sends precoded pilots to ll UEs, (iv) DL Dt where the BS trnsmits dt to ll UEs nd (v) Switch Gurd, which idles the RF chins to llow switching from RX to X or vice vers. E. Hrdwre Requirements o illustrte the required HW cpbilities for the testbed, the vlues from ble II re used to estimte the Gops/s 3 nd the dt shuffling on per OFDM symbol bsis for the generl cse nd specific cse ssuming M = 00 nd K = 2. ) Processing pbilites: ble III summrizes the overll number of rel-vlued rithmetic opertions. For the processing estimtes, it is ssumed tht ech complex multipliction requires four rel multiplictions. lose to the ntenns, M fst-fourier trnsforms (FFs) or inverse FFs (IFFs) re needed equting to 26 Gops/s. Dt precoding nd detection s well s reciprocity compenstion require lrge mtrix nd vector multiplictions, for instnce, n M K mtrix with K vector leding to up to 80 Gops/s. Finlly, when using ZF, the pseudo-inverse mtrix hs to be clculted. Assuming Neumnn-Series pproximtion (K 3 complexity 3 Gops/s is used here, but these cn be seen s GMAs/s, i.e., the number of multiply-ccumulte opertions, s lmost ll opertions involve mtrixmtrix nd mtrix-vector clcultions. Purpose Generl Specific # # Links to cent. proc 2M 200 MB/s MB/s Antenn Rte w ntmf s w nt 3,072 Subcrrier Rte wmf sub w,680 Informtion rte K F sub 20.6 per itertion) [0] or QR decomposition for mtrix inversion nd requirement of finishing within two OFDM symbols pproximtely ops/s re necessry. 2) Dt Shuffling pbilities: ble IV summrizes required interconnect bndwidth nd number of links. ommuniction pths to ech ntenn trnsfer t the smpling rte of F s = MS/s which is decresed to the subcrrier rte F sub = 6.8 MB/s by performing OFDM processing (F s N used /(N FF + N cp )). onsidering M ntenns, the overll subcrrier dt rte is M w 6.8 MB/s, with w being the combined wordlength for the in-phse nd qudrture components in bytes. he informtion rte in n OFDM symbol crrying dt is K 6.8 MB/s ssuming 8 bit per smple, i.e., 256 QAM s highest modultion. Assuming seprte links between centrlized processing nd the ntenn units on UL nd DL, 2M peer-to-peer (P2P) links 4 re needed between the ntenns nd the centrlized MIMO processing. 3) Reconfigurbility: he testbed hs to be reconfigurble nd sclble, to support different system prmeters, different processing lgorithms nd dptive processing. It is lso crucil to hve the possibility to integrte developed intellectul property (IP) blocks for vlidtion nd comprison of hrdwre designs. Vrible center frequencies, run-time djustble RX nd X gins s well s configurble smpling rtes re highly desirble to be ble to dpt to other prmeters thn the ones presented in ble II. IV. GENERI HARDWARE AND PROESSING PARIIONING In this section generic HW nd processing prtitioning is presented to explore the prllelism in MMi, which needs considertion of processing together with dt trnsfer requirements (throughput, ltency, # of P2P links), nd t the sme time provides sclbility. 4 In this discussion, ech interconnection trnsferring dt between physiclly seprted devices is denoted peer-to-peer (P2P) link.

5 5 SDR o- Processor Switch Higher Lyer Processing Switch Switch Switch SDR SDR SDR SDR SDR Arry of Antenns o- Processor ontrol Processing entrlized Processing Distributed Processing 2 n sub RF Front RF Front RF Front Reci. om. Reci. om. Reci. om. OFDM X/RX OFDM X/RX OFDM X/RX 2 (nsub nnt) :(nco) Router B U S Fig. 3. Hierrchicl overview of MMi BS built from off-the-shelf HW. n sub RF Front Reci. om. OFDM X/RX (nco) :(nsub nnt) Router A. Hierrchicl Overview o be ble to build MMi testbed with off-the-shelf HW, hierrchicl distribution s shown in Fig. 3 is proposed. he min blocks re detiled s follows: ) SDR: Softwre-Defined Rdios (SDRs) provide the interfce between the digitl nd rdio-frequency (RF) domin s well s locl processing cpbilities. 2) Switches: Switches ggregte/disggregte dt between different prts of the system, e.g., between SDRs nd the coprocessors. 3) o-processing modules: o-processing modules provide centrlized node to perform MIMO processing. 4) Higher Lyer Processing: Higher lyer processing controls the system, configures the rdios, nd provides run-time sttus metrics of the system. B. Processing nd Dt Distribution For proper bse-bnd processing prtitioning, throughput constrints of HW components hve to be tken into ccount. Assuming ech SDR supports n nt ntenns, the required number becomes M/n nt SDRs, for n M-ntenn system. ) Subsystems: As shown in Fig. 4, RF-Front End, OFDM processing nd reciprocity compenstion re performed on per-ntenn bsis using, the SDRs. his distributes lrge frction of the overll processing nd reduces the dt rte before trnsferring the cquired smples over the bus. Still, the number of direct devices on bus is limited, nd thus, setting up 2M P2P links directly to the co-processors would most likely exceed the number of mximum P2P links for ny resonble number of MMi ntenns. o reduce this number, dt cn be ggregted using the concept of grouping. he different dt strems from severl SDRs re interleved on one common SDR nd then sent vi one P2P link. herefore, subsystems re defined, ech contining n sub SDRs. Dt from ll ntenns within subsystem is ggregted/disggregted on the outer two SDRs nd distributed to the n co co-processors using high-speed routers. At closer look, Fig. 4 revels tht the SDRs on the outer edges which relize the (n nt n sub ) to (n co ) nd (n co ) to (n nt 2 R SDRin/out (n nt ) (w F s ) MB/s = n nt n sub w F s MB/s Fig. 4. A subsystem consisting of n sub SDRs where the two outer SDRs implement n ntenn combiner / BW splitter nd n ntenn splitter / BW combiner, both implemented using high-speed FPGAs routers. Inter-SDR nd SDR to centrl processor connections utilize bus for trnsferring the smples. n sub ) router functionlities, require the highest number of P2P links, nd thus the highest throughput. Hence, the following inequlities hve to be fulfilled for the subsystems not to exceed the constrints for mximum number of P2P links (P2P SDR,mx ) nd mximum bidirectionl throughput (R SDRmx ): R SDRmx > R SDRout = R SDRin = n nt n sub w F sub (0) P2P SDR,mx > P2P SDR = n co + n sub () where it is ssumed tht if n SDR employs more thn one ntenn, the dt is serilized before it is sent to the router on the outer SDRs. 2) o-processors: As shown in Fig. 5, detection, precoding, SI cquisition, symbol mpping nd symbol dempping re integrted in the centrlly loclized co-processor modules which collect dt from ll SDRs. Using SI estimted from UL pilots, MIMO processing s discussed in Sec. II nd symbol mpping/de-mpping is performed. Bsed on the selected OFDM modultion scheme the subcrrier independence cn be exploited llowing ech of the n co co-processors to work on sub-bnd of the overll 20 MHz bndwidth. his efficiently circumvents issues with throughput nd ltency constrints in the MIMO signl processing chin. he co-processors ggregte/disggregte dt from ll the ntenns in the system using reconfigurble high-speed routers, s shown in Fig. 5 for system hving M/(n sub n nt ) subsystems nd n co co-processors. Similrly to the SDRs, the two min constrints for the coprocessors re the mximum number of P2P links denoted P2P O,mx nd the mximum throughput denoted R Omx. he following inequlities hve to hold for the co-processor

6 6 Subsystem Subsystem dm/(n subn nt)e subbnd n co subbnd n co B U S M : () Rn sub n nt R Rnsub nnt R Router M () : nsub nnt Router Router hnnel Estimtion + MIMO Detection MIMO Precoding hnnel Estimtion + MIMO Detection MIMO Precoding Number of ntenn strems = n sub n nt R Oin/out = M/n co w F s MB/s 2 K 6.8MB/s M Router : () M () : nsub nnt Symbol Demp Symbol Mpping n co Symbol Demp Symbol Mpping Fig. 5. Shuffling dt from the M/(n sub n nt) subsystems to the n co coprocessors. he routers use simple round robin scheme to combine/distribute the dt from/to corresponding subsystems. not to exceed these constrints: R Omx > R Oout = R Oin = ( ) M w + K = F sub (2) n co P2P O,mx > P2P O = 2 M/n sub + 2. (3) Using this modulr nd generic system prtitioning, off-theshelf HW pltforms cn be evluted. Note, tht expressions (0) - (3) my lso be used with other system prmeters, e.g., by redefining F s nd F sub. V. LUMAMI ESBED IMPLEMENAION In this section the LuMMi specific implementtion detils re discussed bsed on the forementioned generl rchitecture. Strting with the number of BS ntenns nd UEs, the selected off-the-shelf HW pltform nd evlution of the constrints discussed in previous section re presented. onsequently, the specific frme structure nd other fetures of the system including bse-bnd processing, ntenn rry, mechnicl structure nd synchroniztion re briefly described. A. Number of BS Antenns nd UEs According to theoreticl results, ssuming iid Ryleigh chnnels nd sptil MF, the sum rte optimum is chieved when hving BS ntenns to UE rtio of round two []. However, in rel system there re typiclly non-iid chnnels with power imblnces nd BS ntenn correltion [2]. Hence, the idel rtio is scenrio dependent nd needs further explortion. he LuMMi uses 00 BS ntenns nd cn support 2 UEs, resulting in rtio of 8 between BS ntenns nd UEs, number tht hs shown to work well in rel propgtion environments [3]. 2 2 B. Selected Hrdwre Pltform he hrdwre pltform ws selected bsed on requirements discussed in Sec. III. ble V shows the selected off-theshelf hrdwre from Ntionl Instruments used to implement the LuMMi testbed. he SDRs [4] llow up to 5 P2P links (P2P SDR,mx = 5) with bidirectionl throughput of R SDRmx = 830 MB/s, support vrible center frequency from.2 GHz to 6 GHz nd hve X power of 5 dbm. Ech SDR contins two RF chins, i.e., n nt = 2, nd Kintex-7 FPGA. Selected co-processors [5] llow bidirectionl P2P rte of R Omx = 2.4 GB/s with up to P2P O,mx = 32 P2P links nd employ powerful Kintex-7 FPGA with reported performnce of up to GMAs/s [6]. his is sufficient for 00 BS ntenn MMi testbed due to the fct tht n co coprocessors cn be utilized in prllel. Interconnection mong devices is chieved using 8-slot chssis [7] combined with per-slot expnsion modules [8]. Ech chssis integrtes two switches bsed on Peripherl omponent Interconnect Express (PIe) using direct memory ccess (DMA) chnnels which llow inter-chssis trffic up to 7 GB/s nd intr-chssis trffic up to 3.2 GB/s. he host [9] is n integrted controller, running LbVIEW on stndrd Windows operting system nd is used to configure nd control the system. he integrted hrdwre/softwre stck provided by LbVIEW provides the needed reconfigurbility s it bstrcts the P2P link setup, communiction mong ll devices nd llows FPGA progrmming s well s host processing using single progrmming lnguge. An dditionl feture of LbVIEW is the possibility to semlessly integrte IP blocks generted vi Xilinx Vivdo pltform pving wy to test in-house developed IP. o be ble to synchronize the full BS, Reference lock Source [20] nd Reference clock distribution network [2] re required. heir functionlities will be lter discussed when presenting the overll synchroniztion method.. Subsystems nd Number of o-processors o build the LuMMi testbed with M = 00 ntenns, 50 SDRs re necessry. he mximum possible subsystem size is chosen to minimize the utiliztion of vilble P2P links t the co-processors. By using (0) nd n internl fixed-point wordlength of w = 3 corresponding to 2-bit resolution on the I- nd Q-components, n sub is found to be 8. As this is not n integer divider of 50, the lst subsystem only contins two SDRs. Bsed on ble IV, the combined subcrrier rte for ll ntenns is wmf sub = 5 GB/s nd nother K F sub = 200 MB/s re needed for informtion symbols. o not exceed R Omx t lest three co-processors must be utilized. o further lower the burden on the design of the low-ltency MIMO signl processing chin, n co = 4 is chosen such tht ech coprocessor processes 300 of the overll 200 subcrriers. ble VI summrizes the LuMMi testbed prmeters nd shows tht constrints re met ccording to (0)-(3). It cn lso be seen tht the design is still within the constrints if scling up the number of BS ntenns to M = 28, which hs

7 7 ABLE V SELEED HARDWARE FROM NAIONAL INSRUMENS ype Model Fetures Host SDR o-processor Switch Expnsion Module Reference lock Source PXIe-835 USRP RIO 294XR / 295XR FlexRIO 7976R PXIe-085 PXIe-8374 PXIe GHz Qud-ore PXI Express ontroller Up to 8 GB/s system nd 4 GB/s slot bndwidth 2 RF Front Ends nd Xilinx Kintex-7 FPGA enter frequency vrible from.2 GHz to 6 GHz 830 MB/s bidirectionl throughput on up to 5 DMA chnnels Xilinx Kintex-7 40 FPGA 2.4 GB/s bidirectionl throughput on up to 32 DMA chnnels Industril form fctor 8-slot chssis 7 GB/s bidirectionl throughput per slot 2 switches per chssis with inter-switch trffic up to 3.2 GB/s Links between chssis bound to 7 GB/s bidirectionl PXI Express (x4) hssis Expnsion Module Softwre-trnsprent link without progrmming Str, tree, or disy-chin configurtions 0 MHz reference clock source with < 5 ppb clock ccurcy 6 configurble I/O connections Ref. lock Distribution Octolock 0 MHz 8-chnnel clock nd timing distribution network ABLE VI SYSEM PARAMEERS AND VALIDAION OF ONSRAINS IN HE LUMAMI ESBED. Prmeters Rtes MB/s M 00 R SDRmx = 830 > R SDRout = R SDRin = K 2 R Omx = 2, 400 > R Oout = R Oin =, 460 n nt 2 P2P Links n sub 8 P2P SDR,mx = 5 > P2P SDR = 2 n co 4 P2P O,mx = 32 > P2P O = 8 Note, tht the lst subsystem only consists of two SDRs. subcrrier Fig N used sf ( ms) f (0 ms) DAA slot (0.5 ms) UE0 UE. UEK- UE0 UE. UEK- 7 OFDM symbols UE0 UE. UEK- UE0 UE. UEK- DAA slot (0.5 ms)... legend UE x UL pilot uplink downlink sync precoder computtion time he defult frme structure used in the LuMMi testbed. DL pilots gurd been done in subsequent designs bsed on the sme hrdwre, e.g., [5]. D. Frme Structure he defult frme structure for the LuMMi testbed is shown in Fig. 6. One frme is f = 0 ms nd is divided in ten subfrmes of length sf = ms. Ech subfrme consists of two slots hving length slot = 0.5 ms, where the first subfrme is used for control signls, e.g., to implement overthe-ir synchroniztion, UL power control nd other control signling. he 8 slots in the other nine subfrmes encpsulte seven OFDM symbols ech. ompring to Fig. 2, reciprocity clibrtion cycle is defined over the whole run-time of the BS for simplicity nd due to the fct tht there is no lrge drift fter wrming up the system in controlled environment [4]. he DL pilot cycles nd control cycles re both set to be the length of one frme. Ech frme strts with one control subfrme followed by one subfrme with one DL pilot nd one DL dt symbol wheres ll others use two DL dt symbols. E. Mobility he pilot distnce in time in the defult frme structure given in Fig. 2 is p 430 µs or six OFDM symbols. hus, ν mx 240 Hz for correltion of 0.9. Due to vilbility from network opertor, crrier frequency of f c = 3.7 GHz is selected. Using (8), v mx = 70 km/h is found s mximum supported speed. F. DD urnround ime he pre-coding turnround time requirement for the implementtion cn be nlyzed bsed on (9). he nlog frontend dely of the SDRs ws mesured to be bout 2.25 µs. king the frme structure in Fig. 6 (ssuming rf,x = rf,rx which is not necessrily true), the ltency budget for bsebnd processing is s follows: Overll time for pre-coding fter receiving the UL pilots is 24 µs (3 OFDM symbols). he 2k FF/IFF (ssuming clock frequency of 200 MHz) requires round 35 µs 2 = 70 µs in totl for X nd RX (including smple reordering). As result, the remining time for chnnel estimtion, MIMO processing, nd dt routing is round 40 µs, which is the design constrint for this specific frme structure. An nlysis of the implemented design showed tht the ltency is fr below the requirement for the defult frme

8 8 Fig. 7. Left: Side view of the mechnicl ssembly of the BS. he two rcks sit side by side (not s shown) with the SDRs fcing the sme direction (towrds the ntenn rry). wo columns of USRP SDRs re mounted in ech rck, totling 50 of them. Right: he ssembled LuMMi testbed t Lund University, Sweden. ABLE VII FPGA UILIZAION FOR WO DIFFEREN MIMO PROESSING IMPLEMENAIONS Implementtion Registers LU RAMs DSP48 QRD Neumnn-Series (9.%) (20.3%) (2.5%) (38.7%) (3.%) (.8%) (0.75%) (.4%) structure which mkes it possible to use the testbed for higher mobility scenrios [22], from this point of view. G. Implementtion Fetures ) bse-bnd Processing: A lest-squre SI estimtion lgorithm with zeroth-order hold over K = 2 subcrriers ws implemented. o chieve the required detection mtrix throughput of one mtrix every 2 subcrriers, subcrriers/s/2 = Detection Mtrices/s hve to be clculted. wo versions for detection were implemented. he first one bsed on QR decomposition of the extended chnnel mtrix, formulted into prtil prllel implementtion employing systolic rry, the ltter one bsed on Neumnn-series. In the QR decomposition, ech column is processed using the discrete steps of the modified Grm-Schmidt lgorithm. he logic on the co-processors cn be reconfigured so tht the sme hrdwre resources tht provide the RZF decoder cn lso provide the ZF nd MR decoders, i.e., the detection / precoding schemes discussed in Sec. II re supported with runtime switching. he Neumnn-series bsed ZF detector utilizes the unique property tht in MMi, the Grmin mtrix shows dominnt digonl elements llowing the mtrix inversion to be pproximted with low overll error [0]. he utiliztions for the two FPGA designs re shown in ble VII. lerly, overll processing complexity nd resource utiliztion cn be significntly reduced by exploiting the specil properties of MMi. 2) Host-bsed visuliztion nd dt cpturing: he vilble mrgin of GB/s nd 4 P2P links to the corresponding mximum vlues on the co-processors re used for visuliztion nd system performnce metrics. he host receives decimted equlized constelltions nd rw subcrriers for one UL pilot nd one UL dt symbol per frme. hese fetures dd nother 300 2bytes bytes = 300 MB/s 0ms of dt flowing in nd out of the co-processor. he rw subcrriers re used to perform chnnel estimtion nd UL dt detection on the host computer with floting point precision nd llow fst implementtion of different metrics, like constelltion, chnnel impulse response, power level per ntenn nd user. Another 2 P2P links vilble re utilized to trnsmit nd store rel-time BERs for ll 2 UEs. Moreover, to be ble to cpture dynmics in the chnnel for mobile UEs, SI cn be stored on ms bsis. An integrted 2 GB Dynmic Rndom Access Memory (RAM) (DRAM) buffer on ech of the co-processors ws utilized for this since direct streming to disk would exceed the P2P bndwidth limits. Snpshots cn either be tken for 60 s in 5 ms intervl or over 2 s in ms intervl, both corresponding to 2 GB of dt for 300 subcrriers. 3) Sclbility/Reconfigurbility: Before strtup, the number of deployed BS ntenns cn be rbitrrily set between 4 nd 00. his is chieved by introducing zeros for non-existing ntenns within the lookup-tble (LU)-bsed reconfigurble high-speed routers on the co-processors, thereby llowing to evlute effects of scling the BS ntenns in rel environments [22]. Additionlly, ll 40 OFDM symbols in frme re rbitrrily configurble before strt-up while ech frme lwys repets itself. For instnce, we cn choose to set the first symbol s UL pilots nd ll others s UL dt in sttic UL only scenrio. 4) Reciprocity librtion: Estimtion of the reciprocity clibrtion coefficients ws implemented on the host, minly for two resons: (i) the host cn perform ll opertions in

9 9 floting-point which increses precision nd (ii) the drift of the hrdwre is not significnt once the system reched operting temperture [4]. Estimted reciprocity coefficients re pplied in distributed mnner on the SDRs [22]. H. Mechnicl structure nd electricl chrcteristics wo computer rcks contining ll components mesuring m were used, s shown Fig. 7. An essentil requirement for the LuMMi testbed is to llow tests in different scenrios, e.g., indoor nd outdoor. herefore, the rck mount is ttched on top of 4-wheel trolley m m 2.5 m BS I. Antenn Arry he plnr -shped ntenn rry with 60 dul polrized λ/2 ptch elements ws developed in-house. A 3.2 mm Dicld 880 ws chosen for the printed circuit bord substrte. he upper horizontl rectngle hs 4 25 elements nd the centrl squre hs 0 0 elements (see Fig. 7 right). his yields 320 possible ntenn ports tht cn be used to explore different ntenn rry rrngements, for exmple 0 0 or 4 25 with the ltter one being the defult configurtion. All ntenn elements re center shorted, which improves isoltion nd bndwidth. he mnufctured rry yielded n verge 0 db-bndwidth of 83 MHz centered t 3.7 GHz with isoltion between ntenn ports vrying between 8 db nd 28 db depending on loction in the rry. J. User Equipment Ech UE represents phone or other wireless device with single ntenn cpbilities. One SDR serves s two independent UEs such tht overll six SDRs re required for the 2 UEs. he bse-bnd processing, i.e., OFDM modultion/demodultion nd symbol mpping/dempping re essentilly identicl to the BS implementtion. A lest-squre SI cquisition is performed on precoded DL pilot followed by ZF-equlizer. he DL pilots occupy full OFDM symbol. K. Synchroniztion A MMi BS requires time synchroniztion nd phse coherence between ech RF chin. his is chieved using the 0 MHz reference clock source nd the reference clock nd trigger distribution network (see ble V). he reference clock is used s the source of ech rdio locl oscilltor, providing phse coherence mong devices. he trigger signl is used to provide time reference to ll the rdios in the system. A mster provides n output digitl trigger tht is mplified nd divided mong ll the rdios. Upon receipt of the rising edge of the event trigger, ll SDRs re strted. he bsic structure cn be identified in Fig. 7 on the left. o synchronize the UEs with the BS over-the-ir (OA), the LE Zdoff-hu Primry Synchronistion Signl (PSS) is used, which occupies the center.2 MHz of the overll bndwidth. OA synchroniztion nd frequency offset compenstion re chieved by employing frequency-shifted bnk of replic filters. he process follows two step procedure: finding corse cndidte position by scnning over the whole Fig. 8. he indoor mesurement setup in lecture room including the positions of the 2 UEs. he BS is shown t the right-hnd side nd is situted t the front of the lecture hll. he terminls re plced in groups of four on three different tbles nd distnces to the BS. Fig. 9. One group of four UEs with high user density per unit re to vlidte the sptil multiplexing cpbilities of MMi. rdio frme followed by trcking the PSS in nrrowed window locted round the corse cndidte position. Additionlly, by disciplining the UE SDRs with Globl Positioning System (GPS), frequency offset compenstion my be voided by lowering the frequency offset to < 300 Hz. VI. FIELD RIALS AND EXPERIMENAL RESULS his section describes two experiments performed to vlidte our testbed design, the MMi concept nd its performnce. he first test is performed indoors with high density of users per re unit to stress the sptil multiplexing cpbilities of the system. he second test is conducted outdoors with less dense deployment of UEs nd is primrily designed to test the rnge, sptil multiplexing cpbilities outdoors nd the effect of moving UEs. It hs to be noted tht ll results shown in this section re obtined from rel-time opertion without UL power control. A. Indoor est In this test rel-time uncoded BER curves re mesured, employing MR/MR nd ZF s decoders/precoders. he UL BER curves re obtined by sweeping ll UE X power mplifier (PA) gins synchronously, nd for the DL BER curves

10 0 UE0 UE UE2 UE3 UE4 UE5 UE6 UE7 UE8 UE9 UE0 UE 0 0 Uncoded BER () UL with QPSK (b) UL with 64-QAM Uncoded BER Amplifier Gin in db (c) DL with QPSK Amplifier Gin in db (d) DL with 64-QAM Fig. 0. UL nd DL BERs for 2 UEs with ZF decoder/precoder. the PA gins of the BS X chins while keeping other system prmeters constnt. Note tht the initil prmeteriztion of the system is chosen empiriclly, so it llows smooth BER curves strting t bout 0.5. Ech gin step is held constnt for bout 4 s corresponding to bout nd trnsmitted bits per step for QPSK nd 64-QAM modultion, respectively. ) Scenrio: welve UEs re set up in lecture hll t Lund University with the BS t the front s shown in Fig. 8 including the respective UE plcements. All UEs re pcked in groups of four resulting in high density of UEs per re unit. One of these groups cn be seen in Fig. 9. 2) UL BERs: Fig. 0, () nd (b), show the BERs for ll 2 UEs using ZF detector for QPSK nd 64-QAM modultion, respectively. For both constelltion sizes, the UEs furthest wy, UE0 to UE3 show highest BER. UE0 nd UE even show sudden increse for the BER to 0.5 which ws dignosed to be due to sturtion of their respective PAs. Moreover, their performnce shows severe limittion compred to the other UE, giving cler indiction tht their performnce is interference rther thn power limited. he group closest to the BS, UE9-UE2, shows best performnce lthough the vrition within the group is still quite significnt. Overll, it cn be seen tht the MMi concept works nd tht it is cpble of seprting 2 UEs on the sme time/frequency resource even with high UE density. hus, the spectrl efficiency my be improved by t lest n order of mgnitude compred to current cellulr stndrds. 3) DL BERs: Fig. 0, (c) nd (d), show the DL BERs using ZF precoder for QPSK- nd 64-QAM modultion, respectively. Using QPSK modultion, the group closest to the BS, UE9-UE2, chieves considerbly better performnce thn the other two groups. Using 64-QAM, ll UEs show n error-floor towrds higher X gin vlues which is likely result of imperfect reciprocity clibrtion resulting in interference mong UEs. he tests performed were minly to prove functionlity, nd thus, no specil cre ws tken to chieve best possible ccurcy for the reciprocity clibrtion. However, individul prts re continuously tested to be improved. 4) MR/MR versus ZF: o compre the performnce of MR/MR nd ZF it is beneficil to isolte the nlysis to one UE. Fig. nd Fig. b show the BER for UE7 for QPSK, 6-QAM nd 64-QAM modultions while the BS employs either MR/MR or ZF on the UL nd DL, respectively. Overll, ZF shows n superior performnce trend with incresing PA gins, while the performnce of MR ppers to level off 5. Looking in more detil, ZF is cpble of chieving more thn n order of mgnitude lower BERs, compred to MR. Using higher constelltion sizes, 6-QAM or 64-5 his is expected from theory, s inter-user interference is the min source of error during dt detection. he high density users setup dopted in this experiment highly contributes to this phenomen.

11 0 0 Uncoded BER Amplifier Gin (in db) 0 0 () MR QPSK MR 6-QAM MR 64-QAM ZF QPSK ZF 6-QAM ZF 64-QAM 2nd floor 2nd floor st floor 2nd floor m BS 3rd floor Uncoded BER Amplifier Gin (in db) (b) MR QPSK MR 6-QAM MR 64-QAM ZF QPSK ZF 6-QAM ZF 64-QAM Fig.. BERs for 2 UEs using QPSK, 6-QAM nd 64-QAM modultion. () on the UL for ZF nd MR detector nd (b) on the DL for ZF nd MR precoder. Fig. 2. Scenrio for the outdoor tests. BS plced on the rooftop of the building (third floor) serving eight UEs on the opposite wing, with six UEs on second floor nd two UEs on first floor. QAM, the results for MR show n even more significnt deteriortion. On the DL, ZF lso outperforms MR by fr, the ltter shows significnt error floor towrds higher gins s in the UL cse. Unfortuntely, direct comprison between UL nd DL results shown here is not esy to perform. his is due to the fct tht on the UL, the performnce is isolted to the UL trnsmit power only wheres on the DL combintion of UL chnnel estimte qulity, DL trnsmit power nd reciprocity ccurcy determines overll performnce. B. Outdoor est For the outdoor test, the testbed ws plced on the rooftop of one of the wings of the deprtment building while the UEs where plced on the opposite wing utilizing scffolding mounted to the building. Up to eight UEs were served simultneously in distnce of bout 8 to 22 meters, six on the second floor nd two on the first floor while the testbed ws situted on the third floor (rooftop). he scenrio is shown in Fig. 2. Fig. 3 shows the BS plced on the rooftop of the deprtment building fcing towrds the opposite wing. he plcement for UEs 0 nd is lso mrked. Fig. 4 shows screenshot of the received UL QPSK constelltions for this test setup when using MR nd ZF, respectively. Using MR without error-correcting code (E) for this test, the six UEs show significnt interference. herefore, focus is put on the results obtined with ZF which is Fig. 3. he outdoor test scenrio setup with the BS deployed on the rooftop of the deprtment building mrked with two UEs on the opposite building wing. cpble of seprting up to eight UEs nd shows very cler constelltions, due to the interference suppression, s shown in Fig. 4b. onsidering ZF on the DL, the constelltions for ll 8 UEs cn be seen in Fig. 5. As visible, qulity of the DL constelltions is deteriorted compred to the UL constelltions where some UEs show significnt interference. he results observed in this experiment re representtive for most tests performed so fr, i.e., DL lwys showed to be the more chllenging duplex cse.. Initil Mobility Mesurements In preprtion for mobility mesurements cmpign, n mobile UE ws dded to the outdoor mesurements discussed previously. his ws done to see whether there re ny problems when communicting with moving UE nd to verify tht OA synchroniztion works properly in dynmic environment. Fig. 6 shows picture of the received DL constelltion inside cr while driving t speed of 50 km/h. A cler QPSK constelltion nd flt DL spectrum over the 20 MHz is visible which verifies proper functionlity even for mobile UEs. herefore, first MMi mobility mesurements with the BS plced on the rooftop of building nd up to

12 he implementtion supports rel-time DD trnsmission in 00 the criticl pth is less thn 0.5 ms, enbling reciprocity-bsed this implementtion hve lso been demonstrted. 2 s eo ly nel ztion eo ming N RESUL udget 285 µs he system is build of 64 USRP-2943R, 4 FlexRIO PXIe-7976R nd PXIe-835 host Not used computer which re ll interconnected through PI Express network to llow inter-fpga s well s FPGA-host connections. he system is portioned nd grouped for esy dt routing Not used nd prllel processing : 8 USRPs form sub-group 2 USRPs in ech sub-group re used for () dt ggregtion nd disggregtion Prllel MIMO detection nd precoding is performed over 4 FlexRIOs, ech work on ¼ of the overll bndwidth. Distributed processing to enble low dt shuffling trffic (b) OFDM Demodultion (25%) ontrol nd Dt Reordering (3.6%) OFDM Modultion (9.2%) Front-End Dely (%) Dt rnsfer (7%) Mrgin (54.2%) Fig. 4. UL constelltions for the outdoor experiment: () when using MR with 6 UEs nd (b) when using ZF to serve 8 UEs. () nd the vilble budget is 285 μs in our system. Front-end e lowest contribution. OFDM demodultion occupies whole essing speed is limited by the smpling rte while OFDM t higher clock rte. Dt trnsfer dely ws nlyzed r of 4 hops over the bus. (c) Fig. 5. Received DL constelltions using ZF: () UE0 & UE (b) UE2 & UE3 (c) UE5 & UE8 nd (d) UE9 & UE0. (b) (d) First Mobility est Up to 50km/h nd higher speed test plnned for Fll moving UEs were performed. he video in [23] shows the setup nd rel-time testing results for one scenrio cquired during these mesurements. Detiled nlysis of these mesurements will be published elsewhere. VII. ONLUSION his pper presented the LuMMi (Lund University Mssive MIMO) testbed, which is the first fully opertionl reltime testbed for prototyping mssive MIMO. Bsed on mssive MIMO system requirements, system prmeters were discussed nd defined. Further, detiled generic hrdwre prtitioning to overcome chllenges for dt shuffling nd peer-to-peer link limittions while still llowing sclbility, ws proposed. By grouping Softwre-Defined Rdios nd splitting overll bndwidth, implementtion of mssive MIMO signl processing ws simplified to cope with chllenges like time-division duplex precoding turnround time nd limited peer-to-peer bndwidth enforcing strict design requirements when scling the number of bse sttion ntenns up to 00 or higher. Bsed on the generic system prtitioning nd system requirements, hrdwre pltform ws selected nd evluted. It ws shown tht internl system configurtion is within throughput nd processing cpbilities before the complete LuMMi testbed prmeters were described. Finlly, field tril results including Bit Error Rte performnce mesurements nd constelltions were presented from both indoor nd outdoor mesurement cmpigns. he results showed tht it is possible to seprte up to 2 user equipments on the sme time/frequency resource when using mssive MIMO. Hving estblished flexible pltform for testing new lgorithms nd digitl bse-bnd solutions we re ble to tke mssive MIMO from theory to rel-world tests nd stndrdiztion for next genertion wireless systems. AKNOWLEDGMEN his work ws funded by the Swedish foundtion for strtegic reserch SSF, VR, the strtegic reserch re ELLII, nd the EU Seventh Frmework Progrmme (FP7/ ) under grnt greement n (MAMMOE). Fig km/h. Front pnel picture of n UE operting in cr t speed of up to

13 3 REFERENES []. Mrzett, Noncoopertive ellulr Wireless with Unlimited Numbers of Bse Sttion Antenns, IEEE rnsctions on Wireless ommunictions, vol. 9, no., pp , November 200. [2] F. Rusek, D. Persson, B. K. Lu et l., Scling Up MIMO: Opportunities nd hllenges with Very Lrge Arrys, IEEE Signl Process. Mg., vol. 30, no., pp , Jn [3]. Sheprd, H. Yu, N. Annd et l., Argos: Prcticl Mny-ntenn Bse Sttions, in Proceedings of the 8th Annul Interntionl onference on Mobile omputing nd Networking, ser. Mobicom 2. New York, NY, USA: AM, 202, pp [Online]. Avilble: [4] J. Vieir, S. Mlkowsky, K. Niemn et l., A flexible 00-ntenn testbed for Mssive MIMO, in Globecom Workshops (G Wkshps), 204, pp [5] P. Hrris, S. Zng, A. Nix et l., A Distributed Mssive MIMO estbed to Assess Rel-World Performnce nd Fesibility, in 205 IEEE 8st Vehiculr echnology onference (V Spring), My 205, pp. 2. [6] J. Vieir, F. Rusek, O. Edfors et l., Reciprocity librtion for Mssive MIMO: Proposl, Modeling nd Vlidtion, orr, vol. bs/ , 206. [Online]. Avilble: [7] R. Roglin, O. Y. Burslioglu, H. Ppdopoulos et l., Sclble Synchroniztion nd Reciprocity librtion for Distributed Multiuser MIMO, IEEE rnsctions on Wireless ommunictions, vol. 3, no. 4, pp , April 204. [8] E. Björnson, M. Bengtsson, nd B. Ottersten, Optiml Multiuser rnsmit Bemforming: A Difficult Problem with Simple Solution Structure, IEEE Signl Processing Mgzine, vol. 3, no. 4, pp , 204. [9] E. Björnson, E. G. Lrsson, nd. L. Mrzett, Mssive MIMO: ten myths nd one criticl question, IEEE ommunictions Mgzine, vol. 54, no. 2, pp. 4 23, Februry 206. [0] H. Prbhu, J. Rodrigues, O. Edfors et l., Approximtive mtrix inverse computtions for very-lrge MIMO nd pplictions to liner precoding systems, in IEEE Wireless ommunictions nd Networking onference (WN), 203, pp [] H. Q. Ngo, E. G. Lrsson, nd. L. Mrzett, Energy nd Spectrl Efficiency of Very Lrge Multiuser MIMO Systems, IEEE rnsctions on ommunictions, vol. 6, no. 4, pp , Apr 203. [2] X. Go, M. Zhu, F. Rusek et l., Lrge ntenn rry nd propgtion environment interction, in th Asilomr onference on Signls, Systems nd omputers, Nov 204, pp [3] X. Go, O. Edfors, F. Rusek et l., Mssive MIMO Performnce Evlution Bsed on Mesured Propgtion Dt, IEEE rnsctions on Wireless ommunictions, vol. 4, no. 7, pp , July 205. [4] Ntionl Instruments. (204) USRP-2943R Dt Sheet. com/dtsheet/pdf/en/ds-538 (visited on 4 Oct. 206). [5] Ntionl Instruments. (204, Jul.) FlexRIO 7976R Dt Sheet. (visited on 4 Oct. 206). [6] Xilinx. (206) 7 Series FPGAs Overview: DS80 (v2.0) Product Specifiction. sheets/ds80 7Series Overview.pdf (visited on 4 Oct. 206). [7] Ntionl Instruments. (205) PXIe 085 Mnul. mnuls/37372f.pdf (visited on 4 Oct. 206). [8] Ntionl Instruments. (20) MXI-Express x4 Series User Mnul. http: /0/ (visited on 4 Oct. 206). [9] Ntionl Instruments. (203) PXIe 835 Mnul. mnuls/37376b.pdf (visited on 4 Oct. 206). [20] Ntionl Instruments. (205) PXIe-6674 User Mnul: iming nd Synchroniztion Module for PXI Express. [2] Ettus Reserch. USRP Hrdwre Driver nd USRP Mnul: Octo- lock. octoclock.html (visited on 4 Oct. 206). [22] S. Mlkowsky, J. Vieir, K. Niemn et l., Implementtion of Low- Ltency Signl Processing nd Dt Shuffling for DD Mssive MIMO Systems, in 206 IEEE Interntionl Workshop on Signl Processing Systems (SiPS), Oct 206, pp [23] (206) Mssive MIMO Mobility ests. v=wppmrr4rhmo (visited on 4 Oct. 206). Steffen Mlkowsky received the B.Eng. degree in Electricl Engineering nd Informtion echnology from Pforzheim University, Germny in 20 nd the M.Sc. degree in Electronic Design from Lund University in 203. He is currently PhD student in the Digitl ASI group t the deprtment of Electricl nd Informtion echnology, Lund University. His reserch interests include development of reconfigurble hrdwre nd implementtion of lgorithms for wireless communiction with emphsis on mssive MIMO. For the development of mssive MIMO testbed in collbortion with University of Bristol nd Ntionl Instruments nd set spectrl efficiency world record, he received 5 interntionl wrds from Ntionl Instruments, Xilinx nd Hewlett Pckrd Enterprise. João Vieir received the B.Sc. degree in Electronics nd elecommunictions Engineering from University of Mdeir in 20, nd the M.Sc. degree in Wireless ommunictions from Lund University, Sweden in 203. He is currently working towrds Ph.D. degree t the deprtment of Electricl nd Informtion echnology in Lund University. His min reserch interests regrd prmeter estimtion nd implementtion issues in mssive MIMO systems. Ling Liu received his B.S. nd Ph.D. degree in the Deprtment of Electronics Engineering (2005) nd Micro-electronics (200) from Fudn University, hin. In 200, he ws with Rensseler Polytechnic Institute (RPI), USA s visiting resercher. He joined Lund University s Post-doc in 200. Since 206, he is Associte Professor t Lund University. His reserch interest includes wireless communiction system nd digitl integrted circuits design. He is bord member of the IEEE Swedish SS/AS chpter. He is lso member of the technicl committees of VLSI systems nd pplictions nd AS for communictions of the IEEE circuit nd systems society. Pul Hrris grduted from the University of Portsmouth with st lss Honours degree in Electronic Engineering in 203 nd joined the ommuniction Systems & Networks Group t the University of Bristol in the sme yer to commence PhD. His reserch interests include mssive MIMO system design, performnce evlution in rel-world scenrios, nd the optimistion of techniques such s user grouping or power control using empiricl dt. Working in collbortion with Lund University nd Ntionl Instruments, he implemented 28- ntenn mssive MIMO test system nd led two reserch tems to set spectrl efficiency world records in 206. For this chievement, he received 5 interntionl wrds from Ntionl Instruments, Xilinx nd Hewlett Pckrd Enterprise, nd n honorry mention in the 206 IEEE omsoc Student ompetition for ommunictions echnology hnging the World.

14 4 Dr. Krl Niemn is Senior Wireless Pltform Architect in the Advnced Wireless Reserch tem t Ntionl Instruments. His interests re with reserch nd stndrdiztion of 5G technology, prticulrly Mssive MIMO rchitectures nd signl processing. He hs designed nd implemented severl FPGAbsed rel-time wireless communiction systems, hs mde multiple contributions to 3GPP RAN, nd holds multiple issued nd pending ptents on 5G technologies. He erned his Ph.D. nd M.S. in Electricl Engineering from University of exs t Austin in 204 nd 20, respectively, nd his B.S. in Electricl Engineering from New Mexico Institute of Mining nd echnology in Nikhil Kundrgi is Senior Wireless Pltform Architect in the Advnced Wireless Reserch em t Ntionl Instruments since 202. He leds the Mssive MIMO reserch inititive t NI. He is the 3GPP RAN Delegte for NI nd prticiptes in LE-Advnced nd 5G cellulr stndrdiztion. His reserch interests include Mssive MIMO, Full Dimension MIMO, 5G New Rdio, mmwve, PHY/MA lyer design nd prototyping, Dense LE networks, Rel-time DSP, Softwre Defined Rdio Architectures, ognitive Rdio Networks, Spectrum Sensing, Dynmic Spectrum Access, Anomly Detection nd Sttisticl Signl Processing. He received his Ph.D in Electricl Engineering from the University of exs t Austin in 202. He ws member of WNG (Wireless Networking nd ommunictions Group t U Austin nd formerly Grdute School Fellow t the University of Minnesot from He is lso n ctive member of IEEE Austin omsoc hpter nd served s the Vice-hir for the Industry Forum t IEEE Globecom 205. Dr. In. Wong is Senior Mnger of the Advnced Wireless Reserch group t Ntionl Instruments where he leds the compny s 3GPP nd 802. wireless stndrds strtegy nd pltforms for wireless system design, simultion, prototyping, nd implementtion. From , he ws systems reserch nd stndrds engineer with Freescle Semiconductor where he represented Freescle in the 3GPP LE stndrdiztion efforts. He ws the Industry Progrm hir for IEEE Globecom 205 in Austin, the Director for Industry ommunities for IEEE ommunictions Society , nd senior member of the IEEE. His current reserch interests include 5G wireless systems design nd prototyping, nd design utomtion tools for rpid lgorithm development. Dr. Wong is the co-uthor of the Springer book Resource Alloction for Multiuser Multicrrier Wireless Systems, hs over 0 ptents, over 25 peerreviewed journl nd conference ppers, nd over 40 stndrds contributions. He ws wrded the exs elecommunictions Engineering onsortium Fellowship in , nd the Wireless Networking nd ommunictions Group student ledership wrd in He received the MS nd PhD degrees in electricl engineering from the University of exs t Austin in 2004 nd 2007, respectively, nd BS degree in electronics nd communictions engineering (mgn cum lude) from the University of the Philippines in Fredrik ufvesson received his Ph.D. in 2000 from Lund University in Sweden. After two yers t strtup compny, he joined the deprtment of Electricl nd Informtion echnology t Lund University, where he is now professor of rdio systems. His min reserch interests re chnnel modelling, mesurements nd chrcteriztion for wireless communiction, with pplictions in vrious res such s mssive MIMO, UWB, mm wve communiction, distributed ntenn systems, rdio bsed positioning nd vehiculr communiction. Fredrik hs uthored round 60 journl ppers nd 20 conference ppers, recently he got the Nel Shepherd Memoril Awrd for the best propgtion pper in IEEE rnsctions on Vehiculr echnology. Viktor O wll received the M.Sc. nd Ph.D. degrees in electricl engineering from Lund University, Lund, Sweden, in 988 nd 994, respectively. During 995 to 996, he joined the Electricl Engineering Deprtment, the University of liforni t Los Angeles s Postdoc where he minly worked in the field of multimedi simultions. Since 996, he hs been with the Deprtment of Electricl nd Informtion echnology, Lund University, Lund, Sweden. He is currently full Professor nd since 205 the Den of the Fculty of Engineering. He ws the founder nd the Director of the VINNOVA Industril Excellence enter in System Design on Silicon (SoS) which he heded until 204. His min reserch interest is in the field of digitl hrdwre implementtion, especilly lgorithms nd rchitectures for wireless communiction nd biomedicl pplictions. He ws co-founder of the strt-up compny Phse Hologrphic Imging who develops microscopes utilizing digitl hologrphy. Ove Edfors is Professor of Rdio Systems t the Deprtment of Electricl nd Informtion echnology, Lund University, Sweden. His reserch interests include sttisticl signl processing nd lowcomplexity lgorithms with pplictions in wireless communictions. In the context of Mssive MIMO, his min reserch focus is on how relistic propgtion chrcteristics influence system performnce nd bse-bnd processing complexity.