4GCOVERAGE& SPEED ASSESSMENTIN BOURNEMOUTH, MIDDLESBROUGH &EDINBURGH

Transcription

1 4GCOVERAGE& SPEED ASSESSMENTIN BOURNEMOUTH, MIDDLESBROUGH &EDINBURGH

2 BACKGROUND Mobile connectivity is now viewed as an essential utility service for most people in today?s modern and increasingly interconnected world. Consumers use their mobile devices in their homes, places of business and while commuting and expect a sustained quality of service from their service providers as they go about their daily lives. In the UK, Mobile Network Operators (MNOs) have enforceable minimum coverage thresholds attached to their licenses. The four main MNOs are expected to provide voice coverage across 90% of the UK total land mass by the end of As part of its 4G license obligations, O2 is expected to provide indoor coverage to 98% of the UK population by the end of The other operators have expressed their intention to match O2?s 98% population coverage obligation for their own 4G networks. Providing 4G population coverage is one thing; however, satisfying the much more difficult geographic coverage requirements without any coverage gaps is a much more demanding proposition. Pu blic Saf et y 4G In addition to commercial consumer-grade quality of service coverage requirements, 4G coverage is set to become a part of our critical national infrastructure for public safety and national security in the coming years. Between 2018 and 2020, the UK's emergency services and other public safety organisations will transition off the narrowband Airwave TETRA system onto the new emergency services network (ESN) running over EE's commercial 4G network which will be enhanced for mission critical voice and data requirements including full national coverage. With the safety of the emergency services and the general public paramount to a successful transition, there should be no compromise in network coverage or service quality. The Emergency Service Mobile Communications Programme (ESMCP), which is responsible for managing the roll-out of this critical infrastructure intends to transition users from Airwave to the ESN using a staggered approach based on geographic regions. The first emergency services organisations are expected to transition to ESN by mid-2018 and all organisations across Great Britain to have transitioned before the end of With such a challenging timetable to deliver this new technology for mission critical applications, it is imperative that the right testing frameworks be used during initial and live testing prior to sign off and live operation of the ESN service. This is to ensure that 4G coverage and quality of service is ubiquitous and up to the required specification over the entire UK landmass. Con n ect ed an d Au t on om ou s Veh icles 4G and its evolutionary enhancement releases will likely be a viable technology candidate for vehicle-to-everything (V2X) connectivity which will enable cars to talk to everything around their surroundings, from other vehicles to pedestrians and traffic signals, ushering in the era of the connected cars and the evolution from automated driving to fully autonomous vehicles by the end of the 2020s. The UK is committed to becoming a world-leader in connected and autonomous vehicle technology with the Government providing funding for the research, trialling and development of autonomous vehicles in partnership with the automotive industry. With a vibrant ecosystem increasingly gaining momentum and Qualcomm announcing in early September 2017 that its cellular vehicle-to-everything (C-V2X) chipset will be released by the end of 2018, the mobile network operators need to take advantage of this opportunity and ensure that the required connectivity infrastructure is in place and adequate coverage provided on the motorway and urban road networks. Against this backdrop, AWTG Ltd, a service delivery partner for the telecommunications industry and a leader in the deployment and testing of next generation networks, conducted an assessment of 4G coverage and mobile broadband speeds in 3 cities in the UK: Bournemouth, Middlesbrough and Edinburgh and the motorway road networks from Bournemouth to Edinburgh.

3 M ETHODOLOGY AWTG is a leading telecommunication services provider specialising in Networks, Smart Infrastructure, Internet of Things (IoT) and Machine to Machine (M2M) communications. We have been coverage testing and optimising mobile networks since 2006 and are a trusted partner for central and local government bodies, national regulators, local enterprise partnerships and mobile networks around the world with proven experience in conducting independent mobile network benchmarking tests. AWTG has extensive experience working on next generation technologies and is proud to have partnered with the University of Surrey to help deliver the 'first of its kind in the world' fully comprehensive 5G test bed at the dedicated 5G innovation Centre. The test bed comprises of a 44 site end-to-end self-contained network with all the elements and functionality of a commercial network. We are actively working with various partners in the UK to implement 5G test beds designed to trial specific 5G use cases such as connected and autonomous vehicles, advanced manufacturing and massive IoT. Our consultancy and advisory teams have developed an intuitive cost modelling tool to help Operators estimate the cost of 5G network deployments based on multiple scenarios and use-cases. 4G Cover age Assu r an ce M et h odology There are two main approaches to testing and measuring network performance, either through the tried and tested Drive Testing field campaigns or more recently Application based crowd data analysis. Proponents of the App based approach would claim that their methodology is a viable means of measuring network performance due to the volume of data it can potentially collect. While we do appreciate that this can provide an indication of user experience, in our opinion we don't believe it can be touted as a like-for-like replacement of drive testing, especially when coverage needs to be comprehensively assured as in the case of critical services such as ESN and connected and autonomous vehicles. App based data is not a geographical representation of network performance but rather an experiential characterisation of performance from a dataset that may not be reproducible. Typically the App based approach will use the proportion of time users are on a 4G signal to draw conclusions of 4G coverage availability. AWTG however uses a rigorous, controlled and scientific field testing methodology to collect network coverage samples that are both statistically valid and reproducible. In all areas that we test for coverage availability and network performance, we use consumer-grade mobile devices capable of collecting high volumes of coverage samples and drive along all major and minor roads spanning metropolitan, urban and rural areas whilst ensuring that data is collected at all times of the day. We also complement our drive testing with stationary and indoor data collection at randomised geographical locations to gain a complete and accurate representation of network performance. For this coverage assessment, we drove a total distance of 3,000 miles in September 2017 and conducted stationary tests in over 30 locations collecting over 1.5 million 4G coverage samples. This data was processed and a minimum signal threshold of -115 dbm, which is the signal level that guarantees an acceptable QoS, was applied to determine if a location was covered or not. For network performance, upload and download speed tests were conducted using the speedtest.net application. The results were then aggregated and compared with the Opensignal/ which? report on regional 4G Availability and Speed for Bournemouth, Middlesbrough and Edinburgh published in May

4 SUM M ARY OF FINDINGS 4G Cover age The results of our 4G coverage assessment in Bournemouth, Edinburgh, Middlesbrough and the M3, M1, A1(M) and A697 road networks between Bournemouth and Edinburgh indicate that the overall 4G coverage in each of these geographic locations is above 90% as highlighted in Figure 1 below. Of the 3 cities tested, we found that Bournemouth has the highest geographic coverage with 96.96% availability while Middlesbrough has the lowest 4G coverage with 92.21%. These results are in significant contrast to those published by Opensignal/ Which? in their recent report. The main reason for the difference between Application based crowd-sourced coverage availability data and our coverage availability data is that App based network assessments use time as an indicator of network availability and performance by inferring the degree of availability from the proportion of time the mobile application users report being on a particular technology. However the geographic distribution, movement and dwell time of mobile users in a particular location cannot be controlled, which means that the modelled results may not be truly reflective of the actual network performance on the ground. Almost all the MNOs in the UK configure the 4G minimum receive level threshold at the enodeb level at around -125 dbm, meaning that users below this threshold will not be able to log on to their providers' networks. For coverage availability, AWTG used a more stringent threshold of -115 dbm which ensures a better QOS. Figu r e 1: 4G Cover age Availabilit y

5 SUM M ARY OF FINDINGS 4G Dow n load Speed 4G download speed results in the 3 cities and motorways tested show an overall average speed performance of Mbps as presented in Figure 2 below. Top average 4G download speeds were recorded in Edinburgh at 26.85Mbps, while the average 4G download speeds in Bournemouth were the lowest at Mbps. 4G speeds were measured using the speedtest.net application and testing was conducted while driving on the city roads and motorways as well as at stationary locations within the 3 cities. Indoor testing was also conducted at various locations and during all times of the day to gain a comprehensive picture of 4G data speed performance. Overall for 4G download speeds, AWTG's test results were similar in Bournemouth to Opensignal's results, but considerably different in Edinburgh and Middlesbrough where our results showed a higher throughput. Figu r e 2: 4G Dow n load Speeds

6 SUM M ARY OF FINDINGS 4G Upload Speed 4G upload speed results in the 3 cities and motorways tested show an overall average speed performance of Mbps as presented in Figure 3 below. Top average 4G upload speeds were recorded on the Motorways at 12.43Mbps, while the lowest average 4G upload speeds were recorded in Bournemouth at Mbps. It is interesting to note that comparing between the individual network operators, only one of them managed average upload speeds in excess of 25 Mbps across all the locations tested. The other operators did not even cross the 20 Mbps threshold for average 4G upload speeds. The QoS requirement in the uplink direction will be an important parameter for services such as the ESN and connected and automated vehicles as large amounts of voice, video and sensor data is transferred across the networks. Drive testing provides a scientific and methodical approach to network performance and coverage assurance. Drive routes are carefully planned to deliver a truly representative picture of network performance across the extent of the area under test. At the end of the campaign you can be confident about where network resources need to be focused to improve service, coverage and performance to meet the required quality of service.

7 4G COVERAGE: ROUTES Bou r n em ou t h Edin bu r gh M ot or w ay M iddlesbr ou gh Figu r e 4: 4G cover age in t h e t h r ee cit ies of Bou r n em ou t h, Edin bu r gh an d M iddlesbr ou gh an d t h e M 1, M 3, A1(M ) an d A697 r oad n et w or k s f r om Bou r n em ou t h t o Edin bu r gh.

8 4G COVERAGE: M OTORWAY Figu r e 5: High cover age availabilit y on t h e M 3, M 1, A1(M ) an d A69 r oad n et w or k s bet w een Bou r n em ou t h an d Edin bu r gh. Com bin ed 4G cover age f r om all oper at or s w as m easu r ed at 96.57%

9 4G COVERAGE: EDINBURGH Figu r e 6: 4G geogr aph ic cover age availabilit y in Edin bu r gh sh ow in g a com bin ed availabilit y of 94.94%

10 4G COVERAGE: M IDDLESBROUGH Figu r e 7: 4G geogr aph ic cover age availabilit y in M iddlesbr ou gh sh ow in g a com bin ed availabilit y of 92.21%

11 4G COVERAGE: BOURNEM OUTH Figu r e 8: 4G geogr aph ic cover age availabilit y in Bou r n em ou t h sh ow in g a com bin ed availabilit y of 96.96%

12 Road & Cit y Per f or m an ce per Oper at or Figu r e 9: EE r ecor ded t h e f ast est dow n load speeds on bot h t h e cit ies an d t h e r oad n et w or k s Figu r e 11: Th e f ast est u pload speeds w er e exper ien ced on t h e EE n et w or k on bot h t h e r oad n et w or k s an d in t h e cit ies Figu r e 10: Dat a t r an sf er r eliabilit y st at ist ic pr ovides t h e per cen t age of f ile t r an sf er s t h at w er e su ccessf u l per n et w or k oper at or an d per clu t t er t ype (cit y an d r oads) Figu r e 12: Net w or k lat en cy m easu r es t h e aver age r ou n d-t r ip t im e of t r an sf er r in g dat a f r om a ser ver on t h e In t er n et t o t h e u ser device. Th r ee h as t h e best lat en cy on t h e r oads, w h ile EE w as t h e best in t h e cit ies

13 CONCLUSION In September 2017, AWTG launched a 4G network performance assessment in Bournemouth, Edinburgh and Middlesbrough and the motorway road networks between Bournemouth and Edinburgh to baseline the current network performance in these locations. Comparing the results of AWTG's 4G network assessment to the Opensignal/ which? regional network performance report for UK published in May 2017, we note a significant difference in the 4G coverage availability data with a variation of up to 30%. This type of assessment is absolutely essential to provide assurance and confidence to the user community and the general public at large that commercial networks can be relied upon to deliver mission critical services such as the ESN. In addition, Connected and Autonomous Vehicles will also require ubiquitous and highly reliable mobile coverage along the road network, if the technology is going to be safe, viable and trusted by drivers. The results of the benchmark assessment provide some interesting insights. 4G availability in all locations tested measured above 90%. 4G coverage along the motorways tested was very good with a geographic availability of 96.57%. 4G download and upload speeds indicated a strong performance from the mobile network operators in the areas tested. " When considering mission critical services and applications, it is our firm belief that only dedicated drive testing campaigns can provide the necessary level of assurance that the user community, service providers and operators require" Typically, App based network assessments use time as an indicator of network availability and performance by inferring the degree of availability from the proportion of time the mobile application users sends a report that they are on a particular technology. However, the geographic distribution, movement and dwell time of mobile users in a particular location cannot be controlled, which may in turn lead to a distortion of the results. While App based crowd-sourced data can sit alongside and complement Drive Test data, it should not be seen as direct replacement for it. In addition, the raw data collected during drive testing also includes much more signalling data between the end device and the network allowing for additional Layer 3 analysis in the air interface protocol to be conducted to pinpoint network performance issues and provide solutions for the network operators.

14 Abou t AWTG: AWTG is a leadin g t elecom m u n icat ion s ser vices an d IoT plat f or m pr ovider specialisin g in Net w or k s, Sm ar t In f r ast r u ct u r e, IoT an d M 2M solu t ion s. We deliver a f u ll r an ge of bu sin ess t o bu sin ess an d n et w or k r elat ed ser vices t h at ar e bot h cost ef f ect ive, scalable an d deliver r apid r et u r n on in vest m en t. AWTG w or k s w it h ou r clien t s t o qu ick ly iden t if y n eeds, design an d deploy ef f icien t solu t ion s an d pr ovide on -goin g m ain t en an ce an d cu st om er car e ser vices. Fou n ded in 2006, AWTG pr ovides t h e h igh est degr ee of pr of icien cy, f lexibilit y an d on dem an d scalabilit y t o h elp clien t s low er cost s, der ive valu e by st r eam lin in g oper at ion s an d u lt im at ely im pr ove pr odu ct ivit y. For m or e in f or m at ion on AWTG?s cover age assu r an ce t est in g, con su lt in g an d in -bu ildin g cover age ser vices, please con t act : or en qu ir ies@aw t g.co.u k 8 Can h am Road, Can h am M ew s, Lon don, W3 7SR, UK. w w w.aw t g.co.u k ISO 9001 accredited