Benchmarking latencies of new Azure and AWS regions in Middle East

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

Microsoft Azure and Amazon AWS Launched new data centres in the summer of 2019

First Microsoft Azure and then, soon after, Amazon AWS launched new data centres in the Middle East. Azure launched in United Arab Emirates in June and AWS launched in Bahrain in July. In this report we examine the latencies achieved to these servers and compare them to existing options such as hosting in EU or using CDNs with presence in the Middle East. We compare latencies from 13 countries in the Middle East and identify the winners and losers.

Microsoft and Amazon are not short of ambition. Here is how Sayed Hashish, Regional General Manager, Microsoft Gulf described the new regions:

 

“These new cloud regions in the UAE are the dawn of a new era, driving digital transformation, economic growth and job creation.” 

 

In this report we show which countries should switch to the new data centres if latency and network performance is important and, just as important, which countries should stay where they are (at least for the time being).

SpeedChecker Measurement Methodology

SpeedChecker analysed measurements collected from over 50,000 latency tests taken during business hours for one day in August 2019. This is how the samples were collected:

  • From both cellular and fixed connections
  • The distribution of samples between cellular and fixed connections were approximately equal except for Iraq and Palestein where there were a bigger proportion of fixed connections.
  • The same number of samples were taken from each country to each of the CDN providers used in our report to ensure equal comparison
  • Location of devices determined by geographical information on the device to be within one of the 13 countries in the Middle East as shown in the results
  • Latency tests results include samples from all MNOs/ISPs used by the users during the sampling period.
  • We compare latencies of new Azure and AWS regions with established CDN in Middle East – Cloudflare, which offers decent coverage, performance for a reasonable price. We also compare against Digital Ocean Amsterdam datacenter to have a reference point when content is hosted outside the Middle East.

Our tests are designed to accurately measure the latency experienced by the user at the time of test. We further analyse the results to remove any tests that fail our quality control including where there is some anomaly in the result that makes it unreliable.

Results

Looking at the mean latency from all our results in the Middle East shows that connections to Cloudflare and Digital Ocean provide the lowest latency. It is to be expected that Cloudflare would do well because they have data centres in all of the countries in our report except for Palestine. The second fastest (Digital Ocean – EU) is some 35% slower than Cloudflare.

It is surprising to see that the Bahrain and UAE servers from Amazon AWS and Azure are both slower than the Digital Ocean EU servers despite being closer geographically. There is not much to choose between them with both of them being less around 15% slower than the Digital Ocean EU servers and some 60% slower than Cloudflare’s local Middle East servers.

 

Chart showing Average latency across all Middle East Countries

 

The chart below compares the latency from each country to each of the four data centres. Amazon and Azure do best in Bahrain and UAE (their host countries) but there is little difference in the other countries. Cloudflare produce some good results in Bahrain, Turkey and Qatar.

Mean latency for Middle East countries

Winners: Bahrain and United Arab Emirates

In this chart we compare latencies to the data centres from the United Arab Emirates, Bahrain and other countries (excluding UAE and Bahrain). Not surprisingly we see that UAE and Bahrain get the best results from Azure and AWS data centres located in their countries. What is striking is that UAE gets a very similar performance from the AWS server in Bahrain as it does from its local Azure server.

Bahrain also does well when using Cloudflare servers but below the average of other countries when using Digital Ocean. UAE have slower results than average to both Digital Ocean and Cloudflare data centres making the introduction of local data centres all the more important.

We also see that other countries gain little from the new data centres with average latency being slower to AWS and Azure than to Cloudflare or even the Digital Ocean EU data centres.

 

Chart showing Comparison of latency between Bahrain, UAE and Other ME countries

Conclusion

If latency and network performance is critical for your business in the Middle East, you have to carefully decide which cloud to choose. As indicated in this post there are some staggering differences between latencies to Azure and AWS clouds depending on the countries your users are from. If your business caters to Bahrain or UAE, you cannot go wrong by picking any of the two platforms. However, if you expect decent latencies across the whole Middle East, you should consider using CDNs such as Cloudflare or Akamai which have a good footprint in the region with good connections across the majority of the countries.

The new AWS and Azure data centres are still, unfortunately, not well connected beyond Bahrain and UAE and, as you can see from our data, the latencies are similar to those you would expect by hosting your content in Europe. This could well change and perhaps quite soon and we will continue to monitor performance and report on any improvements.

Download pdf

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

Mobily Overtakes STC in Hajj 2019 Mobile Internet Speeds

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

Hajj 2019: 2.5 Million Visitors Puts a Huge Demand on MNOs

Saudi Arabia has confirmed that nearly 2.5 million pilgrims performed the major Islamic pilgrimage to the holy city of Mecca this year. This puts a strain on the city’s infrastructure and we have looked at how the MNOs coped with the increased demand for Internet access and compared the results with 2018.

We compare the download and upload speeds of the STC, Mobily and Zain on cellular networks between 8th and 15th August 2019 with each other and with their performance in 2018.

For Hajj last year (2018) the King Salman bin Abdulaziz and Crown Prince Mohammed bin Salman issued a directive “to do everything possible to make it easy for pilgrims to perform the rituals of Hajj”. The initiative’s objective was to allow pilgrims to communicate with their families and enable them to access the digital services available in the Smart Hajj initiative. This was a great success as seen in our report from last year. In this report we see that this has continued for 2019 and look forward to 2020.

Speedchecker Measurement Methodology

We analysed cellular data collected from over 50,000 speed tests taken between 8th and 15th August 2019. This data was restricted to include only data that was:

  • Within the city of Mecca
  • Cellular only (not fixed)
  • Provided via the top 3 MNOs operating in Mecca (STC, Mobily and Zain)

Our speed tests are designed to accurately measure the speed experienced by the user at the time of test. We further analyse the results to remove any tests that fail our quality control including where there is some anomaly in the result that makes it unreliable.

We chose to limit to cellular because this is the type of connection most likely to be used by visitors as well as local residents and therefore more likely to be impacted by the surge in demand during Hajj.

The top 3 MNOs (STC, Mobily and Zain) were chosen because they account for almost all connections in Mecca.

Data Collection

Speedchecker collected the data used in this report using the crowd sourced data samples collected in the field on mobile devices. During the time the data collection took place, Speedchecker received over 80,000 data samples and the included statistics and analysis are based on this dataset.

It was important for us to not only measure the real speeds as experienced by the user but also to ensure that we measured to the same points to ensure a fair comparison. For this we included only measurements to servers in Europe. Read the following section “Choice of measurement server” for more about this method.

Choice of measurement server

Every measurement methodology differs in the selection of measured server infrastructure. Some methodologies focus on on-net servers hosted in telecom premises and others focus off-net. Speedchecker believes off-net servers’ measurements are representing real user experience better than on-net as most of the content accessible by end users lies off-net. Most of the content these days is hosted on CDN networks and the best way to measure most relevant download metrics is to choose popular CDN networks. However, as we learned in our March 2019 Middle East Speed Rank report, not all ISPs and MNOs connect to CDNs directly (or peer) and that has major implications on performance.

Our analysis in March shows that 68% of results from Saudi Arabia were routed via European servers. The results in this report are therefore based on results using European servers to ensure meaningful comparisons. We found no significant difference in performance between the MNOs based on which particular EU servers were being used. It is acknowledged that MNOs that use more local servers may achieve faster results (up to 30% faster than EU servers) based on our March 2019 report, giving a potential margin of error of around 10%.

 

Chart showing EU and Local server use by middle east countries.

Results

We saw last year that speeds improved not only during Hajj but also that there was a residual benefit that was sustained afterwards. We were curious to see what speeds were achieved 12 months later and found speed improvements of between 20 and 100%.

Download Speed Improvement

From over 50,000 results we see that there has been an incredible increase in average download speeds of over 70%. Mobily were by far the most improved with their average speed more than double that of Hajj 2018. This gives them an overall average download 1.2 Mbps faster than STC and nearly double that of Zain. Zain should be given credit for a nearly 50% improvement which, in normal circumstances, would have been more noteworthy.

 

Chart showing the % increase in download speeds between 2018 and 2019 for Saudi Arabia MNOs

 

Table showing the % increase in download speeds between 2018 and 2019 for Saudi Arabia MNOs

2019 Results for Mobily, STC and Zain

Comparing download and upload speeds across the top 3 MNOs Mobily are faster for downloads and STC are faster for uploads. Zain compares better for upload speeds than it does for downloads.

 

Chart comparing download and upload speeds for Mobily, STC and Zain

 

Chart comparing download and upload speeds for Mobily, STC and Zain

Conclusion

There is good news for all customers with all three MNOs out-performing their 2018 results. Clearly Mobily have excelled and I suspect this has been helped by their partnership with Ericsson that started in 2018:

Eng. Ahmed Aboudoma, Mobily Chief Executive Officer, Mobily, says: “This agreement with Ericsson is in line with the Saudi Vision 2030 and its objectives that fall within developing the Telecom and IT sector, in addition to preparing best-provided services to customers that comply with its new strategy ‘RISE.’ This includes a concept that raises the level of provided services to customers by using the latest telecom technologies.” [source: Ericsson]

Mobily have also teamed up with Huawei to upgrade its infrastructure towards 5G. This may not have had time to account for the improvement we have seen in 2019 but should help to deliver more improvement in 2020. STC are also upgrading to 5G with help from Nokia as well as Huawei [source: RCR Wireless ].

Once again, Mecca excels at delivering excellent communications as it copes with the huge demand of Hajj.

 

Download pdf version of this post

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

What Happened to the Internet when 7 million people visited Jeddah during Ramadan

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

Introduction 

In 2019 7 million visitors arrived in Jeddah during Ramadan (5th May to 3rd June) compared to its normal population of between 4 and 5 million. Google’s own insight report shows that demand increases during Ramadan as well. In this report we examine how the MNOs (Mobile Network Operators) coped with this increased demand. We analyse speeds between 17th April through to the 13th June to compare average download speeds before, during and after Ramadan.

Ramadan, Jeddah and the Internet

Internet Use Peaks During Ramadan

Internet use in MENA (Middle East and North Africa) during Ramadan is higher than any other time of the year. Google internal data shows huge spikes across several content categories on YouTube throughout the holy month each year, including TV & Comedy, Cooking & Recipes, Religion & Spirituality, Gaming, Auto & Vehicles and Internet & Telecom. Here is a chart from Google’s report in 2015 showing demand for TV and comedy downloads rising during Ramadan and falling after Ramadan.

 

MENA YouTube Viewership Ramadan 2015 from Google:

Chart showing peak demand for TV streaming during Ramadan

 

 

This peak is further emphasised in Google’s 2017 Ramadan Insight report in which they show:

“Watch time for content related to ‘TV drama series’ rises a staggering 151% in Ramadan compared to any other period in the year”

Google’s statistics are backed up by reports of increased smartphone usage during Ramadan such as this article from Mid East Information: Major surge in smartphone usage expected during Ramadan in KSA.

“Ramadan is one of the most active times of the year for social media in the Middle East – on all social media channels – as Muslims reach out to friends and family, prepare for the holy month, and celebrate in the run up to Eid.”

Jeddah Welcomes 7 Million Visitors

The Kingdom of Saudi Arabia issued over 7 million Umrah visas for 2019 and as of May 2019 6,964,943 have arrived in the Kingdom. Most of these will arrive in Jeddah because Jeddah airport is the gateway to the capital of the Kingdom of Saudi Arabia and it is the national air transport center.

Methodology

We analysed  data before, during and after Ramadan between 17th April and 5th June. This data was restricted to include only data that was:

  • Within the city of Jeddah
  • Cellular only (not fixed)
  • Provided via the top 3 MNOs operating in Jeddah (STC, Mobily and Zain)
  • Midday (12 to 2pm) or Evening (8pm to 10pm)

Jeddah was chosen because the number of visitors expected to pass through during this time could be expected to have a significant impact on the service provided by MNOs. We were keen to see how the MNOs coped during this time.

We chose to limit to cellular because this is the type of connection most likely to be used by visitors as well as local residents and therefore more likely to be impacted.

The top 3 MNOs (STC, Mobily and Zain) were chosen because they account for almost all connections in Jeddah.

We chose evening because we anticipated the demand to be high during those hours. Midday was chosen to compare the quality of service during two different parts of the day with different demands on the Internet.

 

Table showing dates when data was collected

Results

STC and Mobily achieved the best results with nothing to choose between them. Although Zain achieved lower speeds credit is given for sustaining their performance during this spike in demand.

 

Chart showing how Internet speed changed before, during and after Ramadan for STC, Mobily and Zain

 

We were also interested to see how performance varied during the day from the quiet of midday to the peak viewing / download time of the evening. Here we see that the speed in the evening was always well below the speeds achieved during the day. Even so, the evening speeds improved by 30% during Ramadan.

 

Chart showing how Internet speed changed before, during and after Ramadan by day

 

For Hajj last year (2018) the King Salman bin Abdulaziz and the Crown Prince Mohammed bin Salman issued a directive “to do everything possible to make it easy for pilgrims to perform the rituals of Hajj”. The initiative’s objective was to allow pilgrims to communicate with their families and enable them to access the digital services available in the Smart Hajj initiative. This was a great access as seen in our report from last year.

Although we didn’t see any similar initiatives being advertised in Jeddah during Ramadan it appears from our results that efforts have been made by MNOs and the Kingdom to sustain performance during this high demand.

Conclusion

We expected to see that as more and more people arrived in or passed through Jeddah during Ramadan that the speeds achieved would fall and then return to previous levels as the demand diminished. However, all three MNOs achieved remarkable results showing an increase in measured speeds during Ramadan compared to the weeks before. Although there was some reduction in speed after Ramadan it seems to have sustained a higher level than before. As discussed previously, the increased demand for Internet was predicted well in advance giving MNOs ample time to prepare. We can only assume that they used this time wisely and their customers (and guests) reaped the benefit.

It is heartening to see that there seems to be a residual benefit to customers of all three MNOs with the speeds after Ramadan being higher than before (if not quite as high as experienced during Ramadan).

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

EE Tops the Charts at Glastonbury

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

EE Tops the Charts at Glastonbury 2019

The 200,000 festival goers at the 2019 Glastonbury Festival were treated to free 5G via temporary masts installed by EE as it became the UK’s first 5G connected festival. In this report we look at what this meant for the music lovers as they enjoyed the music over 5 days in the English countryside. Did EE customers notice an improvement? How did the other MNOs fare?

 

Photo of crowds at Glastonbury
Photo of crowds at Glastonbury. 2017 Image courtesy of creative commons. Cropped to fit.

Temporary 5G Network Installed by EE to Serve 200,000 Users

EE (owned by BT) have installed 5 temporary masts across the 900 acre (3.5 square kilometres) festival site to broadcast 5G for the first time. The same masts are also providing 2G, 3G and 4G. They have a free 5G-powered Wi-Fi for the 200,000 festival goers to connect to.

In previous years the demand for mobile data during the festival was huge and this year will be even greater. During the previous festival (2017) 54 terabytes of mobile data was used and it is expected that this year it will be 40% higher at 70 terabytes. Possibly higher if the temporary EE 5G network is successful. Research from EE shows that the typical Glastonbury-goer will “watch 16 different performers and post a minimum of 12 videos on social media documenting their experiences at Worthy Farm – resulting in 2.4 million uploaded throughout the weekend”.

The image below is an example of the temporary masts erected by EE at Glastonbury to provide improved 2G/3G/4G service as well as 5G.

 

Photo of EE Temporary mast at Glastonbury
Photo of EE Temporary mast at Glastonbury. Image courtesy of pedroc.co.uk

Methodology

We started collecting data when people started to arrive on Wednesday and continued through to Sunday. We concentrated on results in and around Worthy farm and only analysed cellular results. The only residential area was the small village of Pilton (population < 1000) which means that by restricting our analysis to cellular we can be sure that most of the data came from festival goers.

The image below shows the remoteness of the festival and how challenging it can be to provide a good service. The circle shows an approximation of the geographical area covered by our results.

 

Aerial view of Glastonbury showing internet data coverage
Aerial view of Glastonbury showing internet data coverage. Image courtesy of Google.

There are many factors that can affect the quality and accuracy of a speed test on a mobile device and we are able to take these into account to grade each result in terms of reliability. For this study we decided to only use the most reliable 2500 taken during the festival.

Results

Our analysis is based on 2500 measurements from 468 unique devices including 145 different models, 23 manufacturers and 4 MNOs taken over the 5 days of the festival. We compare the performance of the MNOs, the devices and also show how performance varied from day to day.

MNOs

During the festival 4 mobile network operators (MNOs) provided internet to the festival goers with EE, O2 and Three being the most popular. The results show that EE was able to provide nearly twice the average download speed with Vodafone clearly in second place but still very much slower than EE during the festival.

 

Glastonbury speeds for EE, Vodafone, Three and O2

Chart showing Download speeds by MNO
Chart showing Download speeds by MNO

Devices / Phones

During the festival we collected data from 468 unique devices including 145 different models from 23 different manufacturers. Here are the top 10 fastest phones from our data collection during the Glastonbury Festival. The list is dominated by Samsung models 9 and 10 with the S10 being the best by far.

 

Glastonbury speeds for devices samsung galaxy, google pixel huwawi mate

Daily

The speeds were quite good on Wednesday when people started to arrive and the final preparations were being. The best day was Thursday, the first full day of the festival and speeds gradually fell away as more people arrived and as the bigger acts appeared. The bigger the act the more videos being uploaded to the Internet and the higher the demand on the services. EE performed particularly well on the final day being twice as fast as its nearest rival, Vodafone.

 

Glastonbury speeds for EE, Vodafone, Three and O2 by day

Chart showing internet speeds at Glastonbury on each day
Chart showing internet speeds at Glastonbury on each day

Conclusion

Glastonbury festival goers have enjoyed excellent performance whilst enjoying the music in the English countryside with 30% enjoying download speeds in excess of 20 Mb/s. 20 Mb/s is also the average for all customers which is higher than expected and is due to the nearly 35 Mb/s average achieved by EE customers thanks to the provision of temporary masts by EE. Although these masts supported 5G we saw no 5G devices being used in our crowdsourced data sample of 468 unique devices.

Of course the real winner was music but EE must be very happy with their performance throughout the festival.

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

SpeedChecker CPE Wi-fi Performance Ranking

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

SpeedChecker analysed 300,000 speed test results to produce a Wi-Fi performance ranking of CPE (routers) around the World. Our results show that the choice of router has an important impact on the perceived speed of the Internet even for users of the fastest devices and the with the strongest signals.

 

In this white paper we explain how our methodology eliminates other causes of slow speeds and presents a ranking table of routers for the top 20 manufactures. We also show the cost of the routers to identify the routers that offer the best value for money. The conclusion of our study show that there are some clear differences between manufacturers and between the models that each provide.

 

Ranking Methodology

The purpose of this white paper is to rank Customer Premises Equipment (routers) in order of Wi-Fi throughput, to help consumers and providers to diagnose slow internet connections. There is often disagreement between the consumer and the provider regarding where the fault lies for slower than expected Internet connections. There are many causes for a slow internet and this white paper explores the impact of the CPE (router) on the final speed perceived by the consumer.

 

In this section we explain the methodology used to ensure that we are able to confidently rank routers to identify the fast and slow routers. This will help to diagnose poor speeds because the data will allow the consumer or provider to know what to expect from the routers being used. The methodology is designed to allow for a fair comparison between routers.

 

This white paper is based on 300,000 speed results collected between 1st April and 10th June 2019.

 

We analysed these 300K results and filtered them to eliminate other factors that could be reducing the speed of the CPE. We eliminated any speed tests that failed our rigorous accuracy check, any tests that used devices not capable of the fastest speeds and any tests that had a poor signal quality. We also disregarded very strong signals that may not have been representative of the average user. Finally, we disregarded any model of testing device or router that had a small data sample. From the 300K we found 46K that met our criteria.

Because of the filtering, we have reduced the CPE models to the most popular ones used by the users. A more comprehensive study ranking a wider set of CPE models will be published in the future.

Countries

Our data set includes results from over 100 countries however the vast majority of results are from  France, the United Kingdom and the United States.

 

Distribution of results by country

Client Devices

During the time that SpeedChecker collected results there were over 600 different client device models used (mobile phones & tablets). To ensure that the devices themselves were not the cause of a slow connection we removed results from slow devices, those with insufficient results and those that were not using the 2.4 GHz band.

 

We removed devices that had a maximum link speed of less than 100 Mb/s and an average throughput of less than 30 Mb/s.

 

There were 75 devices that met our criteria to be included in the analysis of the CPEs (routers). These are all Android devices because data from iOS devices was not available for this study. SpeedChecker plan to include iOS devices in future studies.

 

The most popular fast devices in order of results per device:

 

The most popular fast devices in order of results per device

 

Samsung are clearly the most popular device but how do they compare in terms of throughput? The following table shows the average throughput in Mb/s for the top devices. Samsung have 5 of the top 10 but the fastest in our sample was the LGE V30.

 

Mobile devices in order of throughput speed

CPE (Routers)

There were over 400 routers in our sample of 300,000. After eliminating the results from the devices as previously described there still remained over 360. However, for many of these routers there were too few samples to be statistically sound. Removing these left 78 routers that we could analyse with confidence.

Methodology:

Wi-Fi Throughput Test

The Wi-Fi Throughput Test shows the average maximum measured transfer speed between the device and the CPE (router) using the UDP protocol. Wi-Fi throughput will normally be higher than or equal to the Internet speed.

Internet Download Test

The Internet Download Test is the actual Internet speed experienced on the device during the SpeedChecker speed test using TCP protocol. Our test uses sophisticated procedures to ensure that the result is as accurate as possible.

How SpeedChecker calculate a slow Wi-Fi connection

In order to determine if a particular result shows that the CPE (router) was responsible for a slow internet experience, SpeedChecker compared the measured Wi-Fi throughput with the Internet Speed test. SpeedChecker looked at the Internet Speed as a % of the Wi-Fi Throughput. Any result that shows an Internet speed greater than 80% of the Wi-Fi Throughput speed indicates a slow CPE (router).

 

The following table assumes that a customer is being provided with a 100 Mb/s Internet connection and compares how a slow CPE impacts on the speed they actually experience. If the throughput is slower than the Internet being provided to the premises then the actual speed being experienced is limited to that throughput speed.

 

In the worst case we see that, although there is 100 Mb/s coming into the premises, the Wi-Fi throughput of the CPE is only 50 Mb/s. A better throughput of 150 Mb/s gives the customer the full 100 Mb/s that is being provided with a ratio of 67%. For our analysis we consider any speed test result that has a throughput speed between 80-100% of the measured Internet speed to be slow. In our ranking tables we include the % of slow results for comparison.

 

Slow results lookup table

Drawbacks of our methodology:

Our results rely on the CPE (router) having UPnP enabled on the router. Although many routers have it turned on by default there are still a significant number of popular routers that will not appear in our list because UPnP is disabled by default on them. Some users may enable it on these but if there are not enough results to be statistically significant they will not appear in our tables.

 

Our methodology mitigates against the impact of the wifi setup such as poor Wi-Fi signal and slow/old devices by only including results with strong signals and those results from devices capable of faster speeds. We also mitigate against local impacts on the results during the test itself by eliminating poor results.

 

The steps that we have taken to ensure accurate ranking of CPEs means that the 300K raw results have been reduced to 50K high quality results to ensure a fair comparison.

How SpeedChecker Filter the Data to Produce Meaningful Results

The purpose of this report is to provide an analysis of the speed results to show the best routers in terms of Wi-Fi Throughput. This will allow comparison of routers (CPEs) and help to identify if the router is causing a poor internet performance.

 

It was important for us to ensure that we only used speed results that provide information that was reliable, significant and relevant to most users. To do this we filtered our results based on :

  • Reliability
  • Device Speeds
  • Router popularity
  • Wi-Fi Signal
  • Wi-Fi Type
  • Number of results per router and device

Reliability

SpeedChecker take great care to ensure that any speed test result that we use is reliable and accurate. The results that are eliminated include incomplete tests, tests with too few samples and tests that take too long to complete. This can be caused by local interference such as the user moving during the test, the user cancelling before completion and interference during the test by other devices or other apps.

Device Speed

Because we are interested in the performance of the CPE we have included only phones and devices that are capable of the faster speeds. Otherwise, it would not be clear if it was the router or the device that was causing the slowness.

 

We used the top 75 devices from over 600 devices in the raw data. These all had maximum link speeds of at least 100 Mb/s and we had sufficient results from each device to be statistically significant. To qualify for this list a device was also required to have a minimum of 250 results.

 

The average Wi-Fi Throughput for these devices ranged from 30 to over 100 Mb/s.

Router Popularity

We list the most popular routers based on the number of data samples for each router and their popularity in Europe. It should be noted that some of these routers are offered as ISP brands. A router needs to have sufficient results to allow for statistically sound analysis.

Wi-Fi Signal

A weak Wi-Fi signal will also produce slow results and would cause a router to be mis-reported as being slow. We looked at the number of results in each band of signal strength and, as seen in the chart, we saw that most of them were between -40 and -70 dBm. We filtered out any results worse than -70 dBm. We also filtered out strong signals (above -40dBm) to ensure that our analysis focuses on the more challenging wireless conditions where users typically struggle with wi-fi, and ideal conditions where most of the CPEs are working well.

 

Distribution of Results by Signal Strength (dBm)
Distribution of Results by Signal Strength (dBm)

 

Wi-Fi Type (2.4 GHz or 5 GHz)

Our analysis restricts results to those using the 2.4 GHz band to make the comparison equal across all routers. We chose 2.4 Ghz because our results show that there are twice as many users of 2.4 GHz than 5 GHz meaning that most users will be using the 2.4 GHz band. We also see that there are 50% more slow results for 2.4 GHz compared to 5 GHz.

Pie chart showing distribution of 2.4 Ghz and 5 GHz bands
Distribution of 2.4 Ghz and 5 GHz bands
Pie charts showing percentage of slow results for 2.4 and 5 Ghz
Percentage of slow results for 2.4 and 5 Ghz

Number of results per router and device

Another filter that we apply to our results is to ensure a minimum number of results per device and per router. The restriction on the device (phone / tablet) was used in identifying the devices to be used to filter the CPE (router) results. These results were then further reduced if a router had insufficient measurements from these faster devices.

CPE Manufacturer Ranking

For the top 20 manufacturers the following table ranks the best CPE from each manufacturer in order of average Wi-Fi Throughput. We also give the average Internet Download speeds as measured by our speed test. The Slow % indicates the % of results for that router that were defined as slow by the metric discussed previously. We provide the cost where available to allow for a Value For Money comparison.

 

We provide further tables that include their other top-performing equipment in the section that follows.

 


Ranking table of best models for each CPE manufacturer

Prices are given for CPEs that were available to buy at the time of writing and are in USD.

 

The Internet speed as measured by our Speed Test is provided to give an indication of the speeds expected by the consumer with most speeds between 20 and 50 Mb/s. A couple are much lower than that, particularly TP-Link Archer_C5. This should not be interpreted as a fault of the router especially in this instance where the slow % is only 2%. i.e. the router is not the cause of the slow internet speed. Most likely the slow internet speeds are linked to the actual router being popular in countries / ISPs which offer internet packages with low speeds.

Vendor-specific CPE Ranking Tables

The previous table ranks all manufacturers by their best-performing CPE (routers). In the following tables we take the top 10 manufacturers and include the overall ranking position.

 

There are no tables for Compal Broadband or Sercomm because they each have only one CPE in our results.

Arris CPE RankingArris CPE Ranking

AVM Berlin CPE RankingAVM CPE Ranking

BT CPE RankingBT CPE Ranking

LinkSys CPE RankingLinkSys CPE Ranking

Netgear CPE RankingNetgear CPE Ranking

Sagemcom CPE RankingSagemcom CPE Ranking

TP-Link CPE RankingTP-Link CPE Ranking

ZyXel CPE RankingZyXel CPE Ranking

Conclusion

The results of our study show that there are some clear differences between manufacturers and between the models that each provide. Most manufacturers provide one or two routers that are significantly better than the rest, reflecting improvement in technology. For example, BT is releasing new versions of its models and we can see clearly from our data the wi-fi throughput increases with every version and showing it is worthwhile upgrading.

 

Although more connections are being made using the 5 GHz band than previous years, 2.4 GHz connections are still used twice as often. It is interesting to see that the percentage of slow connections using 2.4 GHz (20%) is 50% larger than for 5 GHz (13%). We conclude that most customers suffering with slow connections are still using 2.4 GHz and that is why we concentrated on this band. The reasons for this include the CPE (router) not having a 5 GHz band, the user not switching to 5 GHz either because of lack of knowledge or because they get better results on 2.4 GHz (because of distance, walls etc).

 

Routers from BT, Sercomm, Zyxel and Linksys topped the ranking in terms of Wi-Fi throughput, each being 10% faster than their rivals. The Linksys EA7500 and ZyXel EMG2926 were particularly impressive because their slow % was 1% and 6% respectively. Sercomm’s top router (the Livebox) had a faster throughput but a disappointing 19% slow %. If reliable performance is important then choose the Zyxel or Linksys router but if value for money is important then choose BT Home Hub 6 or Sercomm Livebox, each costing less than half of the others. The AVM Berlin Fritz!Box 7490 at $270 does not offer the best VFM, however, their second router (7430) at $110 is a better buy.

 

Only one manufacturer in the top 10 had a slow % below the average: the Arris TG2492LG-85 at a disappointing 40%. With a throughput of 87 Mb/s it performed well but if reliability is important then choosing their second fastest (TG862G) gives a respectable 73 Mb/s but only 4 % slow %.

 

Most manufacturers have some routers that have a wide range of slow %. BT is unusual in that the 6 routers in their table range between 13% and 20% with the throughput correlating with the slow % i.e. the faster the throughput the small the slow %. Netgear in contrast has a flagship router (WNDR4500v2) with great throughput and negligible slowness but 6 others that perform poorly in contrast. Linksys have two great routers and two not so great.

 

Wy care abut Wi-Fi performance?

How can SpeedChecker help?

 

Wi-Fi Measurement data

 

 

Contact Us

For more detailed information please contact us:

email: [email protected]
skype: jezowicz
phone: +44 203 286 3573
Contact Us Here

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

March 2019 Middle East Speed Ranking Report

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

Summary

In this analysis of speed test results taken in countries in the Middle East during February 2019 we compare download speeds between countries, between MNOs & ISPs within each country and also compare how performance varies across different subsidiaries of the major Telecom Groups.

Key Insights

  • Middle East average download speed: 7.3 Mb/s

  • Turkey, Qatar, Bahrain and Jordan have above average download speeds

  • Egypt and Kuwait have the slowest download speeds

  • Vodafone in Qatar, Orange in Jordan and Vodafone Turkey are the fastest amongst MNOs & ISPs with download speeds over 10 Mb/s

  • Inconsistent CDN peering impacts user experience

Country Ranking

Regional average download speed is 7.3 Mb/s.

Turkey (10.7 Mb/s), Qatar (9.0 Mb/s), Bahrain (8.4 Mb/s) and Jordan (8.1 Mb/s) top the chart with above Middle East average download speeds. Egypt (4.4 Mb/s) and Kuwait (5.7 Mb/s) have the poorest performance in our tests. These speeds are represented in the following chart.

compare_download_do

How Telecom Groups Perform in Different Countries

There are a number of international MNOs & ISPs that offer services in more than one Middle East country:

Telecom Group Number of
Countries
Countries

Batelco

2

Bahrain and Jordan

Etisalat

3

Egypt, Saudi Arabia and United Arab Emirates

Link.Net

2

Egypt and Jordan

Ooredoo

3

Kuwait, Oman and Qatar

Orange

2

Egypt and Jordan

STC / Viva

3

Bahrain, Kuwait and Saudi Arabia

Vodafone

3

Egypt, Qatar and Turkey

Zain

4

Bahrain, Jordan, Kuwait and Saudi Arabia

Our next chart superimposes the average speed for each country with the speeds for each Telecom Group so that we can see which Groups produce above or below average in each country that they operate in. Each country is given a unique colour in the following chart to allow a visual comparison of speeds for different Telecom Groups in each country.

Most Telecom Groups outperform the average in each country with the following exceptions:

  • Batelco in Jordan

  • Link.net in Jordan and Egypt

  • Ooredoo in Qatar

  • Vodafone in Turkey

  • Zain in Saudi Arabia, Bahrain and Kuwait

provider_groups_colours_circles

Top MNOs & ISPs in each Country

In this section we look at how the most popular MNOs & ISPs perform in each country. The charts are in order of the fastest available download speeds with a consistent scale of up to 16 Mb/s to make comparison easier.

turkey16
qatar16
jordan16
bahrain16
egypt16
oman16
uae16
sa16
kuwait16

SpeedChecker Measurement Methodology

Data Collection

SpeedChecker collected data in all countries during the same period in February 2019 to ensure a fair comparison. We gathered many data points using the crowd sourced data samples collected in the field on mobile devices. During the time the data collection took place, SpeedChecker received over 60,000 data samples and the included statistics and analysis are based on this dataset.

It was important for us to not only measure the real speeds as experienced by the user but also to ensure that we measured to the same point to ensure a fair comparison. For this we included only measurements to one server in Amsterdam. Read the following section “Choice of measurement server” for more about this method.

Choice of measurement server

Every measurement methodology differs in the selection of measured server infrastructure. Some methodologies focus on on-net servers hosted in telecom premises and others focus off-net. SpeedChecker believes off-net servers’ measurements are representing real user experience better than on-net as most of the content accessible by end users lies off-net. Most of the content these days is hosted on CDN networks and the best way to measure most relevant download metrics is to choose popular CDN networks. However, as we learn in this report, not all ISPs and MNOs connect to CDNs directly (or peer) and that has major implications on performance.

One of the main advantages of CDN peering is to improve connection time and download speeds by bringing the content closer to the end user. Before we could analyse the speed results, we considered how widespread peering was and discovered that there was a variation between 0 and 100% use of local servers.

server_use_chart

The chart above shows the percentage of results from each country that are routed via local Middle East Servers, Europe or Asia. Bahrain, Qatar and Oman are all above 50% Middle East server use with Bahrain at 100%. United Arab Emirates and Egypt have a very low use of local servers. In our case we have used the most popular CDN – Cloudflare.

Speeds achieved to local servers were, on average, around 30% faster than speeds routed via European servers but this is not available to the majority of users. Because of the unbalanced availability of this CDN peering we chose to limit our analysis to results from one particular server based in Amsterdam, Europe.


  • No country will have an unfair advantage as would be the case if local servers were used

  • The majority of content is still hosted in Europe which means the download speeds reflect actual usage for most of the content downloaded by the end user

It is acknowledged that the operators in some countries that are peering with CDNs will achieve better results than those shown in this report.

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

Analysis of Fibre and 4G Deployment in Riyadh

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

This is a report on the state of Fibre and 4G deployment in Riyadh based on data points collected by Speedchecker in September 2018. The report discusses the state of Fibre and Mobile coverage (the extent of coverage and the quality of service) and the Speedchecker Measurement Method. The conclusion shows how Riyadh is placed to take advantage of future improvements to networks.

Current Network Coverage in Riyadh

Summary of Network Coverage

The three main Internet providers in Riyadh are STC, Mobily and Zain. Only STC provide services over 4G, Fibre AND Copper. Mobily offer 4G and Fibre and Zain provide 4G but not Fibre or Copper. Riyadh has excellent 4G coverage and Fibre is well established in the centre of the city and plans are well underway to extend the coverage to the main city areas. Fibre beyond the main city areas is planned but not currently in progress.

coverage_summary

Fibre Coverage

Fibre is widely available across Riyadh particularly in and close to the centre. The map shown below shows that coverage is poor in the South-West of the city and in the rural areas surrounding the city.

Fibre is provided by STC and Mobily with Mobily exclusively covering the South-West and ITC the North-East. Other existing areas are covered jointly by STC and Mobily.

The In Progress areas (yellow on the map) are either STC or ITC with some coverage being provided by Dawiyat.

Zain has no fibre coverage in Riyadh as at October 2018.

riyadh_ftth_coverage

Fibre To The Home (FTTH) Coverage in Riyadh (January 2018)
Source: MCIT (https://www.mcit.gov.sa/en/wbsira-map)

4G Coverage

Riyadh has excellent 4G coverage with 4G being available in all urban districts and along the length of the main roads going into and out of the city.

Speedchecker Measurement Method

Speedchecker uses the billions of data points collected through its passive and active measurement technologies worldwide to provide insights to our customers. This is used by businesses to improve their service and by research establishments to provide invaluable information.

Each data point consists of many KPIs including speed, latency, location, connection type, device info. Our results focus on speed and latency as experienced on the device to provide insightful information on Quality of Service. More detail about the Speedchecker Measurement Method.

This data is then integrated into our customizable map-based dashboards for geospatial analysis.

STC Fibre Coverage

Riyadh has an ongoing plan to implement fibre broadband across the city. Our results clearly show a correlation between the speeds achieved in districts that have fibre and those that do not.

We analysed the fibre results from STC to see if they correlated with the rollout of fibre across Riyadh. Our results on the left show high-speed results in Red / dark orange and slower results in yellow / light orange. These can be compared with the green areas from the MICT rollout plan where fibre is already available and the yellow areas where it is in process. The blue areas show areas that are planned but not yet in process and it is in these areas that the speeds are low.

riyadh_ftth

We are still analysing the results from Mobily fibre and will publish when the analysis is complete.

State of Riyadh Mobile Networks

Speed result data points collected from Riyadh in September 2018 were analysed and allowed the top 3 mobile providers to be compared.

By adding the download speed data to our districts map of Riyadh we can clearly see that STC provided the fastest download speeds followed by Mobily and finally Zain. The maps also show a consistent difference in speeds from district to district. Districts that are the fastest or slowest for one provider tend to be the fastest or slowest for the others even though their actual speeds may vary.

riyadh_mobile_speeds

map_scale_1

 

The following table illustrates the fastest and slowest districts in Riyadh based on the average mobile download speeds. The speeds highlighted in green represent the 5 fastest speeds by provider and the red speeds are the 5 slowest by provider. It is clear from this table and the maps above that STC are getting the fastest mobile speed test results and Zain the slowest.

 

riyadh_districts

Conclusion

Riyadh has excellent 4G coverage provided by STC, Mobily, Zain and other mobile operators. The MCIT (Ministry of Communications and Information Technology) plan for rolling out fibre across Riyadh is well-established and their progress map is accurate.

All 3 companies are providing a good service with STC having more coverage and faster speeds. Our report has highlighted some areas of Riyadh that could need some improvement in service and others that are doing very well. This may inform future plans for infrastructure changes.

This is a good foundation that should ensure Riyadh will be well-placed to continue to take advantage of improvements in technology such as 5G. This will ensure that businesses and residential users can continue to enjoy all the benefits that these advances bring.

logo-smallInterested in more detailed information on the Internet quality and coverage in Middle East and beyond?

Contact us for more information

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

Quality of Service: Lessons from Content Delivery World Conference 2015

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

content_deliveryQuality of Service: Lessons from Content Delivery World Conference 2015

CDW 2015: The evolution of content delivery architectures and workflows, and their role in distributing content throughout the globe.

In October 2015 Speedchecker hosted a stand at the Content Delivery World Conference 2015. Featuring  influential speakers from Canal+, Time Warner Cable, Wuaki.tv, Sky, Telecom Italia, BT, Telefonica, Cisco, Freeview and others.

This is a summary of the event from Steve Gledhill (Head of Content, Speedchecker Ltd) with focus on how to improve the Quality of Service for the end user.

SAMSUNG CSC
Typical activity between presentations

The Agenda

Content Delivery World 2015 brought together players from across the content delivery ecosystem, enabling the exchange of ideas and innovations and formation of partnerships that will fuel the growth of the content delivery industry, furthering the potential of this multi-billion dollar market.

SAMSUNG CSC
Standing room only

Of particular interest was how to measure the Quality of Experience for the end user rather than only the Quality of Service of the suppliers (CDNs, ISPs and Media Providers). Most of the information that we see regarding the performance of Content Delivery focus on latency, bitrate, throughput and other important factors that relate to the efficient data transmission. We were keen to learn from the key players at the conference how they translate these cold figures into a meaningful Quality of Experience. It was heartening to see that QoE was covered in most of the presentations indicating that this is seen as a key factor in improving the industry moving forward.

Measuring Quality of Experience

During a discussion about the need for multi-CDNs (redundancy, control, independence, QoE, peak traffic management) a number of Key Performance Indicators were listed:

  • Availability
  • Throughput
  • Buffer status
  • Routes taken
  • Bandwidth
  • Latency
  • User experience
  • Video impairments
  • Packet loss
  • Concurrent users
  • User viewing patterns

With regards to QoE it was agreed that “quick enough is good enough”; black screens are unacceptable; less than 2 seconds to switch channels is OK.

VTT in Finland have used user panels to discover how much (or how little) latency and buffering will cause their viewers’ QoE to become unacceptable.

Delivering Quality of Experience

Time Warner reported that they have seen a steady 20% annual increase in the growth of IP traffic year on year and they note that 78% of this is video. In order to cope with this steady increase they see a need for TCP Tuning, OS Tuning, NIC Tuning and a general reduction in the protocol overheads. They see the key metrics need to be captured passively and actively and include the monitoring of the system, analysis of logs and simulated clients.

Alcatel made the point that the CDNs need to be more content aware than they are at the present to ensure the highest quality. They also recommend that each CDN end point should be aware of existing cache to prevent unnecessary delays.

Another common discussion was around variable bit rate. For example, Sky Italia use Adaptive Encoding to ensure that users with 2.4 Mbps experience the same quality as those with 9 Mbs using near real-time encoding. This relies on a high quality original input. This bit rate is managed by the Sky CDN Selector at the Edge Server as close as possible to the consumer.

Orange presented some interesting concerns around certain protocols. Their international CDN provides average speeds of 4Mbps but they feel they have a number of issues to contend with. First, they have issues with caching (many presenters referred to caching as being a key area for improvement). Second, they highlighted the HTTPS issues for carriers under HTTP 2.0. Third, the need to be flexible and responsive to changes makes it hard to provide consistently high quality. Finally, they made reference to the Microsoft Smooth Streaming minimum latency requirement being too high to provide live content as they would wish to.

Multicasting

Looking to the future and how to deal with increased demand and higher quality video BT TV talked about Multicasting. They acknowledge that although they can distribute at speeds up to 12GBs at production that they lose control the closer they get to the customer. They have no control over the Home User’s devices or network; their equipment; the wiring in the home or the core/backhaul network all of which can lead to packet loss. Multitasking reduces the number of individual streams required and thus reduces delays and congestion. They can use the application layer to control dropped packets / retransmission and can even use 2 identical streams via different routes to be combined at the receiver in the home thus providing built-in redundancy. Problems are identified by end to end monitoring of the network data and user behaviour. The break-even point for multicasting compared to unicasting is 500 users or more. Quality of Experience is improved in terms of immediacy, quality and
continuity.

Problems with multicasting: Requires Unicast Tunnelling across Gaps where Multicast is not possible; Speeds can fall to slowest bit rate; Unicasting is recommended in the home/office.

Delivering on Mobile

Aventeq predict that by 2020 the average smartphone contract will allow 5Gb of data to be downloaded each month. This confirms that we need to ensure that the user experience for the mobile user is given high priority when considering QoE.

This emphasis was highlighted by EE when they showcased their 4G video offering showing seamless streaming from dense urban areas, to high speed (legal) motorway driving and in to the countryside and forests of England. They say this is possible because of the UK having an average speed over LTE of 20Mbps – the fastest in the World.

EE also took a different approach to improving QoE by not just dealing with latency and speed but with the actual content. They propose giving audiences to live events the option to see a choice of angles and bespoke statistics. This is available in the home but they propose making this available on mobile devices.

CDNs – Content Delivery Networks

CDNs are the backbone of the delivery mechanism and were a common point of discussion and debate throughout the conference

A number of Media providers are using or developing their own CDN (Content Delivery Network) to ensure that they are in control of the users’ QoE. Canal+ in France have 20-25 Free To Air channels and other services that will be transmitted via their own CDN in the next 12 months. Most of their traffic is driven by Live (premium) content and it is important for Canal+ that they maintain a high QoS for this content. They report that their users complain about streaming that doesn’t launch; video quality issues and buffering in roughly equal measure. Their research shows that download speeds of between 2 and 2.5Mbps are acceptable for the QoE of the end user. These speeds are achieved at all times except for the peak times of 7pm and later when problems start. Canal+ plan to provide their content directly to the ISPs instead of using a CDN as is currently the case. This should save money but they also hope that it will improve users’ QoE.

This lead to recommendations for choosing CDNs:

  • Point of Contact if there’s a problem
  • Excellent throughput and latency performance
  • Live content handling
  • Cloud computing support

Net Neutrality, Copyright and DRM

Most of the discussions and presentations dealt with moving the data around as efficiently as possible with no concern with regard to Net Neutrality. That’s not to say that this is ignored but rather, I suggest, that it is acknowledged that best technical practice needs to be modified to comply with regulations in any and all countries. For that matter, DRM and other copyright issues were only touched upon in a few presentations for similar reasons: the focus of the conference was on efficiency and how the technology can be used and improved upon.

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail