The test delivers an accurate measurement of the maximum bandwidth available over a given Internet connection. This is achieved by transferring multiple parallel data streams over separate TCP connections within a predefined amount of time. The transferred data consist of randomly generated data with high entropy. It is not expected that the (pseudo) random number generator meets cryptographic requirements. However, it must effectively prohibit data compression during the transmission.
In order to increase the probability that the test can be performed even within networks protected by firewalls and proxy servers, the data is transferred over a TLS connection (possibly using the TLS Record Protocol without encryption).
Test Environment for Mobile Apps and Java Test:
Test Environment for WebSocket Test
By default, data is transferred between client and server over the TCP port 443 in order to avoid interference with firewalls as much as possible. The ports for communication and data transfers between the different servers themselves are configurable.
In general, the format of the transferred data is JSON. Only the Test Server uses a different protocol to keep the communication simple and therefore the workload of the server at a minimum.
The following constants are used throughout the specification:
n number of parallel TCP connections (1 ≤ n ≤ 10); default: n = 3
s initial size of data block sent during the test; default: s = 4.096 Bytes
d duration of pre-test; default: d = 2 s
t duration of test; default: t = 7 s
p number of “pings” during latency test (i.e. measurement of time elapsed between sending a data packet and receiving the reception response); default: p = 10
c number of received chunks after which the Test Server sends statistical data to the client when processing the Upload Test; default: c = 10
This section is not intended as specification but as overview.
The test follows the procedure outlined below. The test consists of seven phases, which are carried out one after each other, i.e., phase m starts after phase m – 1 has finished. That means that the phases do not overlap.
Phase 1: Initialization
The Test Client tries to connect to the Control Server on the TCP port 443 using TLS. In order to pass through certain firewalls – which might block unencrypted data transmissions – this might be necessary. The data streams themselves are optionally unencrypted. In order for the computational cost of encryption not to deteriorate the measurement results, encryption and authentication parameters should be chosen as low as possible, e.g., TLSv1 (SSL_RSA_WITH_RC4_128_MD5).
The server authentication key has a short length in order to reduce delays due to the TLS handshake. TLS-security is not an objective.
After establishing a proper connection, client and server exchange the information, which is required for running the test.
The client sends its version number, geo-location data and IP address as well as other relevant information to the Control Server.
The Control Server determines the Test Server to be used. It then generates a token consisting of the following components:
- a unique test ID (UUID)
- the time at which the measurement is allowed to start (a string representing Unix time)
- the period of time during which access to the Test Server is allowed (a string representing number of seconds)
- the maximum number of permitted parallel connections from the client to the server (a string representing a positive integer)
- the Base64-encoded HMAC-SHA-1 value  of the string composed of the components mentioned above, where the components are separated by space characters. The HMAC value is computed using a key, which should only be known to the Test Servers
The Control Server then transmits the token as well as all the additional test parameters (IP address of the Test Server, public IP address of the client, number of parallel TCP connections, … ) to the client. The token is used for identifying the test session.
Phase 2: Downlink pre-Test
In Phase 2, the client opens n TCP connections to the assigned Test Server. Within each TCP connection, the client requests and the server sends a data block of size s (randomly generated data with high entropy).
While the duration of the pre-test has not exceeded d, the client requests a data block of double size compared to the last iteration step. The transfer of the last data block will be finished even if the duration has already exceeded d.
At the end of the pre-test, all TCP connections are left open for further use if more than four chunks have been transmitted in the downlink pre-test. Otherwise, n is reduced to 1.
Phase 3: Latency Test
During this phase, the client sends p “pings” in short intervals to the Test Server to test the latency of the connection. One “ping” consists of the transmission of short strings via one of the TCP connections to the Server, which returns short strings as acknowledgement.
The client measures the time between sending and receiving the return message, while the server additionally measures the time between sending its return message and the client’s reception response.
The client stores all measurements, and the median of all measurements is used as result since the test should provide an approximated lower bound as latency.
Phase 4: Downlink Test
Within each of the n TCP connections opened during phase 2, the client simultaneously requests and the server continuously sends data streams consisting of fixed-size chunks of size s (randomly generated data with high entropy).
All transmissions start at the same time, which is denoted as relative time (t) 0. For each TCP connection k, 1≤k≤n, the client records the relative time tk(j) and the amount sk(j) of data received from time 0 to tk(j) for successive values of j, j≥1 (usually after receiving each chunk). After time t, each TCP connection is reset immediately after the next transmitted chunk has been received. For each k, 1≤k≤n, let mk be the number of pairs (sk(j), tk(j)) which have been recorded for TCP connection k. Then
Let Then the amount sk of data received over TCP connection k from time 0 to time t* is approximately
and the data rate R for all TCP connections together is
This value is taken as an approximation of the download capacity.
Phase 5: Uplink pre-Test
In Phase 5, the client opens n TCP connections to the assigned Test Server again. Within each TCP connection, the client sends a data block of size s(randomly generated data with high entropy).
While the duration of the pre-test has not exceeded d, the client sends a data block of double size compared to the last iteration step. The transfer of the last data block will be finished even if the duration has already exceeded d. At the end of the pre-Test, the TCP connections are left open for further use.
Phase 6: Uplink Test
Within each of the n TCP connections opened during phase 5, the client continuously sends data streams consisting of fixed-size chunks of size s (randomly generated data with high entropy). All transmissions start at the same time, which is denoted as relative time 0.
For each TCP connection k, 1≤k≤n, the server gives feedback to the client by sending the relative time tk(j)and the amount sk(j) of data received from time 0 to tk(j)for successive values j, j≥1, (a) after receiving each chunk whose transmission has taken more than 1 ms or
(b) 1 ms after the previous feedback otherwise.
After time t, it is tested at intervals of .25 s whether all feedbacks for chunks sent from the client to the server until time t – 1 s have been received. As soon as this is the case, all TCP connections are reset. Otherwise, the test is aborted after time t + 3 s.
For each k, 1≤k≤n, let mk be the number of pairs (sk(j), tk(j)), which have been recorded for TCP connection k. Then
Let Then the amount sk of data received over TCP connection k from time t=0 to time t* is approximately
and the data rate R for all TCP connections together is
This value is taken as an approximation of the upload capacity.
Phase 7: Finalization
After finishing all tests, the client sends the collected data to the Control Server. All results and additional information on the client are transferred directly to the Control Server.
Both datasets are then compared by the Control Server to check the quality and integrity of the result. The Data Server stores all tests, successful or unsuccessful.
Network quality test (Network neutrality test or QoS test)
Based on the recommendation of the BEREC the network quality test measures some parameters which have a decisive influence on the quality of the internet access. The network quality test will be carried out directly after the upload test and consists of 6 test groups. Individual tests of the test groups and the individual test groups themselves may be performed in parallel or behind one another.
The following test groups are implemented in the network quality test:
(a) Website download & rendering
The device will download one or more defined web pages and build these web pages in a background browser. The process monitors the size of the website in kilobytes, the HTTP response code from the server, the header data of the response and the times of the download and the rendering of the page. The NetTest enables configuration of the URL of the web page and an optional timeout.
(b) HTTP-proxy (transparent proxy)
The network neutrality test server provides download data and maintains a hash value. The client downloads the data and also calculates the hash value. The control server compares the two hashes. When they are different the data has been modified by a proxy during the download. The NetTest enables configuration of the URL of the download data and an optional timeout.
(c) Non-transparent proxy
The client sends a random incorrect, falsified or incomplete HTTP request (e.g. „GET /“ or „GET / HTTR 8.7“) to the network quality test server. This server behaves like an echo service and responds with the same incorrect HTTP header. The control server compares the transmitted and received header data. If they are not identical then a non-transparent proxy has changed the faulty HTTP request. Normally such proxies do complete the faulty request or remove an incorrect request completely. The NetTest enables configuration of the URL to which the request is sent, the port from which the request is sent and the distorted or incomplete request which may be used.
The client sends a DNS query with optional specified resolver (e.g. 18.104.22.168). In the evaluation the response from the resolver that was automatically assigned to the client by the ISP will be compared with the known and correct results or with results of other resolvers. In addition it is checked if the operator allows third-party resolver. The NetTest enables configuration of the test domain and an optional particular resolver.
(e) TCP and UDP
The client sends TCP or UDP packets to the specified port and waits for a reply. The network quality test server receives these packets and responds to them. The control server compares the transmitted and received packets. It measures whether (TCP-/UDP-) packets have arrived or were blocked, how many packets (UDP) have arrived or how many packets had to be resent (UDP). Also the variance of the package (UDP) can be calculated. The NetTest enables configuration of the port and the number of packets to be sent.
The client sends n UDP packets over defined VoIP-ports using a defined time interval. The server receives these packets, measures the time delays between them and calculates any divergence from the default delay. The NetTest enables configuration of port numbers, amount of packets and default delay.
The client sends UDP packets with defined amount of allowed hops to the server and receives back the answer of the last hopping interface to calculate the maximum amount of hops to the server. The NetTest enables configuration of timeout, target URL and maximum amount of allowed hops.