Methodology

System Components

Test Environment for Mobile Apps and Java Test:

Test Environment for WebSocket Test

Global Constants

The following constants are used throughout the specification:

n  number of parallel TCP connections (1 ≤ n ≤ 10); default: n = 3

s   initial size of data block sent during the test; default: s = 4.096 Bytes

d  duration of pre-test; default: d = 2 s

t   duration of test; default: t = 7 s

p  number of “pings” during latency test (i.e. measurement of time elapsed between sending a data packet and receiving the reception response); default: p = 10

c   number of received chunks after which the Test Server sends statistical data to the client when processing the Upload Test;  default: c = 10

Test Procedure 

This section is not intended as specification but as overview.

The test follows the procedure outlined below. The test consists of seven phases, which are carried out one after each other, i.e., phase m starts after phase m – 1 has finished. That means that the phases do not overlap.

Phase 1: Initialization

The Test Client tries to connect to the Control Server on the TCP port 443 using TLS. In order to pass through certain firewalls - which might block unencrypted data transmissions - this might be necessary. The data streams themselves are optionally unencrypted. In order for the computational cost of encryption not to deteriorate the measurement results, encryption and authentication parameters should be chosen as low as possible, e.g., TLSv1 (SSL_RSA_WITH_RC4_128_MD5).

The server authentication key has a short length in order to reduce delays due to the TLS handshake. TLS-security is not an objective.

After establishing a proper connection, client and server exchange the information, which is required for running the test.

The client sends its version number, geo-location data and IP address as well as other relevant information to the Control Server.

The Control Server determines the Test Server to be used. It then generates a token consisting of the following components:

• a unique test ID (UUID)
• the time at which the measurement is allowed to start (a string representing Unix time)
• the period of time during which access to the Test Server is allowed (a string representing number of seconds)
• the maximum number of permitted parallel connections from the client to the server (a string representing a positive integer)
• the Base64-encoded HMAC-SHA-1 value [1] of the string composed of the components mentioned above, where the components are separated by space characters. The HMAC value is computed using a key, which should only be known to the Test Servers

The Control Server then transmits the token as well as all the additional test parameters (IP address of the Test  Server, public IP address of the client, number of parallel TCP connections, … ) to the client. The token is used for identifying the test session.

Phase 2: Downlink pre-Test

In Phase 2, the client opens n TCP connections to the assigned Test Server. Within each TCP connection, the client requests and the server sends a data block of size s (randomly generated data with high entropy). 

While the duration of the pre-test has not exceeded d, the client requests a data block of double size compared to the last iteration step. The transfer of the last data block will be finished even if the duration has already exceeded d. 

At the end of the pre-test, all TCP connections are left open for further use if more than four chunks have been transmitted in the downlink pre-test. Otherwise, n is reduced to 1.

Phase 3: Latency Test

During this phase, the client sends p “pings” in short intervals to the Test Server to test the latency of the connection. One “ping” consists of the transmission of short strings via one of the TCP connections to the Server, which returns short strings as acknowledgement. 

The client measures the time between sending and receiving the return message, while the server additionally measures the time between sending its return message and the client’s reception response. 

The client stores all measurements, and the median of all measurements is used as result since the test should provide an approximated lower bound as latency.

Phase 4: Downlink Test

Within each of the n TCP connections opened during phase 2, the client simultaneously requests and the server continuously sends data streams consisting of fixed-size chunks of size s (randomly generated data with high entropy).

All transmissions start at the same time, which is denoted as relative time (t) 0. For each TCP connection k, 1≤k≤n, the client records the relative time t_k^(j)  and the amount s_k^(j) of data received from time 0 to t_k^(j)  for successive values of j, j≥1 (usually after receiving each chunk). After time t, each TCP connection is reset immediately after the next transmitted chunk has been received. For each k, 1≤k≤n, let m_k be the number of pairs (s_k^(j), t_k^(j)) which have been recorded for TCP connection k. Then

Let  Then the amount s_k of data received over TCP connection k from time 0 to time t* is approximately

and the data rate R for all TCP connections together is

This value is taken as an approximation of the download capacity.

Phase 5: Uplink pre-Test

In Phase 5, the client opens n TCP connections to the assigned Test Server again. Within each TCP connection, the client sends a data block of size s(randomly  generated data with high entropy). 

While the duration of the pre-test has not exceeded d, the client sends a data block of double size compared to the last iteration step. The transfer of the last data block will be finished even if the duration has already exceeded d. At the end of the pre-Test, the TCP connections are left open for further use.

Phase 6: Uplink Test

Within each of the n TCP connections opened during phase 5, the client continuously sends data streams consisting of fixed-size chunks of size s (randomly generated data with high entropy). All transmissions start at the same time, which is denoted as relative time 0. 

For each TCP connection k, 1≤k≤n, the server gives feedback to the client by sending the relative time t_k^(j) and the amount s_k^(j) of data received from time 0 to t_k^(j) for successive values j, j≥1,