Here's a list of MTProto transport protocols (see the ISO/OSI recap for a full explanation):
The server recognizes these different protocols (and distinguishes them from HTTP, too) by the header. Additionally, the following transport features can be used:
Example implementations for these protocols can be seen in tdlib and MadelineProto.
The lightest protocol available.
Payload structure:
+-+----...----+
|l| payload |
+-+----...----+
OR
+-+---+----...----+
|h|len| payload +
+-+---+----...----+Before sending anything into the underlying socket (see transports), the client must first send 0xef as the first byte (the server will not send 0xef as the first byte in the first reply).
Then, payloads are wrapped in the following envelope:
0x01..0x7e.If the packet length divided by four is bigger than or equal to 127 (>= 0x7f), the following envelope must be used, instead:
0x7fIn case 4-byte data alignment is needed, an intermediate version of the original protocol may be used.
Payload structure:
+----+----...----+
+len.+ payload +
+----+----...----+Before sending anything into the underlying socket (see transports), the client must first send 0xeeeeeeee as the first int (four bytes, the server will not send 0xeeeeeeee as the first int in the first reply).
Then, payloads are wrapped in the following envelope:
Padded version of the intermediate protocol, to use with obfuscation enabled to bypass ISP blocks.
Before sending anything into the underlying socket (see transports), the client must first send 0xdddddddd as the first int (four bytes, the server will not send 0xdddddddd as the first int in the first reply).
Then, payloads are wrapped in the following envelope:
+----+----...----+----...----+
|tlen| payload | padding |
+----+----...----+----...----+Envelope description:
0-15The basic MTProto transport protocol
Payload structure:
+----+----+----...----+----+
|len.|seq.| payload |crc.|
+----+----+----...----+----+Envelope description:
Additionally, the following transport features can be used:
These MTProto transport protocols have support for quick acknowledgment. In this case, the client sets the highest-order length bit in the query packet, and the server responds with a special 4 bytes as a separate packet. They are the 32 higher-order bits of SHA256 of the encrypted portion of the packet prepended by 32 bytes from the authorization key (the same hash as computed for verifying the message key), with the most significant bit set to make clear that this is not the length of a regular server response packet; if the abridged version is used, bswap is applied to these four bytes.
In the event of a transport error (missing auth key, transport flood, etc.), the server may send a packet with a signed little-endian number of 4 bytes, whose absolute value contains the error code (the error itself is actually negative).
For example, error Code 403 corresponds to situations where the corresponding HTTP error would have been returned by the HTTP protocol.
Error 404 (auth key not found) is returned when the specified auth key ID cannot be found by the DC.
Error 429 (transport flood) is returned when too many transport connections are established to the same IP in a too short lapse of time, or if any of the container/service message limits are reached.
Error 444 (invalid DC) is returned while creating an auth key, connecting to an MTProxy or in other contexts if an invalid DC ID is specified.
Transport obfuscation is required to use the websocket transports.
Transport obfuscation to prevent ISP blocks is implemented using the following protocol, situated under the MTProto transport in the ISO/OSI stack, see the recap; this means that the payload is first wrapped in the MTProto transport envelope (all transports are supported), and then obfuscated:
Prior to establishing the connection (and eventually sending the protocol header of a specific MTProto transport), a 64-byte (512-bit) random initialization payload is generated.
During the generation process, special care must be taken in order to avoid a situation where that the first int (first four bytes) of the random string are equal to one of the known protocol identifiers (see above).
In particular, the first four bytes must not be equal to 0xdddddddd (padded intermediate), 0xeeeeeeee (intermediate), POST, GET, HEAD, or any of the HTTP methods that are accepted by the MTProto servers.
The first byte must also not be equal to 0xef (abridged).
Bytes 4-8 must also not be equal to 0x00000000, since that would indicate use of the full transport with the initial TCP sequence number (0).
The protocol identifier, if present, must be inserted in the initialization payload at byte offset 56: if its length is less than 4, it must be padded using the protocol identifier itself, to make its length 4 (0xef => 0xefefefef): the standalone protocol identifier must be not be sent aftwerwards.
This protocol is also (but not exclusively) used when connecting to MTProxies: only in this case, the DC ID in a specially encoded form must also be inserted in the initialization payload at offset 60.
The encoding simply consists of the DC ID in two-byte signed little-endian form; 10000 has to be added to the DC ID to connect to the test servers; it has to be made negative if the DC we're connecting to is a media (not CDN) DC.
Next, a secondary initialization payload is generated by reversing the primary initialization payload.
Two keys are extracted from both initialization payloads, using bytes at offsets 8-40: the key extracted from the primary payload is used as encryption key, the key extracted from the secondary payload is used as decryption key.
Two IVs are extracted from both initialization payloads, using bytes at offsets 40-56: the IV extracted from the primary payload is used as encryption IV, the IV extracted from the secondary payload is used as decryption IV.
Only if using MTProxy, the secret is used to provide connection with the MTProxy server. The secret is a 16-byte string, usually distributed in its hexadecimal form along with the MTProxy host and port in proxy deep links ».
Often, a 17-byte version of the secret can be found: this simply indicates that the client should use a specific MTProto transport (based on the first byte, usually it's 0xdd, to indicate that the padded intermediate protocol should be used 0xdddddddd; however, clients should default to the padded intermediate transport whenever an additional byte in the secret is encountered).
The extracted encryption and decryption keys must be concatenated with the secret (the first byte of which should be ignored if it's the 17-byte version), and the SHA256 hash of such string should be used as encryption/decryption key.
The obtained encryption and decryption key/IV pairs must then be used with AES-256-CTR to encrypt and decrypt all outgoing and incoming payloads.
The final value of the encryption/decryption counter after handling an MTProto payload must be used as IV for the next payload, until the TCP/WS connection is closed.
In other words, reuse the two encryption and decryption AES-256-CTR OpenSSL instances until the connection is closed.
The first thing that must be encrypted using the encryption key is the initialization payload itself.
Then bytes 56-64 of the encrypted initialization payload are substituted in the original initialization payload: this is the part that contains the constant MTProto transport protocol identifier and the DC ID (only for MTProxies).
The final initialization payload must then be sent in the socket as first 64 bytes after the TCP handshake.
Example pseudocode for the generation of an MTProxy connection payload (media DC 4) using the obfuscated padded intermediate transport. Warning: do not use the specified proxy secret in any MTProxy exposed on the internet.
protocol := 0xdddddddd
dc := 0xfcff
while 1:
init := (56 random bytes) + protocol + dc + (2 random bytes)
if init[0] == 0xef:
continue
first_int := substr(init, 0, 4)
if first_int == 0x44414548 || first_int == 0x54534f50 || first_int == 0x20544547 || first_int == 0x4954504f || first_int == 0x02010316 || first_int == 0xdddddddd || first_int == 0xeeeeeeee:
continue
second_int := substr(init, 4, 4)
if second_int == 0x00000000:
continue
break
initRev := strrev(init)
encryptKey := substr(init, 8, 32)
encryptIV := substr(init, 40, 16)
decryptKey := substr(initRev, 8, 32)
decryptIV := substr(initRev, 40, 16)
secret := substr(0xdd99999999999999999999999999999999, 1, 16)
encryptKey = SHA256(encryptKey + secret)
decryptKey = SHA256(decryptKey + secret)
encryptedInit := CTR(encryptKey, encryptIV, init)
finalInit := substr(init, 0, 56) + substr(encryptedInit, 56, 8)
write(finalInit)