diff --git a/data/corefork.telegram.org/api/end-to-end.html b/data/corefork.telegram.org/api/end-to-end.html index 27fd5216fd..e57180859e 100644 --- a/data/corefork.telegram.org/api/end-to-end.html +++ b/data/corefork.telegram.org/api/end-to-end.html @@ -60,7 +60,7 @@ MTProto v.1.0 is deprecated and is currently being phased out.

-

Secret Chats are one-on-one chats wherein messages are encrypted with a key held only by the chat’s participants. Note that the schema for these end-to-end encrypted Secret Chats is different from what is used for cloud chats:

+

Secret Chats are one-on-one chats wherein messages are encrypted with a key held only by the chat's participants. Note that the schema for these end-to-end encrypted Secret Chats is different from what is used for cloud chats:

@@ -82,8 +82,8 @@ The principal differences from version 1.0 (describ

Executing this method before each new key generation procedure is of vital importance. It makes sense to cache the values of the parameters together with the version in order to avoid having to receive all of the values every time. If the version stored on the client is still up-to-date, the server will return the constructor messages.dhConfigNotModified.

Client is expected to check whether p is a safe 2048-bit prime (meaning that both p and (p-1)/2 are prime, and that 2^2047 < p < 2^2048), and that g generates a cyclic subgroup of prime order (p-1)/2, i.e. is a quadratic residue mod p. Since g is always equal to 2, 3, 4, 5, 6 or 7, this is easily done using quadratic reciprocity law, yielding a simple condition on p mod 4g -- namely, p mod 8 = 7 for g = 2; p mod 3 = 2 for g = 3; no extra condition for g = 4; p mod 5 = 1 or 4 for g = 5; p mod 24 = 19 or 23 for g = 6; and p mod 7 = 3, 5 or 6 for g = 7. After g and p have been checked by the client, it makes sense to cache the result, so as to avoid repeating lengthy computations in future. This cache might be shared with one used for Authorization Key generation.

If the client needs additional entropy for the random number generator, it can pass the random_length parameter (random_length> 0) so the server generates its own random sequence random of the appropriate length. -Important: using the server’s random sequence in its raw form may be unsafe, it must be combined with a client sequence.

-

Client A computes a 2048-bit number a (using sufficient entropy or the server’s random; see above) and executes messages.requestEncryption after passing in g_a := pow(g, a) mod dh_prime.

+Important: using the server's random sequence in its raw form may be unsafe, it must be combined with a client sequence.

+

Client A computes a 2048-bit number a (using sufficient entropy or the server's random; see above) and executes messages.requestEncryption after passing in g_a := pow(g, a) mod dh_prime.

User B receives the update updateEncryption for all associated authorization keys (all authorized devices) with the chat constructor encryptedChatRequested. The user must be shown basic information about User A and must be prompted to accept or reject the request.

Both clients are to check that g, g_a and g_b are greater than one and smaller than p-1. We recommend checking that g_a and g_b are between 2^{2048-64} and p - 2^{2048-64} as well.

Accepting a Request

@@ -92,7 +92,7 @@ The principal differences from version 1.0 (describ

Note 1: in this particular case SHA1 is used here even for MTProto 2.0 secret chats.

Note 2: this fingerprint is used as a sanity check for the key exchange procedure to detect bugs when developing client software — it is not connected to the key visualization used on the clients as means of external authentication in secret chats. Key visualizations on the clients are generated using the first 128 bits of SHA1(initial key) followed by the first 160 bits of SHA256(key used when secret chat was updated to layer 46).

Client B executes messages.acceptEncryption after passing it g_b := pow(g, b) mod dh_prime and key_fingerprint.

-

For all of Client B’s authorized devices, except the current one, updateEncryption updates are sent with the constructor encryptedChatDiscarded. Thereafter, the only device that will be able to access the secret chat is Device B, which made the call to messages.acceptEncryption.

+

For all of Client B's authorized devices, except the current one, updateEncryption updates are sent with the constructor encryptedChatDiscarded. Thereafter, the only device that will be able to access the secret chat is Device B, which made the call to messages.acceptEncryption.

User A will be sent an updateEncryption update with the constructor encryptedChat, for the authorization key that initiated the chat.

With g_b from the update, Client A can also compute the shared key key = (pow(g_b, a) mod dh_prime). If key length < 256 bytes, add several leading zero bytes as padding — so that the key is exactly 256 bytes long. If the fingerprint for the received key is identical to the one that was passed to encryptedChat, incoming messages can be sent and processed. Otherwise, messages.discardEncryption must be executed and the user notified.

Perfect Forward Secrecy

@@ -125,7 +125,7 @@ The principal differences from version 1.0 (describ

Encrypted data is embedded into a messages.sendEncrypted API call and passed to Telegram server for delivery to the other party of the Secret Chat.

Upgrading to MTProto 2.0 from MTProto 1.0

As soon as both parties in a secret chat are using at least Layer 73, they should only use MTProto 2.0 for all outgoing messages. Some of the first received messages may use MTProto 1.0, if a sufficiently high starting layer has not been negotiated during the creation of the secret chat. After the first message encrypted with MTProto 2.0 (or the first message with Layer 73 or higher) is received, all messages with higher sequence numbers must be encrypted with MTProto 2.0 as well.

-

As long as the current layer is lower than 73, each party should try to decrypt received messages with MTProto 1.0, and if this is not successfull (msg_key does not match), try MTProto 2.0. Once the first MTProto 2.0-encrypted message arrives (or the layer is upgraded to 73), there is no need to try MTProto 1.0 decryption for any of the further messages (unless the client is still waiting for some gaps to be closed).

+

As long as the current layer is lower than 73, each party should try to decrypt received messages with MTProto 1.0, and if this is not successful (msg_key does not match), try MTProto 2.0. Once the first MTProto 2.0-encrypted message arrives (or the layer is upgraded to 73), there is no need to try MTProto 1.0 decryption for any of the further messages (unless the client is still waiting for some gaps to be closed).

Decrypting an Incoming Message

The steps above are performed in reverse order. When an encrypted message is received, you must check that msg_key is in fact equal to the 128 middle bits of the SHA256 hash of the decrypted message, prepended by 32 bytes taken from the shared key. @@ -137,7 +137,7 @@ If the message layer is greater than the one supported by the client, the user m

Please note that your client must support sequence numbers in Secret Chats to be compatible with official Telegram clients.

Sending Encrypted Files

-

All files sent to secret chats are encrypted with one-time keys that are in no way related to the chat’s shared key. Before an encrypted file is sent, it is assumed that the encrypted file’s address will be attached to the outside of an encrypted message using the file parameter of the messages.sendEncryptedFile method and that the key for direct decryption will be sent in the body of the message (the key parameter in the constructors decryptedMessageMediaPhoto, decryptedMessageMediaVideo and decryptedMessageMediaFile.

+

All files sent to secret chats are encrypted with one-time keys that are in no way related to the chat's shared key. Before an encrypted file is sent, it is assumed that the encrypted file's address will be attached to the outside of an encrypted message using the file parameter of the messages.sendEncryptedFile method and that the key for direct decryption will be sent in the body of the message (the key parameter in the constructors decryptedMessageMediaPhoto, decryptedMessageMediaVideo and decryptedMessageMediaFile.

Prior to a file being sent to a secret chat, 2 random 256-bit numbers are computed which will serve as the AES key and initialization vector used to encrypt the file. AES-256 encryption with infinite garble extension (IGE) is used in like manner.

The key fingerprint is computed as follows:

-

Secret Chats are one-on-one chats wherein messages are encrypted with a key held only by the chat’s participants. Please note that the schema for end-to-end encrypted Secret Chats is different from what is used for cloud chats:

+

Secret Chats are one-on-one chats wherein messages are encrypted with a key held only by the chat's participants. Please note that the schema for end-to-end encrypted Secret Chats is different from what is used for cloud chats:

@@ -69,8 +69,8 @@ For information on end-to-end encryption used in up-to-date Telegram clients, ki

Executing this method before each new key generation procedure is of vital importance. It makes sense to cache the values of the parameters together with the version in order to avoid having to receive all of the values every time. If the version stored on the client is still up-to-date, the server will return the constructor messages.dhConfigNotModified.

Client is expected to check whether p is a safe 2048-bit prime (meaning that both p and (p-1)/2 are prime, and that 2^2047 < p < 2^2048), and that g generates a cyclic subgroup of prime order (p-1)/2, i.e. is a quadratic residue mod p. Since g is always equal to 2, 3, 4, 5, 6 or 7, this is easily done using quadratic reciprocity law, yielding a simple condition on p mod 4g -- namely, p mod 8 = 7 for g = 2; p mod 3 = 2 for g = 3; no extra condition for g = 4; p mod 5 = 1 or 4 for g = 5; p mod 24 = 19 or 23 for g = 6; and p mod 7 = 3, 5 or 6 for g = 7. After g and p have been checked by the client, it makes sense to cache the result, so as to avoid repeating lengthy computations in future. This cache might be shared with one used for Authorization Key generation.

If the client has an inadequate random number generator, it makes sense to pass the random_length parameter (random_length> 0) so the server generates its own random sequence random of the appropriate length. -Important: using the server’s random sequence in its raw form may be unsafe. It must be combined with a client sequence, for example, by generating a client random number of the same length (client_random) and using final_random := random XOR client_random.

-

Client A computes a 2048-bit number a (using sufficient entropy or the server’s random; see above) and executes messages.requestEncryption after passing in g_a := pow(g, a) mod dh_prime.

+Important: using the server's random sequence in its raw form may be unsafe. It must be combined with a client sequence, for example, by generating a client random number of the same length (client_random) and using final_random := random XOR client_random.

+

Client A computes a 2048-bit number a (using sufficient entropy or the server's random; see above) and executes messages.requestEncryption after passing in g_a := pow(g, a) mod dh_prime.

User B receives the update updateEncryption for all associated authorization keys (all authorized devices) with the chat constructor encryptedChatRequested. The user must be shown basic information about User A and must be prompted to accept or reject the request.

Both clients are to check that g, g_a and g_b are greater than one and smaller than p-1. We recommend checking that g_a and g_b are between 2^{2048-64} and p - 2^{2048-64} as well.

Accepting a Request

@@ -78,7 +78,7 @@ For information on end-to-end encryption used in up-to-date Telegram clients, ki

Having received g_a from the update with encryptedChatRequested, it can immediately generate the final shared key: key = (pow(g_a, b) mod dh_prime). If key length < 256 bytes, add several leading zero bytes as padding — so that the key is exactly 256 bytes long. Its fingerprint, key_fingerprint, is equal to the 64 last bits of SHA1 (key).

Note: this fingerprint is used as a sanity check for the key exchange procedure to detect bugs while developing client software — it is not connected to the key visualization used on the clients as means of external authentication in secret chats. Key visualizations on the clients are generated using the first 128 bits of SHA1(initial key) followed by the first 160 bits of SHA256(key used when secret chat was updated to layer 46).

Client B executes messages.acceptEncryption after passing it g_b := pow(g, b) mod dh_prime and key_fingerprint.

-

For all of Client B’s authorized devices, except the current one, updateEncryption updates are sent with the constructor encryptedChatDiscarded. Thereafter, the only device that will be able to access the secret chat is Device B, which made the call to messages.acceptEncryption.

+

For all of Client B's authorized devices, except the current one, updateEncryption updates are sent with the constructor encryptedChatDiscarded. Thereafter, the only device that will be able to access the secret chat is Device B, which made the call to messages.acceptEncryption.

User A will be sent an updateEncryption update with the constructor encryptedChat, for the authorization key that initiated the chat.

With g_b from the update, Client A can also receive the shared key key = (pow(g_b, a) mod dh_prime). If key length < 256 bytes, add several leading zero bytes as padding — so that the key is exactly 256 bytes long. If the fingerprint for the received key is identical to the one that was passed to encryptedChat, incoming messages can be sent and processed. Otherwise, messages.discardEncryption must be executed and the user notified.

Perfect Forward Secrecy

@@ -117,7 +117,7 @@ If the message layer is greater than the one supported by the client, the user m

Please note that your client must support sequence numbers in Secret Chats to be compatible with official Telegram clients.

Sending Encrypted Files

-

All files sent to secret chats are encrypted with one-time keys that are in no way related to the chat’s shared key. Before an encrypted file is sent, it is assumed that the encrypted file’s address will be attached to the outside of an encrypted message using the file parameter of the messages.sendEncryptedFile method and that the key for direct decryption will be sent in the body of the message (the key parameter in the constructors decryptedMessageMediaPhoto, decryptedMessageMediaVideo and decryptedMessageMediaFile.

+

All files sent to secret chats are encrypted with one-time keys that are in no way related to the chat's shared key. Before an encrypted file is sent, it is assumed that the encrypted file's address will be attached to the outside of an encrypted message using the file parameter of the messages.sendEncryptedFile method and that the key for direct decryption will be sent in the body of the message (the key parameter in the constructors decryptedMessageMediaPhoto, decryptedMessageMediaVideo and decryptedMessageMediaFile.

Prior to a file being sent to a secret chat, 2 random 256-bit numbers are computed which will serve as the AES key and initialization vector used to encrypt the file. AES-256 encryption with infinite garble extension (IGE) is used in like manner.

The key fingerprint is computed as follows:

If we use [T] to denote the concrete type corresponding to the abstract T, and [E] to denote an element of [T] corresponding to the value E of type T, then the last rule may be written as:

Values of the built-in clothed types Int and String and serialized as if they were defined using int x:int = Int; and string s:string = String;, i.e. the serialization of integer constant or a string is preceded by number of the int or string combinator (constructor). In S-expressions, this may be written as (int 5) or (string "Test").

However, what has been described above does not account for certain subtleties, such as the existence of naked types, or the difference between functions (active combinators whose application may be reduced, e.g. calculated) and constructors (passive combinators for which there are not and cannot be reduction rules). Furthermore, we have not explained how to handle polymorphic types and optional combinator parameters. We will attempt to explain this now.

@@ -87,7 +87,7 @@ pcons hd:Pair tl:PairList = PairList;

In practice, we most frequently need constant values (for storage and passing any data structures, in particular, responses to RPC queries) and surface expressions (for example, as RPC queries: then the functional combinator of the outer level is the name of the RPC function that we want to call, while its parameters are the arguments, which are constant values, for invoking the function). In some cases, arbitrary functional expressions are helpful (for example, it we want to remotely transmit the result of one RPC query to a different RPC query).

We will use c(T) to denote a subtype of the abstract type T, whose values are constant expressions of type T. Clearly, c(T) possesses approximately the same constructors as T itself (with the types of all arguments Ti replaced by c(Ti), but it does not have functional combinators.

-

Analogously, we will use f(T) to denote a subtype of T, whose values are surface expressions of type T. Clearly, the combinators of f(T) are essentially functional combinators of type T, but c() applies to the types of these combinators’ arguments: The combinator A : T1->...->Tr->T turns into A' : c(T1)->...->c(Tr)->f(T). (See the clarification of this rule below.)

+

Analogously, we will use f(T) to denote a subtype of T, whose values are surface expressions of type T. Clearly, the combinators of f(T) are essentially functional combinators of type T, but c() applies to the types of these combinators' arguments: The combinator A : T1->...->Tr->T turns into A' : c(T1)->...->c(Tr)->f(T). (See the clarification of this rule below.)

Thus, we have defined two “functionals” c : Type -> Type and f : Type -> Type, such that forall T : Type, c(T) :- T and forall T : Type, f(T) :- T (writing T :- T' means that T is contained in T', or that T is a subtype of T').

We will assume that c and f are idempotent.

Naked types

@@ -107,12 +107,12 @@ pcons hd:Pair tl:PairList = PairList; 
pair {X Y : Type} x:X y:Y = Pair X Y;  // constructor
 seq_pair {X Y : Type} x:!X y:!Y = Pair X Y; // functional wrapper for sequential computation
 par_pair {X Y : Type} x:!X y:!Y = Pair X Y; // functional wrapper for parallel computation
-

Now the RPC query (seq_pair (factorial 2) (factorial 3)) : Pair int int first calculates factorial 2, then factorial 3, and returns the pair (pair 2 6). In this case, the sequence of operations isn’t important, because they do not have side effects. It would have been just as well to use (par_pair (factorial 2) (factorial 3)). However, this is not always the case.

+

Now the RPC query (seq_pair (factorial 2) (factorial 3)) : Pair int int first calculates factorial 2, then factorial 3, and returns the pair (pair 2 6). In this case, the sequence of operations isn't important, because they do not have side effects. It would have been just as well to use (par_pair (factorial 2) (factorial 3)). However, this is not always the case.

We can also define an analogy to a “comma” operation:

comma {X Y : Type} x:!X y:!Y = Y;

For example, this operation could first calculate x, then forget the result, calculate y, and return y.

Note that the semantics of the seq_pair, par_pair and comma wrappers are indeed defined where they are implemented (like the semantics of all other functional combinators), not by their TL declaration.

-

In principle, polymorphic wrappers like set_timeout can also be applied, for example, to “annotate” a RPC response’s constant values. For example, the server might return a response to a query together with the time it was calculated. However, a value of type !X must be constant, because that is what is expected as the enclosing expression’s value. In other words, set_timeout 239 E is a constant/surface value of type X if and only if E is such itself.

+

In principle, polymorphic wrappers like set_timeout can also be applied, for example, to “annotate” a RPC response's constant values. For example, the server might return a response to a query together with the time it was calculated. However, a value of type !X must be constant, because that is what is expected as the enclosing expression's value. In other words, set_timeout 239 E is a constant/surface value of type X if and only if E is such itself.

$ modifier

The idempotent modifier $ permits the use of arbitrary functional values of an appropriate type in contexts where only constants or surface values are usually allowed. It recursively transforms all combinators for all of the types involved, canceling the action of % and affixing $ to the parameter types and result of all combinators ($ is also added to the front of the transformed combinators). Moreover, built-in types are also transformed (in the final stage): $int = Int and $string = String.

This may be useful to create an RPC query that performs a “deep computation” of the expression passed to it:

@@ -122,9 +122,9 @@ par_pair {X Y : Type} x:!X y:!Y = Pair X Y; // functional wrapper for parallel c

(Note that the three has become clothed; the combinator $factorial has type $int -> $int).

This is very powerful tool. It does not have to be implemented in very simple versions of TL. $ is not encountered in currently used TL schemas.

More on modifiers

-

In fact, at least in terms of its application to serialization, the TL language by default implies the c() modifier around all combinators’ parameter types and results, while ! and $ cancel it (more accurately, ! only cancels, and in some sense $ reverses the meaning). This is why there is no explicit c() modifier in TL and why it is assumed that all functions only accept constant values and return constant results, unless otherwise specified.

+

In fact, at least in terms of its application to serialization, the TL language by default implies the c() modifier around all combinators' parameter types and results, while ! and $ cancel it (more accurately, ! only cancels, and in some sense $ reverses the meaning). This is why there is no explicit c() modifier in TL and why it is assumed that all functions only accept constant values and return constant results, unless otherwise specified.

You may think that some functional combinators may have a type such as partial_factorial n:int = $int; and that the RPC query (partial_factorial 3) might then unexpectedly return ($product (int 3) ($product (int 2) ($product (int 1) (int 1)))) : $int ...

-

It is probably more correct to think about the ! modifier as follows. All types initially include only constant values (and only constructors). The ! modifier makes a new type (it’s twin) out of each type. This new type has no inherent constructors. Functional combinators differ from constructors in that ! is implicitly added in front of their result’s type. After this, the (local or remote) process of calculating the expression can be represented using the polymorphic function eval : !X -> X.

+

It is probably more correct to think about the ! modifier as follows. All types initially include only constant values (and only constructors). The ! modifier makes a new type (it's twin) out of each type. This new type has no inherent constructors. Functional combinators differ from constructors in that ! is implicitly added in front of their result's type. After this, the (local or remote) process of calculating the expression can be represented using the polymorphic function eval : !X -> X.

Optional combinator parameters and their values

See Optional combinator parameters and their values.

diff --git a/data/corefork.telegram.org/mtproto/TL-combinators.html b/data/corefork.telegram.org/mtproto/TL-combinators.html index db917e56e8..865ce9ec91 100644 --- a/data/corefork.telegram.org/mtproto/TL-combinators.html +++ b/data/corefork.telegram.org/mtproto/TL-combinators.html @@ -82,13 +82,13 @@ Combinators in TL are…">

Any identifier that begins with an uppercase or lowercase letter and which does not contain references to a namespace can be a field (variable) identifier. Using uc-ident for identifiers of variable types and lc-indent for other variables is good practice.

  • -

    Next a combinator declaration contains the equals sign (=) and the result type (it may be composite or appearing for the first time). The result type may be polymorphic and/or dependent; any fields of the defined constructor’s fields of type Type or # may be returned (as subexpressions).

    +

    Next a combinator declaration contains the equals sign (=) and the result type (it may be composite or appearing for the first time). The result type may be polymorphic and/or dependent; any fields of the defined constructor's fields of type Type or # may be returned (as subexpressions).

  • A combinator declaration is terminated with a semicolon.

    • -

    In what follows, a constructor’s fields, variables, and arguments mean the same thing.

    +

    In what follows, a constructor's fields, variables, and arguments mean the same thing.

    Optional field declarations

    Example:

    diff --git a/data/corefork.telegram.org/mtproto/TL-polymorph.html b/data/corefork.telegram.org/mtproto/TL-polymorph.html index d37d6318ac..c7e6052faf 100644 --- a/data/corefork.telegram.org/mtproto/TL-polymorph.html +++ b/data/corefork.telegram.org/mtproto/TL-polymorph.html @@ -81,15 +81,15 @@ nil : forall (X:Type), X -> List X;
  • serialize the head of the list (hd) as a value of type X;
  • serialize the tail of the list as a value of the polymorphic type List X.
    • -

    In the first step, the natural question is which string exactly will be used to calculate the CRC32. It is proposed to take "cons X:Type hd:X tl:List X = List X” without the terminating semicolon and without any parentheses (closed type expressions are unambiguously reconstructed based on their construction’s prefix).

    +

    In the first step, the natural question is which string exactly will be used to calculate the CRC32. It is proposed to take "cons X:Type hd:X tl:List X = List X” without the terminating semicolon and without any parentheses (closed type expressions are unambiguously reconstructed based on their construction's prefix).

    In the last step, we recursively resolve the very same problem of serializing a value of type List X; we will consider it resolved based on the assumption of induction in the construction of the value being serialized. We will similarly consider the third step understandable (induction in the construction of the value being serialized).

    -

    We still need to describe how to transmit (serialize) types, e.g. values of type Type. Types in TL schemas currently appear only as constructors’ optional parameters and are therefore never serialized explicitly. Rather, their values are inferred from the previously known type of the value being serialized.

    +

    We still need to describe how to transmit (serialize) types, e.g. values of type Type. Types in TL schemas currently appear only as constructors' optional parameters and are therefore never serialized explicitly. Rather, their values are inferred from the previously known type of the value being serialized.

    For completeness we will describe how it would be possible to serialize types (values of type Type). However, keep in mind that for now this information is not useful. See Type serialization.

    Optional arguments in polymorphic constructors

    -

    It was stated above that any subset of (the first few) parameters of any constructor can be identified as optional (by enclosing their declarations in curly brackets), but this is not actually entirely accurate. First, these optional parameters can only be of type Type or # (natural numbers). Second, optional parameters must share the return value’s type, otherwise their value cannot be determined.

    -

    Note that @'''constr-id''' means the constructor’s “full form” (in which all optional parameters become required), while '''constr-id'’ denotes its abbreviated form (without the optional arguments). If there are no optional arguments, then these two forms are the same. Constructors’ full forms are never used at present.

    +

    It was stated above that any subset of (the first few) parameters of any constructor can be identified as optional (by enclosing their declarations in curly brackets), but this is not actually entirely accurate. First, these optional parameters can only be of type Type or # (natural numbers). Second, optional parameters must share the return value's type, otherwise their value cannot be determined.

    +

    Note that @'''constr-id''' means the constructor's “full form” (in which all optional parameters become required), while '''constr-id'' denotes its abbreviated form (without the optional arguments). If there are no optional arguments, then these two forms are the same. Constructors' full forms are never used at present.

    Bare polymorphic types

    -

    There is a small problem: if we want to serialize the value of the bare type ‘%pair string int’ or ‘%pair string Y’ (which in TL is usually denoted simply as “pair”, though the form “%Pair” is preferable), we cannot simultaneously use both the full constructor @pair and the partial pair, because the constructor’s name will not be serialized. Therefore, we must differentiate the bare types %@pair (type X, type Y, value x:X, and value y:Y are serialized) and %pair (only x:X and y:Y are serialized; types X and Y are known from the context). In practice, we nearly almost always need the bare type %pair, and this is precisely what “pair” means in the type’s context in TL. Therefore,

    +

    There is a small problem: if we want to serialize the value of the bare type ‘%pair string int' or ‘%pair string Y' (which in TL is usually denoted simply as “pair”, though the form “%Pair” is preferable), we cannot simultaneously use both the full constructor @pair and the partial pair, because the constructor's name will not be serialized. Therefore, we must differentiate the bare types %@pair (type X, type Y, value x:X, and value y:Y are serialized) and %pair (only x:X and y:Y are serialized; types X and Y are known from the context). In practice, we nearly almost always need the bare type %pair, and this is precisely what “pair” means in the type's context in TL. Therefore, 

    record name:string map:(List (pair int string)) = Record;

    will be serialized approximately like we want it to be (the serialization of list elements will consist of the serialization of int and the serialization of string, without any additional headers, types, or combinator names). Incidentally, when calculating the “record” combinator's name 'record' in the example given above, the CRC32 of record name:string map:List pair int string = Record will be computed.

    diff --git a/data/corefork.telegram.org/mtproto/TL-tl.html b/data/corefork.telegram.org/mtproto/TL-tl.html index 6eb37b1df6..81942ff457 100644 --- a/data/corefork.telegram.org/mtproto/TL-tl.html +++ b/data/corefork.telegram.org/mtproto/TL-tl.html @@ -97,8 +97,8 @@ tls.typeExpr name:int flags:int children_num:# children:children_num*[tls.Expr]

    Schema serialization (version 2) always begins with the index number of the tls.schema_v2 constructor for tls.Schema. Because the CRC32 of the string

    tls.schema_v2 version:int date:int types_num:# types:types_num*[ tls.Type ] constructor_num:# constructors:constructor_num*[ tls.Combinator ] functions_num:# functions:functions_num*[ tls.Combinator ] = tls.Schema
    -

    is 0x3a2f9be2, this constant is in fact the magic number for tlo files in the current version’s format. -If the format is extended in the future (for example, if TL’s additional features are supported), then a tls.schema_v3 constructor with a different number will appear.

    +

    is 0x3a2f9be2, this constant is in fact the magic number for tlo files in the current version's format. +If the format is extended in the future (for example, if TL's additional features are supported), then a tls.schema_v3 constructor with a different number will appear.

    Example

    If one adds declarations for the used built-in types (like int ? = Int;) from the file common.tl before tl.tl and serialize the resulting schema, the following binary data is obtained (tl.tlo):

    diff --git a/data/corefork.telegram.org/mtproto/TL-types.html b/data/corefork.telegram.org/mtproto/TL-types.html index 28c1e276c4..a5edce8017 100644 --- a/data/corefork.telegram.org/mtproto/TL-types.html +++ b/data/corefork.telegram.org/mtproto/TL-types.html @@ -44,8 +44,8 @@ It remains to describe how types, e.g. values of type Type, are transmitted (ser

    See Polymorphism in TL and TL Language.

    It remains to describe how types, e.g. values of type Type, are transmitted (serialized). In general, there is nothing unexpected going on here: we have type constructors of various arities (for example, List is an arity-1 constructor, but IntList is a 0-arity constructor); and if we know that a 32-bit “name” is assigned to each type constructor, there are no further questions -- values of type Type are serialized exactly like values of any other recursive type with a defined set of constructors of differing arity.

    How can a 32-bit “name” be assigned to a type (a type constructor, to be more exact) such as List or IntList? -It is proposed to use the sum of the names of all of its constructors, plus the CRC32 of the string with the designation of the type's name and all of its parameters such as “IntList = Type” or “List X:Type = Type”. This way, the List constructor’s “name” is the sum of the CRC32s of the three strings "List X:Type = Type", "cons X:Type hd:X tl:List X = List X", and "nil X:Type = List X". -For “bare” types (which, formally speaking, are subtypes of the corresponding “boxed” type), the situation is somewhat more complicated; the logical negation of the corresponding constructor’s name is used. For built-in bareand boxed types (for example, int and Int), a pseudo-declaration is used (for example, int ? = Int").

    +It is proposed to use the sum of the names of all of its constructors, plus the CRC32 of the string with the designation of the type's name and all of its parameters such as “IntList = Type” or “List X:Type = Type”. This way, the List constructor's “name” is the sum of the CRC32s of the three strings "List X:Type = Type", "cons X:Type hd:X tl:List X = List X", and "nil X:Type = List X". +For “bare” types (which, formally speaking, are subtypes of the corresponding “boxed” type), the situation is somewhat more complicated; the logical negation of the corresponding constructor's name is used. For built-in bareand boxed types (for example, int and Int), a pseudo-declaration is used (for example, int ? = Int").

    diff --git a/data/corefork.telegram.org/mtproto/auth_key.html b/data/corefork.telegram.org/mtproto/auth_key.html index 32dc7a40dd..6aa9af42c0 100644 --- a/data/corefork.telegram.org/mtproto/auth_key.html +++ b/data/corefork.telegram.org/mtproto/auth_key.html @@ -41,100 +41,152 @@

    The query format is described using Binary Data Serialization and the TL Language. All large numbers are transmitted as strings containing the required sequence of bytes in big endian order. Hash functions, such as SHA1, return strings (of 20 bytes) which can also be interpreted as big endian numbers. Small numbers (int, long, int128, int256) are normally little endian; however, if they are part of SHA1, the bytes are not rearranged. This way, if long x is the 64 lower-order bits of SHA1 of string s, then the final 8 bytes of 20-byte string SHA1(s) are taken and interpreted as a 64-bit integer.

    Prior to sending off unencrypted messages (required in this instance to generate an authorization key), the client must undergo (p,q) authorization as follows.

    -

    DH exchange initiation

    -

    1) Client sends query to server

    -
    req_pq_multi#be7e8ef1 nonce:int128 = ResPQ;
    +

    DH exchange initiation

    +
      +
    1. +

      Client sends query to server

      +

      req_pq_multi#be7e8ef1 nonce:int128 = ResPQ;

      +
      2. +

    The value of nonce is selected randomly by the client (random number) and identifies the client within this communication. Following this step, it is known to all.

    -

    2) Server sends response of the form

    -
    resPQ#05162463 nonce:int128 server_nonce:int128 pq:string server_public_key_fingerprints:Vector long = ResPQ;
    +
      +
    1. +

      Server sends response of the form

      +

      resPQ#05162463 nonce:int128 server_nonce:int128 pq:string server_public_key_fingerprints:Vector long = ResPQ;

      +
      2. +

    Here, string pq is a representation of a natural number (in binary big endian format). This number is the product of two different odd prime numbers. Normally, pq is less than or equal to 2^63-1. The value of server_nonce is selected randomly by the server; following this step, it is known to all.

    server_public_key_fingerprints is a list of public RSA key fingerprints (64 lower-order bits of SHA1 (server_public_key); the public key is represented as a bare type rsa_public_key n:string e:string = RSAPublicKey, where, as usual, n and е are numbers in big endian format serialized as strings of bytes, following which SHA1 is computed) received by the server.

    All subsequent messages contain the pair (nonce, server_nonce) both in the plain-text, and the encrypted portions which makes it possible to identify a “temporary session” — one run of the key generation protocol described on this page that uses the same (nonce, server_nonce) pair. An intruder could not create a parallel session with the server with the same parameters and reuse parts of server- or client-encrypted messages for its own purposes in such a parallel session, because a different server_nonce would be selected by the server for any new “temporary session”.

    -

    Proof of work

    -

    3) Client decomposes pq into prime factors such that p < q.

    +

    Proof of work

    +
      +
    1. Client decomposes pq into prime factors such that p < q.
      2. +

    This starts a round of Diffie-Hellman key exchanges.

    -

    Presenting proof of work; Server authentication

    -

    4) Client sends query to server

    -
    req_DH_params#d712e4be nonce:int128 server_nonce:int128 p:string q:string public_key_fingerprint:long encrypted_data:string = Server_DH_Params
    +

    Presenting proof of work; Server authentication

    +
      +
    1. +

      Client sends query to server

      +

      req_DH_params#d712e4be nonce:int128 server_nonce:int128 p:string q:string public_key_fingerprint:long encrypted_data:string = Server_DH_Params

      +
      2. +

    Here, encrypted_data is obtained as follows:

    -

    Someone might intercept the query and replace it with their own, independently decomposing pq into factors instead of the client. The only field that it makes sense to modify is new_nonce which would be the one an intruder would have to re-generate (because an intruder cannot decrypt the encrypted data sent by the client). Since all subsequent messages are encrypted using new_nonce or contain new_nonce_hash, they will not be processed by the client (an intruder would not be able to make it look as though they had been generated by the server because they would not contain new_nonce). Therefore, this intercept will only result in the intruder’s completing the authorization key generation protocol in place of the client and creating a new key (that has nothing to do with the client); however, the same effect could be achieved simply by creating a new key in one's own name.

    +

    Someone might intercept the query and replace it with their own, independently decomposing pq into factors instead of the client. The only field that it makes sense to modify is new_nonce which would be the one an intruder would have to re-generate (because an intruder cannot decrypt the encrypted data sent by the client). Since all subsequent messages are encrypted using new_nonce or contain new_nonce_hash, they will not be processed by the client (an intruder would not be able to make it look as though they had been generated by the server because they would not contain new_nonce). Therefore, this intercept will only result in the intruder's completing the authorization key generation protocol in place of the client and creating a new key (that has nothing to do with the client); however, the same effect could be achieved simply by creating a new key in one's own name.

    An alternative form of inner data (p_q_inner_data_temp_dc) is used to create temporary keys, that are only stored in the server RAM and are discarded after at most expires_in seconds. The server is free to discard its copy earlier. In all other respects the temporary key generation protocol is the same. After a temporary key is created, the client usually binds it to its principal authorisation key by means of the auth.bindTempAuthKey method, and uses it for all client-server communication until it expires; then a new temporary key is generated. Thus Perfect Forward Secrecy (PFS) in client-server communication is achieved. Read more about PFS »

    4.1) RSA_PAD(data, server_public_key) mentioned above is implemented as follows:

    -

    5) Server responds with:

    -
    server_DH_params_ok#d0e8075c nonce:int128 server_nonce:int128 encrypted_answer:string = Server_DH_Params;
    +
      +
    1. +

      Server responds with:

      +

      server_DH_params_ok#d0e8075c nonce:int128 server_nonce:int128 encrypted_answer:string = Server_DH_Params;

      +
      2. +

    If the query is incorrect, the server returns a -404 error and the handshake must be restarted (any subsequent request also returns -404, even if it is correct).

    Here, encrypted_answer is obtained as follows:

    Following this step, new_nonce is still known to client and server only. The client is certain that it is the server that responded and that the response was generated specifically in response to client query req_DH_params, since the response data are encrypted using new_nonce.

    -

    Client is expected to check whether p = dh_prime is a safe 2048-bit prime (meaning that both p and (p-1)/2 are prime, and that 22047 < p < 22048), and that g generates a cyclic subgroup of prime order (p-1)/2, i.e. is a quadratic residue mod p. Since g is always equal to 2, 3, 4, 5, 6 or 7, this is easily done using quadratic reciprocity law, yielding a simple condition on p mod 4g — namely, p mod 8 = 7 for g = 2; p mod 3 = 2 for g = 3; no extra condition for g = 4; p mod 5 = 1 or 4 for g = 5; p mod 24 = 19 or 23 for g = 6; and p mod 7 = 3, 5 or 6 for g = 7. After g and p have been checked by the client, it makes sense to cache the result, so as not to repeat lengthy computations in future.

    -

    If the verification takes too long time (which is the case for older mobile devices), one might initially run only 15 Miller—Rabin iterations for verifying primeness of p and (p - 1)/2 with error probability not exceeding one billionth, and do more iterations later in the background.

    -

    Another optimization is to embed into the client application code a small table with some known “good” couples (g,p) (or just known safe primes p, since the condition on g is easily verified during execution), checked during code generation phase, so as to avoid doing such verification during runtime altogether. Server changes these values rarely, thus one usually has to put the current value of server's dh_prime into such a table. For example, current value of dh_prime equals (in big-endian byte order)

    +

    Client is expected to check whether p = dh_prime is a safe 2048-bit prime (meaning that both p and (p-1)/2 are prime, and that 2^2047 < p < 2^2048), and that g generates a cyclic subgroup of prime order (p-1)/2, i.e. is a quadratic residue mod p. Since g is always equal to 2, 3, 4, 5, 6 or 7, this is easily done using quadratic reciprocity law, yielding a simple condition on p mod 4g -- namely, p mod 8 = 7 for g = 2; p mod 3 = 2 for g = 3; no extra condition for g = 4; p mod 5 = 1 or 4 for g = 5; p mod 24 = 19 or 23 for g = 6; and p mod 7 = 3, 5 or 6 for g = 7. After g and p have been checked by the client, it makes sense to cache the result, so as not to repeat lengthy computations in future.

    +

    If the verification takes too long time (which is the case for older mobile devices), one might initially run only 15 Miller--Rabin iterations for verifying primeness of p and (p - 1)/2 with error probability not exceeding one billionth, and do more iterations later in the background.

    +

    Another optimization is to embed into the client application code a small table with some known "good" couples (g,p) (or just known safe primes p, since the condition on g is easily verified during execution), checked during code generation phase, so as to avoid doing such verification during runtime altogether. Server changes these values rarely, thus one usually has to put the current value of server's dh_prime into such a table. For example, current value of dh_prime equals (in big-endian byte order)

    C7 1C AE B9 C6 B1 C9 04 8E 6C 52 2F 70 F1 3F 73 98 0D 40 23 8E 3E 21 C1 49 34 D0 37 56 3D 93 0F 48 19 8A 0A A7 C1 40 58 22 94 93 D2 25 30 F4 DB FA 33 6F 6E 0A C9 25 13 95 43 AE D4 4C CE 7C 37 20 FD 51 F6 94 58 70 5A C6 8C D4 FE 6B 6B 13 AB DC 97 46 51 29 69 32 84 54 F1 8F AF 8C 59 5F 64 24 77 FE 96 BB 2A 94 1D 5B CD 1D 4A C8 CC 49 88 07 08 FA 9B 37 8E 3C 4F 3A 90 60 BE E6 7C F9 A4 A4 A6 95 81 10 51 90 7E 16 27 53 B5 6B 0F 6B 41 0D BA 74 D8 A8 4B 2A 14 B3 14 4E 0E F1 28 47 54 FD 17 ED 95 0D 59 65 B4 B9 DD 46 58 2D B1 17 8D 16 9C 6B C4 65 B0 D6 FF 9C A3 92 8F EF 5B 9A E4 E4 18 FC 15 E8 3E BE A0 F8 7F A9 FF 5E ED 70 05 0D ED 28 49 F4 7B F9 59 D9 56 85 0C E9 29 85 1F 0D 81 15 F6 35 B1 05 EE 2E 4E 15 D0 4B 24 54 BF 6F 4F AD F0 34 B1 04 03 11 9C D8 E3 B9 2F CC 5B
    -

    6) Client computes random 2048-bit number b (using a sufficient amount of entropy) and sends the server a message

    -
    set_client_DH_params#f5045f1f nonce:int128 server_nonce:int128 encrypted_data:string = Set_client_DH_params_answer;
    +
      +
    1. +

      Client computes random 2048-bit number b (using a sufficient amount of entropy) and sends the server a message

      +

      set_client_DH_params#f5045f1f nonce:int128 server_nonce:int128 encrypted_data:string = Set_client_DH_params_answer;

      +
      2. +

    Here, encrypted_data is obtained thus:

    The retry_id field is equal to zero at the time of the first attempt; otherwise, it is equal to auth_key_aux_hash from the previous failed attempt (see Item 9).

    -

    7) Thereafter, auth_key equals pow(g, {ab}) mod dh_prime; on the server, it is computed as pow(g_b, a) mod dh_prime, and on the client as (g_a)^b mod dh_prime.

    -

    8) auth_key_hash is computed := 64 lower-order bits of SHA1 (auth_key). The server checks whether there already is another key with the same auth_key_hash and responds in one of the following ways.

    -

    DH key exchange complete

    -

    9) Server responds in one of three ways:

    -
    dh_gen_ok#3bcbf734 nonce:int128 server_nonce:int128 new_nonce_hash1:int128 = Set_client_DH_params_answer;
+
      
+
    1. 
+
      Thereafter, auth_key equals pow(g, {ab}) mod dh_prime; on the server, it is computed as pow(g_b, a) mod dh_prime, and on the client as (g_a)^b mod dh_prime.
      
+
      2. 
+
    2. 
+
      auth_key_hash is computed := 64 lower-order bits of SHA1 (auth_key). The server checks whether there already is another key with the same auth_key_hash and responds in one of the following ways.
      
+
      3. 
+
    
+
    DH key exchange complete
    
+
      
+
    1. 
+
      Server responds in one of three ways:
      
+
      dh_gen_ok#3bcbf734 nonce:int128 server_nonce:int128 new_nonce_hash1:int128 = Set_client_DH_params_answer;
 dh_gen_retry#46dc1fb9 nonce:int128 server_nonce:int128 new_nonce_hash2:int128 = Set_client_DH_params_answer;
-dh_gen_fail#a69dae02 nonce:int128 server_nonce:int128 new_nonce_hash3:int128 = Set_client_DH_params_answer;
    +dh_gen_fail#a69dae02 nonce:int128 server_nonce:int128 new_nonce_hash3:int128 = Set_client_DH_params_answer;

    + + -

    In the other case, the client goes to Item 6) generating a new b.
    In the first case, the client and the server have negotiated auth_key, following which they forget all other temporary data, and the client creates another encrypted session using auth_key. At the same time, server_salt is initially set to substr(new_nonce, 0, 8) XOR substr(server_nonce, 0, 8). If required, the client stores the difference between server_time received in 5) and its local time, to be able always to have a good approximation of server time which is required to generate correct message identifiers.

    +

    In the other case, the client goes to Item 6) generating a new b. +In the first case, the client and the server have negotiated auth_key, following which they forget all other temporary data, and the client creates another encrypted session using auth_key. At the same time, server_salt is initially set to substr(new_nonce, 0, 8) XOR substr(server_nonce, 0, 8). If required, the client stores the difference between server_time received in 5) and its local time, to be able always to have a good approximation of server time which is required to generate correct message identifiers.

    IMPORTANT: Apart from the conditions on the Diffie-Hellman prime dh_prime and generator g, both sides are to check that g, g_a and g_b are greater than 1 and less than dh_prime - 1. We recommend checking that g_a and g_b are between 2^{2048-64} and dh_prime - 2^{2048-64} as well.

    -

    Error Handling (Lost Queries and Responses)

    +

    Error Handling (Lost Queries and Responses)

    If the client fails to receive any response to its query from the server within a certain time interval, it may simply re-send the query. If the server has already sent a response to this query (exactly the same request and not just similar: all the parameters during the repeat request must take on the same values) but it did not get to the client, the server will simply re-send the same response. The server remembers the response for up to 10 minutes after having received the query in 1). If the server has already forgotten the response or the requisite temporary data, the client will have to start from the beginning.

    The server may consider that if the client has already sent in the next query using the data from the previous server response to the specific client, the response is known to have been received by the client and may be forgotten by the server.

    -

    Usage Example

    -

    An example of a complete list of queries required to generate an authorization key is shown on a separate page.

    -
    +

    Usage Example

    +

    An example of a complete list of queries required to generate an authorization key is shown on a separate page.

    diff --git a/data/corefork.telegram.org/mtproto/description.html b/data/corefork.telegram.org/mtproto/description.html index f1fffd7c32..7946155646 100644 --- a/data/corefork.telegram.org/mtproto/description.html +++ b/data/corefork.telegram.org/mtproto/description.html @@ -126,7 +126,7 @@ MTProto v.1.0 is deprecated and is currently being phased out.

    On top of this, msg_id values that belong over 30 seconds in the future or over 300 seconds in the past are to be ignored. This is especially important for the server. The client would also find this useful (to protect from a replay attack), but only if it is certain of its time (for example, if its time has been synchronized with that of the server).

    Certain client-to-server service messages containing data sent by the client to the server (for example, msg_id of a recent client query) may, nonetheless, be processed on the client even if the time appears to be “incorrect”. This is especially true of messages to change server_salt and notifications of invalid client time. See Mobile Protocol: Service Messages.

    Storing an Authorization Key on a Client Device

    -

    It may be suggested to users concerned with security that they password protect the authorization key in approximately the same way as in ssh. This can be accomplished by prepending the value of cryptographic hash function, such as SHA-256, of the key to the front of the key, following which the entire string is encrypted using AES in CBC mode and a key equal to the user’s (text) password. When the user inputs the password, the stored protected password is decrypted and verified by checking the SHA-256 value. From the user’s standpoint, this is practically the same as using an application or a website password.

    +

    It may be suggested to users concerned with security that they password protect the authorization key in approximately the same way as in ssh. This can be accomplished by prepending the value of cryptographic hash function, such as SHA-256, of the key to the front of the key, following which the entire string is encrypted using AES in CBC mode and a key equal to the user's (text) password. When the user inputs the password, the stored protected password is decrypted and verified by checking the SHA-256 value. From the user's standpoint, this is practically the same as using an application or a website password.

    Unencrypted Messages

    Special plain-text messages may be used to create an authorization key as well as to perform a time synchronization. They begin with auth_key_id = 0 (64 bits) which means that there is no auth_key. This is followed directly by the message body in serialized format without internal or external headers. A message identifier (64 bits) and body length in bytes (32 bytes) are added before the message body.

    Only a very limited number of messages of special types can be transmitted as plain text.

    diff --git a/data/corefork.telegram.org/mtproto/description_v1.html b/data/corefork.telegram.org/mtproto/description_v1.html index be348fa35a..6584530424 100644 --- a/data/corefork.telegram.org/mtproto/description_v1.html +++ b/data/corefork.telegram.org/mtproto/description_v1.html @@ -97,7 +97,7 @@ For information on encryption used in up-to-date Telegram clients, kindly see In addition, msg_id values that belong over 30 seconds in the future or over 300 seconds in the past are to be ignored. This is especially important for the server. The client would also find this useful (to protect from a replay attack), but only if it is certain of its time (for example, if its time has been synchronized with that of the server).

    Certain client-to-server service messages containing data sent by the client to the server (for example, msg_id of a recent client query) may, nonetheless, be processed on the client even if the time appears to be “incorrect”. This is especially true of messages to change server_salt and notifications of invalid client time. See Mobile Protocol: Service Messages.

    Storing an Authorization Key on a Client Device

    -

    It may be suggested to users concerned with security that they password protect the authorization key in approximately the same way as in ssh. This is accomplished by adding the SHA1 of the key to the front of the key, following which the entire string is encrypted using AES in CBC mode and a key equal to the user’s (text) password. When the user inputs the password, the stored protected password is decrypted and verified by being compared with SHA1. From the user’s standpoint, this is practically the same as using an application or a website password.

    +

    It may be suggested to users concerned with security that they password protect the authorization key in approximately the same way as in ssh. This is accomplished by adding the SHA1 of the key to the front of the key, following which the entire string is encrypted using AES in CBC mode and a key equal to the user's (text) password. When the user inputs the password, the stored protected password is decrypted and verified by being compared with SHA1. From the user's standpoint, this is practically the same as using an application or a website password.

    Unencrypted Messages

    Special plain-text messages may be used to create an authorization key as well as to perform a time synchronization. They begin with auth_key_id = 0 (64 bits) which means that there is no auth_key. This is followed directly by the message body in serialized format without internal or external headers. A message identifier (64 bits) and body length in bytes (32 bytes) are added before the message body.

    Only a very limited number of messages of special types can be transmitted as plain text.

    diff --git a/data/corefork.telegram.org/mtproto/serialize.html b/data/corefork.telegram.org/mtproto/serialize.html index e2028a2386..5a3241c06b 100644 --- a/data/corefork.telegram.org/mtproto/serialize.html +++ b/data/corefork.telegram.org/mtproto/serialize.html @@ -49,7 +49,7 @@ The TL language is used to describe the data types to
  • Value, in this case, is the same as a string in Alphabet A, i. e. a finite (possibly, empty) sequence of 32-bit numbers. The set of all such sequences is designated as A*.
  • Type, for our purposes, is the same as the set of legal values of a type, i. e. some set T which is a subset of A* and is a prefix code (i. e. no element of T may be a prefix for any other element). Therefore, any sequence from A* can contain no more than one prefix that is a member of T.
  • Value of Type T is any sequence (value) which is a member of T as a subset of A*.
    • -
  • Compatible Types are the types T and T’ not intersecting as subsets of A*, such that the union of T and T' is a prefix code.
    • +
  • Compatible Types are the types T and T' not intersecting as subsets of A*, such that the union of T and T' is a prefix code.
  • Coordinated System of Types is a finite or infinite set of types T_1, ..., T_n, ..., such that any two types from this set are compatible.
  • Data Type is the same as type in the sense of the definition above.
  • Functional Type is a type describing a function; it is not a type in the sense of the definition above. Initially, we ignore the existence of functional types and describe only the data types; however, in reality, functional types will later be implemented in some extension of this system using the so-called temporary combinators.
    • @@ -66,13 +66,13 @@ The TL language is used to describe the data types to

    Combinator identifier is an identifier beginning with a lowercase Roman letter that uniquely identifies a combinator.

  • -

    Combinator number or combinator name is a 32-bit number (i.e., an element of A) that uniquely identifies a combinator. Most often, it is CRC32 of the string containing the combinator description without the final semicolon, and with one space between contiguous lexemes. This always falls in the range from 0x01000000 to 0xffffff00. The highest 256 values are reserved for the so-called temporal-logic combinators used to transmit functions. We frequently denote as combinator the combinator name with single quotes: ‘combinator’.

    +

    Combinator number or combinator name is a 32-bit number (i.e., an element of A) that uniquely identifies a combinator. Most often, it is CRC32 of the string containing the combinator description without the final semicolon, and with one space between contiguous lexemes. This always falls in the range from 0x01000000 to 0xffffff00. The highest 256 values are reserved for the so-called temporal-logic combinators used to transmit functions. We frequently denote as combinator the combinator name with single quotes: ‘combinator'.

  • Combinator description is a string of format combinator_name type_arg_1 ... type_arg_N = type_res; where N stands for the arity of the combinator, type_arg_i is the type of the i-th argument (or rather, a string with the combinator name), and type_res is the combinator value type.

  • -

    Constructor is a combinator that cannot be computed (reduced). This is used to represent composite data types. For example, combinator ‘int_tree’ with description int_tree IntTree int IntTree = IntTree, alongside combinator empty_tree = IntTree, may be used to define a composite data type called “IntTree” that takes on values in the form of binary trees with integers as nodes.

    +

    Constructor is a combinator that cannot be computed (reduced). This is used to represent composite data types. For example, combinator ‘int_tree' with description int_tree IntTree int IntTree = IntTree, alongside combinator empty_tree = IntTree, may be used to define a composite data type called “IntTree” that takes on values in the form of binary trees with integers as nodes.

  • Function (functional combinator) is a combinator which may be computed (reduced) on condition that the requisite number of arguments of requisite types are provided. The result of the computation is an expression consisting of constructors and base type values only.

    @@ -87,17 +87,17 @@ The TL language is used to describe the data types to

    Type number or type name is a 32-bit number that uniquely identifies a type; it normally is the sum of the CRC32 values of the descriptions of the type constructors.

  • -

    Description of (composite) Type T is a collection of the descriptions of all constructors that take on Type T values. This is normally written as text with each string containing the description of a single constructor. Here is a description of Type ‘IntTree’, for example:

    +

    Description of (composite) Type T is a collection of the descriptions of all constructors that take on Type T values. This is normally written as text with each string containing the description of a single constructor. Here is a description of Type ‘IntTree', for example:

    int_tree IntTree int IntTree = IntTree; empty_tree = IntTree;

  • -

    Polymorphic type is a type whose description contains parameters (type variables) in lieu of actual types; approximately, what would be a template in C++. Here is a description of Type List alpha where List is a polymorphic type of arity 1 (i. e., dependent on a single argument), and alpha is a type variable which appears as the constructor’s optional parameter (in curly braces):

    +

    Polymorphic type is a type whose description contains parameters (type variables) in lieu of actual types; approximately, what would be a template in C++. Here is a description of Type List alpha where List is a polymorphic type of arity 1 (i. e., dependent on a single argument), and alpha is a type variable which appears as the constructor's optional parameter (in curly braces):

    cons {alpha:Type} alpha (List alpha) = List alpha; nil {alpha:Type} = List alpha;

  • -

    Value of (composite) Type T is any sequence from A* in the format constr_num arg1 ... argN, where constr_num is the index number of some Constructor C which takes on values of Type T, and arg_i is a value of Type T_i which is the type of the i-th argument to Constructor C. For example, let Combinator int_tree have the index number 17, whereas Combinator empty_tree has the index number 239. Then, the value of Type IntTree is, for example, 17 17 239 1 239 2 239 which is more conveniently written as 'int_tree' 'int_tree' 'empty_tree' 1 'empty_tree' 2 ‘empty_tree’. From the standpoint of a high-level language, this is int_tree (int_tree (empty_tree) 1 (empty_tree)) 2 (empty_tree): IntTree.

    +

    Value of (composite) Type T is any sequence from A* in the format constr_num arg1 ... argN, where constr_num is the index number of some Constructor C which takes on values of Type T, and arg_i is a value of Type T_i which is the type of the i-th argument to Constructor C. For example, let Combinator int_tree have the index number 17, whereas Combinator empty_tree has the index number 239. Then, the value of Type IntTree is, for example, 17 17 239 1 239 2 239 which is more conveniently written as 'int_tree' 'int_tree' 'empty_tree' 1 'empty_tree' 2 ‘empty_tree'. From the standpoint of a high-level language, this is int_tree (int_tree (empty_tree) 1 (empty_tree)) 2 (empty_tree): IntTree.

  • Schema is a collection of all the (composite) data type descriptions. This is used to define some agreed-to system of types.

    diff --git a/data/corefork.telegram.org/mtproto/service_messages.html b/data/corefork.telegram.org/mtproto/service_messages.html index 7ef9827419..652f38acef 100644 --- a/data/corefork.telegram.org/mtproto/service_messages.html +++ b/data/corefork.telegram.org/mtproto/service_messages.html @@ -47,20 +47,20 @@ rpc_result#f35c6d01 req_msg_id:long…">

    A response to an RPC query is normally wrapped as follows:

    rpc_result#f35c6d01 req_msg_id:long result:Object = RpcResult;

    Here req_msg_id is the identifier of the message sent by the other party and containing an RPC query. This way, the recipient knows that the result is a response to the specific RPC query in question. -At the same time, this response serves as acknowledgment of the other party’s receipt of the req_msg_id message.

    +At the same time, this response serves as acknowledgment of the other party's receipt of the req_msg_id message.

    Note that the response to an RPC query must also be acknowledged. Most frequently, this coincides with the transmission of the next message (which may have a container attached to it carrying a service message with the acknowledgment).

    RPC Error

    The result field returned in response to any RPC query may also contain an error message in the following format:

    rpc_error#2144ca19 error_code:int error_message:string = RpcError;

    Cancellation of an RPC Query

    -

    In certain situations, the client does not want to receive a response to an already transmitted RPC query, for example because the response turns out to be long and the client has decided to do without it because of insufficient link capacity. Simply interrupting the TCP connection will not have any effect because the server would re-send the missing response at the first opportunity. Therefore, the client needs a way to cancel receipt of the RPC response message, actually acknowledging its receipt prior to it being in fact received, which will settle the server down and prevent it from re-sending the response. However, the client does not know the RPC response’s msg_id prior to receiving the response; the only thing it knows is the req_msg_id. i. e. the msg_id of the relevant RPC query. Therefore, a special query is used:

    +

    In certain situations, the client does not want to receive a response to an already transmitted RPC query, for example because the response turns out to be long and the client has decided to do without it because of insufficient link capacity. Simply interrupting the TCP connection will not have any effect because the server would re-send the missing response at the first opportunity. Therefore, the client needs a way to cancel receipt of the RPC response message, actually acknowledging its receipt prior to it being in fact received, which will settle the server down and prevent it from re-sending the response. However, the client does not know the RPC response's msg_id prior to receiving the response; the only thing it knows is the req_msg_id. i. e. the msg_id of the relevant RPC query. Therefore, a special query is used:

    rpc_drop_answer#58e4a740 req_msg_id:long = RpcDropAnswer;

    The response to this query returns as one of the following messages wrapped in rpc_result and requiring an acknowledgment: 

    rpc_answer_unknown#5e2ad36e = RpcDropAnswer;
 rpc_answer_dropped_running#cd78e586 = RpcDropAnswer;
 rpc_answer_dropped#a43ad8b7 msg_id:long seq_no:int bytes:int = RpcDropAnswer;
    -

    The first version of the response is used if the server remembers nothing of the incoming req_msg_id (if it has already been responded to, for example). The second version is used if the response was canceled while the RPC query was being processed (where the RPC query itself was still fully processed); in this case, the same rpc_answer_dropped_running is also returned in response to the original query, and both of these responses require an acknowledgment from the client. The final version means that the RPC response was removed from the server’s outgoing queue, and its msg_id, seq_no, and length in bytes are transmitted to the client.

    -

    Note that rpc_answer_dropped_running and rpc_answer_dropped serve as acknowledgments of the server’s receipt of the original query (the same one, the response to which we wish to forget). In addition, same as for any RPC queries, any response to rpc_drop_answer is an acknowledgment for rpc_drop_answer itself.

    +

    The first version of the response is used if the server remembers nothing of the incoming req_msg_id (if it has already been responded to, for example). The second version is used if the response was canceled while the RPC query was being processed (where the RPC query itself was still fully processed); in this case, the same rpc_answer_dropped_running is also returned in response to the original query, and both of these responses require an acknowledgment from the client. The final version means that the RPC response was removed from the server's outgoing queue, and its msg_id, seq_no, and length in bytes are transmitted to the client.

    +

    Note that rpc_answer_dropped_running and rpc_answer_dropped serve as acknowledgments of the server's receipt of the original query (the same one, the response to which we wish to forget). In addition, same as for any RPC queries, any response to rpc_drop_answer is an acknowledgment for rpc_drop_answer itself.

    As an alternative to using rpc_drop_answer, a new session may be created after the connection is reset and the old session is removed through destroy_session.

    Messages associated with querying, changing, and receiving the status of other messages

    See Mobile Protocol: Service Messages about Messages

    @@ -69,7 +69,7 @@ rpc_answer_dropped#a43ad8b7 msg_id:long seq_no:int bytes:int = RpcDropAnswer;get_future_salts#b921bd04 num:int = FutureSalts; future_salt#0949d9dc valid_since:int valid_until:int salt:long = FutureSalt; future_salts#ae500895 req_msg_id:long now:int salts:vector<future_salt> = FutureSalts; -

    The client must check to see that the response’s req_msg_id in fact coincides with msg_id of the query for get_future_salts. The server returns a maximum of num future server salts (may return fewer). The response serves as the acknowledgment of the query and does not require an acknowledgment itself.

    +

    The client must check to see that the response's req_msg_id in fact coincides with msg_id of the query for get_future_salts. The server returns a maximum of num future server salts (may return fewer). The response serves as the acknowledgment of the query and does not require an acknowledgment itself.

    Ping Messages (PING/PONG)

    ping#7abe77ec ping_id:long = Pong;

    A response is usually returned to the same connection:

    @@ -84,7 +84,7 @@ future_salts#ae500895 req_msg_id:long now:int salts:vector<future_salt> = destroy_session_ok#e22045fc session_id:long = DestroySessionRes; destroy_session_none#62d350c9 session_id:long = DestroySessionRes;

    New Session Creation Notification

    -

    The server notifies the client that a new session (from the server’s standpoint) had to be created to handle a client message. If, after this, the server receives a message with an even smaller msg_id within the same session, a similar notification will be generated for this msg_id as well. No such notifications are generated for high msg_id values.

    +

    The server notifies the client that a new session (from the server's standpoint) had to be created to handle a client message. If, after this, the server receives a message with an even smaller msg_id within the same session, a similar notification will be generated for this msg_id as well. No such notifications are generated for high msg_id values.

    new_session_created#9ec20908 first_msg_id:long unique_id:long server_salt:long = NewSession

    The unique_id parameter is generated by the server every time a session is (re-)created.

    This notification must be acknowledged by the client. It is necessary, for instance, for the client to understand that there is, in fact, a “gap” in the stream of long poll notifications received from the server (the user may have failed to receive notifications during some period of time).

    @@ -105,7 +105,7 @@ Clients should group acknowledgments, state requests and message resend requests

    Message Copies

    In some situations, an old message with a msg_id that is no longer valid needs to be re-sent. Then, it is wrapped in a copy container:

    msg_copy#e06046b2 orig_message:Message = MessageCopy;
    -

    Once received, the message is processed as if the wrapper were not there. However, if it is known for certain that the message orig_message.msg_id was received, then the new message is not processed (while at the same time, it and orig_message.msg_id are acknowledged). The value of orig_message.msg_id must be lower than the container’s msg_id.

    +

    Once received, the message is processed as if the wrapper were not there. However, if it is known for certain that the message orig_message.msg_id was received, then the new message is not processed (while at the same time, it and orig_message.msg_id are acknowledged). The value of orig_message.msg_id must be lower than the container's msg_id.

    This is not used at this time, because an old message can be wrapped in a simple container with the same result.

    Packed Object

    Used to replace any other object (or rather, a serialization thereof) with its archived (gzipped) representation:

    @@ -119,7 +119,7 @@ Clients should group acknowledgments, state requests and message resend requests

    At the same time, the max_delay parameter has higher priority than wait_after, and max_wait has higher priority than max_delay.

    This message does not require a response or an acknowledgement. If the container transmitted over HTTP carries several such messages, the behavior is undefined (in fact, the latest parameter will be used).

    If no http_wait is present in container, default values max_delay=0 (milliseconds), wait_after=0 (milliseconds), and max_wait=25000 (milliseconds) are used.

    -

    If the client’s ping of the server takes a long time, it may make sense to set max_delay to a value that is comparable in magnitude to ping time.

    +

    If the client's ping of the server takes a long time, it may make sense to set max_delay to a value that is comparable in magnitude to ping time.

    diff --git a/data/corefork.telegram.org/mtproto/service_messages_about_messages.html b/data/corefork.telegram.org/mtproto/service_messages_about_messages.html index d271a6e13d..9fb9074f53 100644 --- a/data/corefork.telegram.org/mtproto/service_messages_about_messages.html +++ b/data/corefork.telegram.org/mtproto/service_messages_about_messages.html @@ -68,7 +68,7 @@ bad_server_salt#edab447b bad_msg_id:long bad_msg_seqno:int error_code:int new_se

    The intention is that error_code values are grouped (error_code >> 4): for example, the codes 0x40 - 0x4f correspond to errors in container decomposition.

    Notifications of an ignored message do not require acknowledgment (i.e., are irrelevant).

    -

    Important: if server_salt has changed on the server or if client time is incorrect, any query will result in a notification in the above format. The client must check that it has, in fact, recently sent a message with the specified msg_id, and if that is the case, update its time correction value (the difference between the client’s and the server’s clocks) and the server salt based on msg_id and the server_salt notification, so as to use these to (re)send future messages. In the meantime, the original message (the one that caused the error message to be returned) must also be re-sent with a better msg_id and/or server_salt.

    +

    Important: if server_salt has changed on the server or if client time is incorrect, any query will result in a notification in the above format. The client must check that it has, in fact, recently sent a message with the specified msg_id, and if that is the case, update its time correction value (the difference between the client's and the server's clocks) and the server salt based on msg_id and the server_salt notification, so as to use these to (re)send future messages. In the meantime, the original message (the one that caused the error message to be returned) must also be re-sent with a better msg_id and/or server_salt.

    In addition, the client can update the server_salt value used to send messages to the server, based on the values of RPC responses or containers carrying an RPC response, provided that this RPC response is actually a match for the query sent recently. (If there is doubt, it is best not to update since there is risk of a replay attack).

    Request for Message Status Information

    If either party has not received information on the status of its outgoing messages for a while, it may explicitly request it from the other party: