OpenSource/telegram-crawler
OpenSource/telegram-crawler
1
0
Fork 0
mirror of https://github.com/MarshalX/telegram-crawler.git synced 2024-12-28 07:20:31 +01:00
Code Issues Projects Releases Packages Wiki Activity

Update content of files

Browse source
This commit is contained in:
GitHub Action 2022-12-03 13:00:16 +00:00
parent c78f136327
commit dd88a6b1bb
3 changed files with 96 additions and 177 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

91
data/web/blogfork.telegram.org/mtproto.html
View file

@ -47,45 +47,24 @@ Client developers are required to comply with the Security…">
<div id="dev_page_content"><!-- scroll_nav -->
<blockquote>
<p>Please feel free to check out our <a href="http://core.telegram.org/techfaq">FAQ for the Technically Inclined</a>.
Client developers are required to comply with the <a href="/mtproto/security_guidelines">Security Guidelines</a>.</p>
<p>Please feel free to check out our <a href="http://core.telegram.org/techfaq">FAQ for the Technically Inclined</a>.<br>Client developers are required to comply with the <a href="/mtproto/security_guidelines">Security Guidelines</a>.</p>
</blockquote>
<h3><a class="anchor" href="#related-articles" id="related-articles" name="related-articles"><i class="anchor-icon"></i></a>Related articles</h3>
<h3><a class="anchor" name="related-articles" href="#related-articles"><i class="anchor-icon"></i></a>Related articles</h3>
<p><div class="dev_page_nav_wrap"></p>
<ul>
<li>
<p><a href="/mtproto/description">Mobile Protocol: Detailed Description</a></p>
<li><a href="/mtproto/description">Mobile Protocol: Detailed Description</a></li>
<li><a href="/mtproto/auth_key">Creating an Authorization Key</a></li>
<li><a href="/mtproto/samples-auth_key">Creating an Authorization Key: Example</a></li>
<li><a href="/mtproto/service_messages">Mobile Protocol: Service Messages</a></li>
<li><a href="/mtproto/service_messages_about_messages">Mobile Protocol: Service Messages about Messages</a></li>
<li><a href="/mtproto/serialize">Binary Data Serialization</a></li>
<li><p><a href="/mtproto/TL">TL Language</a></p>
</li>
<li>
<p><a href="/mtproto/auth_key">Creating an Authorization Key</a></p>
</li>
<li>
<p><a href="/mtproto/samples-auth_key">Creating an Authorization Key: Example</a></p>
</li>
<li>
<p><a href="/mtproto/service_messages">Mobile Protocol: Service Messages</a></p>
</li>
<li>
<p><a href="/mtproto/service_messages_about_messages">Mobile Protocol: Service Messages about Messages</a></p>
</li>
<li>
<p><a href="/mtproto/serialize">Binary Data Serialization</a></p>
</li>
<li>
<p><a href="/mtproto/TL">TL Language</a></p>
</li>
<li>
<p><a href="/schema/mtproto">MTProto TL-schema</a></p>
</li>
<li>
<p><a href="/api/end-to-end">End-to-end encryption, Secret Chats</a></p>
</li>
<li>
<p><a href="/schema/end-to-end">End-to-end TL-schema</a></p>
</li>
<li>
<p><a href="/mtproto/security_guidelines">Security Guidelines for Client Software Developers</a></p>
<li><p><a href="/schema/mtproto">MTProto TL-schema</a></p>
</li>
<li><a href="/api/end-to-end">End-to-end encryption, Secret Chats</a></li>
<li><a href="/schema/end-to-end">End-to-end TL-schema</a></li>
<li><a href="/mtproto/security_guidelines">Security Guidelines for Client Software Developers</a></li>
</ul>
<p></div></p>
<hr>
@ -94,7 +73,7 @@ Client developers are required to comply with the <a href="/mtproto/security_gui
<li><a href="/api/end-to-end">Secret Chats, end-to-end-encryption</a></li>
<li><a href="/api/end-to-end/voice-calls">End-to-end encrypted Voice Calls</a></li>
</ul>
<h3><a class="anchor" href="#general-description" id="general-description" name="general-description"><i class="anchor-icon"></i></a>General Description</h3>
<h3><a class="anchor" name="general-description" href="#general-description"><i class="anchor-icon"></i></a>General Description</h3>
<p>The protocol is designed for access to a server API from applications running on mobile devices. It must be emphasized that a web browser is not such an application.</p>
<p>The protocol is subdivided into three virtually independent components:</p>
<ul>
@ -103,14 +82,14 @@ Client developers are required to comply with the <a href="/mtproto/security_gui
<li>Transport component: defines the method for the client and the server to transmit messages over some other existing network protocol (such as HTTP, HTTPS, WS (plain websockets), WSS (websockets over HTTPS), TCP, UDP).</li>
</ul>
<div><a href="/file/811140746/2/CzMyJPVnPo8.81605/c2310d6ede1a5e220f">
<img src="/file/811140746/2/CzMyJPVnPo8.81605/c2310d6ede1a5e220f" alt="MTProto 2.0, server-client encryption, cloud chats" class="dev_page_image" style="max-width: 600px;">
<img src="/file/811140746/2/CzMyJPVnPo8.81605/c2310d6ede1a5e220f" alt="MTProto 2.0, server-client encryption, cloud chats" class="dev_page_image" style="max-width: 600px;" />
</a></div>
<blockquote>
<p>As of version 4.6, major Telegram clients are using <strong>MTProto 2.0</strong>, described in this article.
MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is deprecated and is currently being phased out. </p>
<p>As of version 4.6, major Telegram clients are using <strong>MTProto 2.0</strong>, described in this article.<br>MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is deprecated and is currently being phased out. </p>
</blockquote>
<h3><a class="anchor" href="#brief-component-summary" id="brief-component-summary" name="brief-component-summary"><i class="anchor-icon"></i></a>Brief Component Summary</h3>
<h4><a class="anchor" href="#high-level-component-rpc-query-languageapi" id="high-level-component-rpc-query-languageapi" name="high-level-component-rpc-query-languageapi"><i class="anchor-icon"></i></a>High-Level Component (RPC Query Language/API)</h4>
<h3><a class="anchor" name="brief-component-summary" href="#brief-component-summary"><i class="anchor-icon"></i></a>Brief Component Summary</h3>
<h4><a class="anchor" name="high-level-component-rpc-query-language-api" href="#high-level-component-rpc-query-language-api"><i class="anchor-icon"></i></a>High-Level Component (RPC Query Language/API)</h4>
<p>From the standpoint of the high-level component, the client and the server exchange <em>messages</em> inside a <em>session</em>. The session is attached to the client device (the application, to be more exact) rather than a specific websocket/http/https/tcp connection. In addition, each session is attached to a <em>user key ID</em> by which authorization is actually accomplished.</p>
<p>Several connections to a server may be open; messages may be sent in either direction through any of the connections (a response to a query is not necessarily returned through the same connection that carried the original query, although most often, that is the case; however, in no case can a message be returned through a connection belonging to a different session). When the UDP protocol is used, a response might be returned by a different IP address than the one to which the query had been sent.</p>
<p>There are several types of messages:</p>
@ -125,23 +104,18 @@ MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is depreca
<p>Each message, either individual or inside a container, consists of a <em>message identifier</em> (64 bits, see below), a <em>message sequence number within a session</em> (32 bits), the <em>length</em> (of the message body in bytes; 32 bits), and a <em>body</em> (any size which is a multiple of 4 bytes). In addition, when a container or a single message is sent, an <em>internal header</em> is added at the top (see below), then the entire message is encrypted, and an <em>external header</em> is placed at the top of the message (a 64-bit <em>key identifier</em> and a 128-bit <em>message key</em>).</p>
<p>A <em>message body</em> normally consists of a 32-bit <em>message type</em> followed by type-dependent <em>parameters</em>. In particular, each RPC function has a corresponding message type. For more detail, see <a href="/mtproto/serialize">Binary Data Serialization</a>, <a href="/mtproto/service_messages">Mobile Protocol: Service Messages</a>.</p>
<p>All numbers are written as little endian. However, very large numbers (2048-bit) used in RSA and DH are written in the big endian format because that is how the OpenSSL library does it.</p>
<h4><a class="anchor" href="#authorization-and-encryption" id="authorization-and-encryption" name="authorization-and-encryption"><i class="anchor-icon"></i></a>Authorization and Encryption</h4>
<h4><a class="anchor" name="authorization-and-encryption" href="#authorization-and-encryption"><i class="anchor-icon"></i></a>Authorization and Encryption</h4>
<p>Prior to a message (or a multipart message) being transmitted over a network using a transport protocol, it is encrypted in a certain way, and an <em>external header</em> is added at the top of the message which is: a 64-bit <em>key identifier</em> (that uniquely identifies an <em>authorization key</em> for the server as well as the <em>user</em>) and a 128-bit <em>message key</em>. A user key together with the message key defines an actual 256-bit key which is what encrypts the message using AES-256 encryption. Note that the initial part of the message to be encrypted contains variable data (session, message ID, sequence number, server salt) that obviously influences the message key (and thus the AES key and iv). The message key is defined as the 128 middle bits of the SHA256 of the message body (including session, message ID, etc.), including the padding bytes, prepended by 32 bytes taken from the authorization key. Multipart messages are encrypted as a single message.</p>
<blockquote>
<p>For a technical specification, see <a href="/mtproto/description">Mobile Protocol: Detailed Description</a></p>
</blockquote>
<p>The first thing a client application must do is <a href="/mtproto/auth_key">create an authorization key</a> which is normally generated when it is first run and almost never changes.</p>
<p>The protocol's principal drawback is that an intruder passively intercepting messages and then somehow appropriating the authorization key (for example, by stealing a device) will be able to decrypt all the intercepted messages <em>post factum</em>. This probably is not too much of a problem (by stealing a device, one could also gain access to all the information cached on the device without decrypting anything); however, the following steps could be taken to overcome this weakness:</p>
<ul>
<li><em>Session keys</em> generated using the Diffie-Hellman protocol and used in conjunction with the authorization and the message keys to select AES parameters. To create these, the first thing a client must do after creating a new session is send a special RPC query to the server (“generate session key”) to which the server will respond, whereupon all subsequent messages within the session are encrypted using the session key as well.</li>
<li>Protecting the key stored on the client device with a (text) password; this password is never stored in memory and is entered by a user when starting the application or more frequently (depending on application settings).</li>
<li>Data stored (cached) on the user device can also be protected by encryption using an authorization key which, in turn, is to be password-protected. Then, a password will be required to gain access even to that data.</li>
</ul>
<h4><a class="anchor" href="#time-synchronization" id="time-synchronization" name="time-synchronization"><i class="anchor-icon"></i></a>Time Synchronization</h4>
<p>If client time diverges widely from server time, a server may start ignoring client messages, or vice versa, because of an invalid message identifier (which is closely related to creation time). Under these circumstances, the server will send the client a special message containing the correct time and a certain 128-bit salt (either explicitly provided by the client in a special RPC synchronization request or equal to the key of the latest message received from the client during the current session). This message could be the first one in a container that includes other messages (if the time discrepancy is significant but does not as yet result in the client's messages being ignored).</p>
<p>Having received such a message or a container holding it, the client first performs a time synchronization (in effect, simply storing the difference between the server's time and its own to be able to compute the “correct” time in the future) and then verifies that the message identifiers for correctness.</p>
<p>To prevent attackers potentially intercepting encrypted messages from decrypting them <em>post factum</em> by somehow appropriating the authorization key (for example, by stealing a device – even though in that case one could also gain access to all the information cached on the device without decrypting anything), MTProto supports <a href="https://core.telegram.org/api/pfs">Perfect Forward Secrecy</a> in both <a href="https://core.telegram.org/api/pfs">cloud chats</a> and <a href="https://core.telegram.org/api/end-to-end/pfs">secret chats</a>.</p>
<h4><a class="anchor" name="time-synchronization" href="#time-synchronization"><i class="anchor-icon"></i></a>Time Synchronization</h4>
<p>If client time diverges widely from server time, a server may start ignoring client messages, or vice versa, because of an invalid message identifier (which is closely related to creation time). Under these circumstances, the server will send the client a special message containing the correct time and a certain 128-bit salt (either explicitly provided by the client in a special RPC synchronization request or equal to the key of the latest message received from the client during the current session). This message could be the first one in a container that includes other messages (if the time discrepancy is significant but does not as yet result in the client&#39;s messages being ignored).</p>
<p>Having received such a message or a container holding it, the client first performs a time synchronization (in effect, simply storing the difference between the server&#39;s time and its own to be able to compute the “correct” time in the future) and then verifies that the message identifiers for correctness.</p>
<p>Where a correction has been neglected, the client will have to generate a new session to assure the monotonicity of message identifiers.</p>
<h3><a class="anchor" href="#mtproto-transport" id="mtproto-transport" name="mtproto-transport"><i class="anchor-icon"></i></a>MTProto transport</h3>
<h3><a class="anchor" name="mtproto-transport" href="#mtproto-transport"><i class="anchor-icon"></i></a>MTProto transport</h3>
<p>Before being sent using the selected transport protocol, the payload has to be wrapped in a secondary protocol header, defined by the appropriate MTProto transport protocol. </p>
<ul>
<li><a href="mtproto/mtproto-transports#abridged">Abridged</a></li>
@ -149,17 +123,15 @@ MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is depreca
<li><a href="mtproto/mtproto-transports#padded-intermediate">Padded intermediate</a></li>
<li><a href="mtproto/mtproto-transports#full">Full</a></li>
</ul>
<p>The server recognizes these different protocols (and distinguishes them from HTTP, too) by the header.
Additionally, the following transport features can be used: </p>
<p>The server recognizes these different protocols (and distinguishes them from HTTP, too) by the header.<br>Additionally, the following transport features can be used: </p>
<ul>
<li><a href="mtproto/mtproto-transports#quick-ack">Quick ack</a></li>
<li><a href="mtproto/mtproto-transports#transport-errors">Transport errors</a></li>
<li><a href="mtproto/mtproto-transports#transport-obfuscation">Transport obfuscation</a></li>
</ul>
<p>Example implementations for these protocols can be seen in <a href="https://github.com/tdlib/td/blob/master/td/mtproto/TcpTransport.cpp">tdlib</a> and <a href="https://github.com/danog/MadelineProto/tree/master/src/danog/MadelineProto/Stream/MTProtoTransport">MadelineProto</a>.</p>
<h3><a class="anchor" href="#transport" id="transport" name="transport"><i class="anchor-icon"></i></a>Transport</h3>
<p>Enables the delivery of encrypted containers together with the external header (hereinafter, <em>Payload</em>) from client to server and back.
Multiple transport protocols are defined:</p>
<h3><a class="anchor" name="transport" href="#transport"><i class="anchor-icon"></i></a>Transport</h3>
<p>Enables the delivery of encrypted containers together with the external header (hereinafter, <em>Payload</em>) from client to server and back.<br>Multiple transport protocols are defined:</p>
<ul>
<li><a href="/mtproto/transports#tcp">TCP</a></li>
<li><a href="/mtproto/transports#websocket">Websocket</a></li>
@ -169,7 +141,7 @@ Multiple transport protocols are defined:</p>
<li>UDP</li>
</ul>
<p>(We shall examine only the first five types.)</p>
<h3><a class="anchor" href="#recap" id="recap" name="recap"><i class="anchor-icon"></i></a>Recap</h3>
<h3><a class="anchor" name="recap" href="#recap"><i class="anchor-icon"></i></a>Recap</h3>
<p>To recap, using the <a href="https://en.wikipedia.org/wiki/OSI_model#Layer_architecture">ISO/OSI stack as comparison</a>: </p>
<ul>
<li>Layer 7 (Application): <a href="#high-level-component-rpc-query-languageapi">High-level RPC API</a></li>
@ -183,8 +155,9 @@ Multiple transport protocols are defined:</p>
</li>
<li>Layer 3 (Network): IP</li>
<li>Layer 2 (Data link): MAC/LLC</li>
<li>Layer 1 (Physical): IEEE 802.3, IEEE 802.11, etc...</li>
</ul></div>
<li>Layer 1 (Physical): IEEE 802.3, IEEE 802.11, etc…</li>
</ul>
</div>
</div>

91
data/web/core.telegram.org/mtproto.html
View file

@ -47,45 +47,24 @@ Client developers are required to comply with the Security…">
<div id="dev_page_content"><!-- scroll_nav -->
<blockquote>
<p>Please feel free to check out our <a href="http://core.telegram.org/techfaq">FAQ for the Technically Inclined</a>.
Client developers are required to comply with the <a href="/mtproto/security_guidelines">Security Guidelines</a>.</p>
<p>Please feel free to check out our <a href="http://core.telegram.org/techfaq">FAQ for the Technically Inclined</a>.<br>Client developers are required to comply with the <a href="/mtproto/security_guidelines">Security Guidelines</a>.</p>
</blockquote>
<h3><a class="anchor" href="#related-articles" id="related-articles" name="related-articles"><i class="anchor-icon"></i></a>Related articles</h3>
<h3><a class="anchor" name="related-articles" href="#related-articles"><i class="anchor-icon"></i></a>Related articles</h3>
<p><div class="dev_page_nav_wrap"></p>
<ul>
<li>
<p><a href="/mtproto/description">Mobile Protocol: Detailed Description</a></p>
<li><a href="/mtproto/description">Mobile Protocol: Detailed Description</a></li>
<li><a href="/mtproto/auth_key">Creating an Authorization Key</a></li>
<li><a href="/mtproto/samples-auth_key">Creating an Authorization Key: Example</a></li>
<li><a href="/mtproto/service_messages">Mobile Protocol: Service Messages</a></li>
<li><a href="/mtproto/service_messages_about_messages">Mobile Protocol: Service Messages about Messages</a></li>
<li><a href="/mtproto/serialize">Binary Data Serialization</a></li>
<li><p><a href="/mtproto/TL">TL Language</a></p>
</li>
<li>
<p><a href="/mtproto/auth_key">Creating an Authorization Key</a></p>
</li>
<li>
<p><a href="/mtproto/samples-auth_key">Creating an Authorization Key: Example</a></p>
</li>
<li>
<p><a href="/mtproto/service_messages">Mobile Protocol: Service Messages</a></p>
</li>
<li>
<p><a href="/mtproto/service_messages_about_messages">Mobile Protocol: Service Messages about Messages</a></p>
</li>
<li>
<p><a href="/mtproto/serialize">Binary Data Serialization</a></p>
</li>
<li>
<p><a href="/mtproto/TL">TL Language</a></p>
</li>
<li>
<p><a href="/schema/mtproto">MTProto TL-schema</a></p>
</li>
<li>
<p><a href="/api/end-to-end">End-to-end encryption, Secret Chats</a></p>
</li>
<li>
<p><a href="/schema/end-to-end">End-to-end TL-schema</a></p>
</li>
<li>
<p><a href="/mtproto/security_guidelines">Security Guidelines for Client Software Developers</a></p>
<li><p><a href="/schema/mtproto">MTProto TL-schema</a></p>
</li>
<li><a href="/api/end-to-end">End-to-end encryption, Secret Chats</a></li>
<li><a href="/schema/end-to-end">End-to-end TL-schema</a></li>
<li><a href="/mtproto/security_guidelines">Security Guidelines for Client Software Developers</a></li>
</ul>
<p></div></p>
<hr>
@ -94,7 +73,7 @@ Client developers are required to comply with the <a href="/mtproto/security_gui
<li><a href="/api/end-to-end">Secret Chats, end-to-end-encryption</a></li>
<li><a href="/api/end-to-end/voice-calls">End-to-end encrypted Voice Calls</a></li>
</ul>
<h3><a class="anchor" href="#general-description" id="general-description" name="general-description"><i class="anchor-icon"></i></a>General Description</h3>
<h3><a class="anchor" name="general-description" href="#general-description"><i class="anchor-icon"></i></a>General Description</h3>
<p>The protocol is designed for access to a server API from applications running on mobile devices. It must be emphasized that a web browser is not such an application.</p>
<p>The protocol is subdivided into three virtually independent components:</p>
<ul>
@ -103,14 +82,14 @@ Client developers are required to comply with the <a href="/mtproto/security_gui
<li>Transport component: defines the method for the client and the server to transmit messages over some other existing network protocol (such as HTTP, HTTPS, WS (plain websockets), WSS (websockets over HTTPS), TCP, UDP).</li>
</ul>
<div><a href="/file/811140746/2/CzMyJPVnPo8.81605/c2310d6ede1a5e220f">
<img src="/file/811140746/2/CzMyJPVnPo8.81605/c2310d6ede1a5e220f" alt="MTProto 2.0, server-client encryption, cloud chats" class="dev_page_image" style="max-width: 600px;">
<img src="/file/811140746/2/CzMyJPVnPo8.81605/c2310d6ede1a5e220f" alt="MTProto 2.0, server-client encryption, cloud chats" class="dev_page_image" style="max-width: 600px;" />
</a></div>
<blockquote>
<p>As of version 4.6, major Telegram clients are using <strong>MTProto 2.0</strong>, described in this article.
MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is deprecated and is currently being phased out. </p>
<p>As of version 4.6, major Telegram clients are using <strong>MTProto 2.0</strong>, described in this article.<br>MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is deprecated and is currently being phased out. </p>
</blockquote>
<h3><a class="anchor" href="#brief-component-summary" id="brief-component-summary" name="brief-component-summary"><i class="anchor-icon"></i></a>Brief Component Summary</h3>
<h4><a class="anchor" href="#high-level-component-rpc-query-languageapi" id="high-level-component-rpc-query-languageapi" name="high-level-component-rpc-query-languageapi"><i class="anchor-icon"></i></a>High-Level Component (RPC Query Language/API)</h4>
<h3><a class="anchor" name="brief-component-summary" href="#brief-component-summary"><i class="anchor-icon"></i></a>Brief Component Summary</h3>
<h4><a class="anchor" name="high-level-component-rpc-query-language-api" href="#high-level-component-rpc-query-language-api"><i class="anchor-icon"></i></a>High-Level Component (RPC Query Language/API)</h4>
<p>From the standpoint of the high-level component, the client and the server exchange <em>messages</em> inside a <em>session</em>. The session is attached to the client device (the application, to be more exact) rather than a specific websocket/http/https/tcp connection. In addition, each session is attached to a <em>user key ID</em> by which authorization is actually accomplished.</p>
<p>Several connections to a server may be open; messages may be sent in either direction through any of the connections (a response to a query is not necessarily returned through the same connection that carried the original query, although most often, that is the case; however, in no case can a message be returned through a connection belonging to a different session). When the UDP protocol is used, a response might be returned by a different IP address than the one to which the query had been sent.</p>
<p>There are several types of messages:</p>
@ -125,23 +104,18 @@ MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is depreca
<p>Each message, either individual or inside a container, consists of a <em>message identifier</em> (64 bits, see below), a <em>message sequence number within a session</em> (32 bits), the <em>length</em> (of the message body in bytes; 32 bits), and a <em>body</em> (any size which is a multiple of 4 bytes). In addition, when a container or a single message is sent, an <em>internal header</em> is added at the top (see below), then the entire message is encrypted, and an <em>external header</em> is placed at the top of the message (a 64-bit <em>key identifier</em> and a 128-bit <em>message key</em>).</p>
<p>A <em>message body</em> normally consists of a 32-bit <em>message type</em> followed by type-dependent <em>parameters</em>. In particular, each RPC function has a corresponding message type. For more detail, see <a href="/mtproto/serialize">Binary Data Serialization</a>, <a href="/mtproto/service_messages">Mobile Protocol: Service Messages</a>.</p>
<p>All numbers are written as little endian. However, very large numbers (2048-bit) used in RSA and DH are written in the big endian format because that is how the OpenSSL library does it.</p>
<h4><a class="anchor" href="#authorization-and-encryption" id="authorization-and-encryption" name="authorization-and-encryption"><i class="anchor-icon"></i></a>Authorization and Encryption</h4>
<h4><a class="anchor" name="authorization-and-encryption" href="#authorization-and-encryption"><i class="anchor-icon"></i></a>Authorization and Encryption</h4>
<p>Prior to a message (or a multipart message) being transmitted over a network using a transport protocol, it is encrypted in a certain way, and an <em>external header</em> is added at the top of the message which is: a 64-bit <em>key identifier</em> (that uniquely identifies an <em>authorization key</em> for the server as well as the <em>user</em>) and a 128-bit <em>message key</em>. A user key together with the message key defines an actual 256-bit key which is what encrypts the message using AES-256 encryption. Note that the initial part of the message to be encrypted contains variable data (session, message ID, sequence number, server salt) that obviously influences the message key (and thus the AES key and iv). The message key is defined as the 128 middle bits of the SHA256 of the message body (including session, message ID, etc.), including the padding bytes, prepended by 32 bytes taken from the authorization key. Multipart messages are encrypted as a single message.</p>
<blockquote>
<p>For a technical specification, see <a href="/mtproto/description">Mobile Protocol: Detailed Description</a></p>
</blockquote>
<p>The first thing a client application must do is <a href="/mtproto/auth_key">create an authorization key</a> which is normally generated when it is first run and almost never changes.</p>
<p>The protocol's principal drawback is that an intruder passively intercepting messages and then somehow appropriating the authorization key (for example, by stealing a device) will be able to decrypt all the intercepted messages <em>post factum</em>. This probably is not too much of a problem (by stealing a device, one could also gain access to all the information cached on the device without decrypting anything); however, the following steps could be taken to overcome this weakness:</p>
<ul>
<li><em>Session keys</em> generated using the Diffie-Hellman protocol and used in conjunction with the authorization and the message keys to select AES parameters. To create these, the first thing a client must do after creating a new session is send a special RPC query to the server (“generate session key”) to which the server will respond, whereupon all subsequent messages within the session are encrypted using the session key as well.</li>
<li>Protecting the key stored on the client device with a (text) password; this password is never stored in memory and is entered by a user when starting the application or more frequently (depending on application settings).</li>
<li>Data stored (cached) on the user device can also be protected by encryption using an authorization key which, in turn, is to be password-protected. Then, a password will be required to gain access even to that data.</li>
</ul>
<h4><a class="anchor" href="#time-synchronization" id="time-synchronization" name="time-synchronization"><i class="anchor-icon"></i></a>Time Synchronization</h4>
<p>If client time diverges widely from server time, a server may start ignoring client messages, or vice versa, because of an invalid message identifier (which is closely related to creation time). Under these circumstances, the server will send the client a special message containing the correct time and a certain 128-bit salt (either explicitly provided by the client in a special RPC synchronization request or equal to the key of the latest message received from the client during the current session). This message could be the first one in a container that includes other messages (if the time discrepancy is significant but does not as yet result in the client's messages being ignored).</p>
<p>Having received such a message or a container holding it, the client first performs a time synchronization (in effect, simply storing the difference between the server's time and its own to be able to compute the “correct” time in the future) and then verifies that the message identifiers for correctness.</p>
<p>To prevent attackers potentially intercepting encrypted messages from decrypting them <em>post factum</em> by somehow appropriating the authorization key (for example, by stealing a device – even though in that case one could also gain access to all the information cached on the device without decrypting anything), MTProto supports <a href="https://core.telegram.org/api/pfs">Perfect Forward Secrecy</a> in both <a href="https://core.telegram.org/api/pfs">cloud chats</a> and <a href="https://core.telegram.org/api/end-to-end/pfs">secret chats</a>.</p>
<h4><a class="anchor" name="time-synchronization" href="#time-synchronization"><i class="anchor-icon"></i></a>Time Synchronization</h4>
<p>If client time diverges widely from server time, a server may start ignoring client messages, or vice versa, because of an invalid message identifier (which is closely related to creation time). Under these circumstances, the server will send the client a special message containing the correct time and a certain 128-bit salt (either explicitly provided by the client in a special RPC synchronization request or equal to the key of the latest message received from the client during the current session). This message could be the first one in a container that includes other messages (if the time discrepancy is significant but does not as yet result in the client&#39;s messages being ignored).</p>
<p>Having received such a message or a container holding it, the client first performs a time synchronization (in effect, simply storing the difference between the server&#39;s time and its own to be able to compute the “correct” time in the future) and then verifies that the message identifiers for correctness.</p>
<p>Where a correction has been neglected, the client will have to generate a new session to assure the monotonicity of message identifiers.</p>
<h3><a class="anchor" href="#mtproto-transport" id="mtproto-transport" name="mtproto-transport"><i class="anchor-icon"></i></a>MTProto transport</h3>
<h3><a class="anchor" name="mtproto-transport" href="#mtproto-transport"><i class="anchor-icon"></i></a>MTProto transport</h3>
<p>Before being sent using the selected transport protocol, the payload has to be wrapped in a secondary protocol header, defined by the appropriate MTProto transport protocol. </p>
<ul>
<li><a href="mtproto/mtproto-transports#abridged">Abridged</a></li>
@ -149,17 +123,15 @@ MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is depreca
<li><a href="mtproto/mtproto-transports#padded-intermediate">Padded intermediate</a></li>
<li><a href="mtproto/mtproto-transports#full">Full</a></li>
</ul>
<p>The server recognizes these different protocols (and distinguishes them from HTTP, too) by the header.
Additionally, the following transport features can be used: </p>
<p>The server recognizes these different protocols (and distinguishes them from HTTP, too) by the header.<br>Additionally, the following transport features can be used: </p>
<ul>
<li><a href="mtproto/mtproto-transports#quick-ack">Quick ack</a></li>
<li><a href="mtproto/mtproto-transports#transport-errors">Transport errors</a></li>
<li><a href="mtproto/mtproto-transports#transport-obfuscation">Transport obfuscation</a></li>
</ul>
<p>Example implementations for these protocols can be seen in <a href="https://github.com/tdlib/td/blob/master/td/mtproto/TcpTransport.cpp">tdlib</a> and <a href="https://github.com/danog/MadelineProto/tree/master/src/danog/MadelineProto/Stream/MTProtoTransport">MadelineProto</a>.</p>
<h3><a class="anchor" href="#transport" id="transport" name="transport"><i class="anchor-icon"></i></a>Transport</h3>
<p>Enables the delivery of encrypted containers together with the external header (hereinafter, <em>Payload</em>) from client to server and back.
Multiple transport protocols are defined:</p>
<h3><a class="anchor" name="transport" href="#transport"><i class="anchor-icon"></i></a>Transport</h3>
<p>Enables the delivery of encrypted containers together with the external header (hereinafter, <em>Payload</em>) from client to server and back.<br>Multiple transport protocols are defined:</p>
<ul>
<li><a href="/mtproto/transports#tcp">TCP</a></li>
<li><a href="/mtproto/transports#websocket">Websocket</a></li>
@ -169,7 +141,7 @@ Multiple transport protocols are defined:</p>
<li>UDP</li>
</ul>
<p>(We shall examine only the first five types.)</p>
<h3><a class="anchor" href="#recap" id="recap" name="recap"><i class="anchor-icon"></i></a>Recap</h3>
<h3><a class="anchor" name="recap" href="#recap"><i class="anchor-icon"></i></a>Recap</h3>
<p>To recap, using the <a href="https://en.wikipedia.org/wiki/OSI_model#Layer_architecture">ISO/OSI stack as comparison</a>: </p>
<ul>
<li>Layer 7 (Application): <a href="#high-level-component-rpc-query-languageapi">High-level RPC API</a></li>
@ -183,8 +155,9 @@ Multiple transport protocols are defined:</p>
</li>
<li>Layer 3 (Network): IP</li>
<li>Layer 2 (Data link): MAC/LLC</li>
<li>Layer 1 (Physical): IEEE 802.3, IEEE 802.11, etc...</li>
</ul></div>
<li>Layer 1 (Physical): IEEE 802.3, IEEE 802.11, etc…</li>
</ul>
</div>
</div>

91
data/web/corefork.telegram.org/mtproto.html
View file

@ -47,45 +47,24 @@ Client developers are required to comply with the Security…">
<div id="dev_page_content"><!-- scroll_nav -->
<blockquote>
<p>Please feel free to check out our <a href="http://core.telegram.org/techfaq">FAQ for the Technically Inclined</a>.
Client developers are required to comply with the <a href="/mtproto/security_guidelines">Security Guidelines</a>.</p>
<p>Please feel free to check out our <a href="http://core.telegram.org/techfaq">FAQ for the Technically Inclined</a>.<br>Client developers are required to comply with the <a href="/mtproto/security_guidelines">Security Guidelines</a>.</p>
</blockquote>
<h3><a class="anchor" href="#related-articles" id="related-articles" name="related-articles"><i class="anchor-icon"></i></a>Related articles</h3>
<h3><a class="anchor" name="related-articles" href="#related-articles"><i class="anchor-icon"></i></a>Related articles</h3>
<p><div class="dev_page_nav_wrap"></p>
<ul>
<li>
<p><a href="/mtproto/description">Mobile Protocol: Detailed Description</a></p>
<li><a href="/mtproto/description">Mobile Protocol: Detailed Description</a></li>
<li><a href="/mtproto/auth_key">Creating an Authorization Key</a></li>
<li><a href="/mtproto/samples-auth_key">Creating an Authorization Key: Example</a></li>
<li><a href="/mtproto/service_messages">Mobile Protocol: Service Messages</a></li>
<li><a href="/mtproto/service_messages_about_messages">Mobile Protocol: Service Messages about Messages</a></li>
<li><a href="/mtproto/serialize">Binary Data Serialization</a></li>
<li><p><a href="/mtproto/TL">TL Language</a></p>
</li>
<li>
<p><a href="/mtproto/auth_key">Creating an Authorization Key</a></p>
</li>
<li>
<p><a href="/mtproto/samples-auth_key">Creating an Authorization Key: Example</a></p>
</li>
<li>
<p><a href="/mtproto/service_messages">Mobile Protocol: Service Messages</a></p>
</li>
<li>
<p><a href="/mtproto/service_messages_about_messages">Mobile Protocol: Service Messages about Messages</a></p>
</li>
<li>
<p><a href="/mtproto/serialize">Binary Data Serialization</a></p>
</li>
<li>
<p><a href="/mtproto/TL">TL Language</a></p>
</li>
<li>
<p><a href="/schema/mtproto">MTProto TL-schema</a></p>
</li>
<li>
<p><a href="/api/end-to-end">End-to-end encryption, Secret Chats</a></p>
</li>
<li>
<p><a href="/schema/end-to-end">End-to-end TL-schema</a></p>
</li>
<li>
<p><a href="/mtproto/security_guidelines">Security Guidelines for Client Software Developers</a></p>
<li><p><a href="/schema/mtproto">MTProto TL-schema</a></p>
</li>
<li><a href="/api/end-to-end">End-to-end encryption, Secret Chats</a></li>
<li><a href="/schema/end-to-end">End-to-end TL-schema</a></li>
<li><a href="/mtproto/security_guidelines">Security Guidelines for Client Software Developers</a></li>
</ul>
<p></div></p>
<hr>
@ -94,7 +73,7 @@ Client developers are required to comply with the <a href="/mtproto/security_gui
<li><a href="/api/end-to-end">Secret Chats, end-to-end-encryption</a></li>
<li><a href="/api/end-to-end/voice-calls">End-to-end encrypted Voice Calls</a></li>
</ul>
<h3><a class="anchor" href="#general-description" id="general-description" name="general-description"><i class="anchor-icon"></i></a>General Description</h3>
<h3><a class="anchor" name="general-description" href="#general-description"><i class="anchor-icon"></i></a>General Description</h3>
<p>The protocol is designed for access to a server API from applications running on mobile devices. It must be emphasized that a web browser is not such an application.</p>
<p>The protocol is subdivided into three virtually independent components:</p>
<ul>
@ -103,14 +82,14 @@ Client developers are required to comply with the <a href="/mtproto/security_gui
<li>Transport component: defines the method for the client and the server to transmit messages over some other existing network protocol (such as HTTP, HTTPS, WS (plain websockets), WSS (websockets over HTTPS), TCP, UDP).</li>
</ul>
<div><a href="/file/811140746/2/CzMyJPVnPo8.81605/c2310d6ede1a5e220f">
<img src="/file/811140746/2/CzMyJPVnPo8.81605/c2310d6ede1a5e220f" alt="MTProto 2.0, server-client encryption, cloud chats" class="dev_page_image" style="max-width: 600px;">
<img src="/file/811140746/2/CzMyJPVnPo8.81605/c2310d6ede1a5e220f" alt="MTProto 2.0, server-client encryption, cloud chats" class="dev_page_image" style="max-width: 600px;" />
</a></div>
<blockquote>
<p>As of version 4.6, major Telegram clients are using <strong>MTProto 2.0</strong>, described in this article.
MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is deprecated and is currently being phased out. </p>
<p>As of version 4.6, major Telegram clients are using <strong>MTProto 2.0</strong>, described in this article.<br>MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is deprecated and is currently being phased out. </p>
</blockquote>
<h3><a class="anchor" href="#brief-component-summary" id="brief-component-summary" name="brief-component-summary"><i class="anchor-icon"></i></a>Brief Component Summary</h3>
<h4><a class="anchor" href="#high-level-component-rpc-query-languageapi" id="high-level-component-rpc-query-languageapi" name="high-level-component-rpc-query-languageapi"><i class="anchor-icon"></i></a>High-Level Component (RPC Query Language/API)</h4>
<h3><a class="anchor" name="brief-component-summary" href="#brief-component-summary"><i class="anchor-icon"></i></a>Brief Component Summary</h3>
<h4><a class="anchor" name="high-level-component-rpc-query-language-api" href="#high-level-component-rpc-query-language-api"><i class="anchor-icon"></i></a>High-Level Component (RPC Query Language/API)</h4>
<p>From the standpoint of the high-level component, the client and the server exchange <em>messages</em> inside a <em>session</em>. The session is attached to the client device (the application, to be more exact) rather than a specific websocket/http/https/tcp connection. In addition, each session is attached to a <em>user key ID</em> by which authorization is actually accomplished.</p>
<p>Several connections to a server may be open; messages may be sent in either direction through any of the connections (a response to a query is not necessarily returned through the same connection that carried the original query, although most often, that is the case; however, in no case can a message be returned through a connection belonging to a different session). When the UDP protocol is used, a response might be returned by a different IP address than the one to which the query had been sent.</p>
<p>There are several types of messages:</p>
@ -125,23 +104,18 @@ MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is depreca
<p>Each message, either individual or inside a container, consists of a <em>message identifier</em> (64 bits, see below), a <em>message sequence number within a session</em> (32 bits), the <em>length</em> (of the message body in bytes; 32 bits), and a <em>body</em> (any size which is a multiple of 4 bytes). In addition, when a container or a single message is sent, an <em>internal header</em> is added at the top (see below), then the entire message is encrypted, and an <em>external header</em> is placed at the top of the message (a 64-bit <em>key identifier</em> and a 128-bit <em>message key</em>).</p>
<p>A <em>message body</em> normally consists of a 32-bit <em>message type</em> followed by type-dependent <em>parameters</em>. In particular, each RPC function has a corresponding message type. For more detail, see <a href="/mtproto/serialize">Binary Data Serialization</a>, <a href="/mtproto/service_messages">Mobile Protocol: Service Messages</a>.</p>
<p>All numbers are written as little endian. However, very large numbers (2048-bit) used in RSA and DH are written in the big endian format because that is how the OpenSSL library does it.</p>
<h4><a class="anchor" href="#authorization-and-encryption" id="authorization-and-encryption" name="authorization-and-encryption"><i class="anchor-icon"></i></a>Authorization and Encryption</h4>
<h4><a class="anchor" name="authorization-and-encryption" href="#authorization-and-encryption"><i class="anchor-icon"></i></a>Authorization and Encryption</h4>
<p>Prior to a message (or a multipart message) being transmitted over a network using a transport protocol, it is encrypted in a certain way, and an <em>external header</em> is added at the top of the message which is: a 64-bit <em>key identifier</em> (that uniquely identifies an <em>authorization key</em> for the server as well as the <em>user</em>) and a 128-bit <em>message key</em>. A user key together with the message key defines an actual 256-bit key which is what encrypts the message using AES-256 encryption. Note that the initial part of the message to be encrypted contains variable data (session, message ID, sequence number, server salt) that obviously influences the message key (and thus the AES key and iv). The message key is defined as the 128 middle bits of the SHA256 of the message body (including session, message ID, etc.), including the padding bytes, prepended by 32 bytes taken from the authorization key. Multipart messages are encrypted as a single message.</p>
<blockquote>
<p>For a technical specification, see <a href="/mtproto/description">Mobile Protocol: Detailed Description</a></p>
</blockquote>
<p>The first thing a client application must do is <a href="/mtproto/auth_key">create an authorization key</a> which is normally generated when it is first run and almost never changes.</p>
<p>The protocol's principal drawback is that an intruder passively intercepting messages and then somehow appropriating the authorization key (for example, by stealing a device) will be able to decrypt all the intercepted messages <em>post factum</em>. This probably is not too much of a problem (by stealing a device, one could also gain access to all the information cached on the device without decrypting anything); however, the following steps could be taken to overcome this weakness:</p>
<ul>
<li><em>Session keys</em> generated using the Diffie-Hellman protocol and used in conjunction with the authorization and the message keys to select AES parameters. To create these, the first thing a client must do after creating a new session is send a special RPC query to the server (“generate session key”) to which the server will respond, whereupon all subsequent messages within the session are encrypted using the session key as well.</li>
<li>Protecting the key stored on the client device with a (text) password; this password is never stored in memory and is entered by a user when starting the application or more frequently (depending on application settings).</li>
<li>Data stored (cached) on the user device can also be protected by encryption using an authorization key which, in turn, is to be password-protected. Then, a password will be required to gain access even to that data.</li>
</ul>
<h4><a class="anchor" href="#time-synchronization" id="time-synchronization" name="time-synchronization"><i class="anchor-icon"></i></a>Time Synchronization</h4>
<p>If client time diverges widely from server time, a server may start ignoring client messages, or vice versa, because of an invalid message identifier (which is closely related to creation time). Under these circumstances, the server will send the client a special message containing the correct time and a certain 128-bit salt (either explicitly provided by the client in a special RPC synchronization request or equal to the key of the latest message received from the client during the current session). This message could be the first one in a container that includes other messages (if the time discrepancy is significant but does not as yet result in the client's messages being ignored).</p>
<p>Having received such a message or a container holding it, the client first performs a time synchronization (in effect, simply storing the difference between the server's time and its own to be able to compute the “correct” time in the future) and then verifies that the message identifiers for correctness.</p>
<p>To prevent attackers potentially intercepting encrypted messages from decrypting them <em>post factum</em> by somehow appropriating the authorization key (for example, by stealing a device – even though in that case one could also gain access to all the information cached on the device without decrypting anything), MTProto supports <a href="https://core.telegram.org/api/pfs">Perfect Forward Secrecy</a> in both <a href="https://core.telegram.org/api/pfs">cloud chats</a> and <a href="https://core.telegram.org/api/end-to-end/pfs">secret chats</a>.</p>
<h4><a class="anchor" name="time-synchronization" href="#time-synchronization"><i class="anchor-icon"></i></a>Time Synchronization</h4>
<p>If client time diverges widely from server time, a server may start ignoring client messages, or vice versa, because of an invalid message identifier (which is closely related to creation time). Under these circumstances, the server will send the client a special message containing the correct time and a certain 128-bit salt (either explicitly provided by the client in a special RPC synchronization request or equal to the key of the latest message received from the client during the current session). This message could be the first one in a container that includes other messages (if the time discrepancy is significant but does not as yet result in the client&#39;s messages being ignored).</p>
<p>Having received such a message or a container holding it, the client first performs a time synchronization (in effect, simply storing the difference between the server&#39;s time and its own to be able to compute the “correct” time in the future) and then verifies that the message identifiers for correctness.</p>
<p>Where a correction has been neglected, the client will have to generate a new session to assure the monotonicity of message identifiers.</p>
<h3><a class="anchor" href="#mtproto-transport" id="mtproto-transport" name="mtproto-transport"><i class="anchor-icon"></i></a>MTProto transport</h3>
<h3><a class="anchor" name="mtproto-transport" href="#mtproto-transport"><i class="anchor-icon"></i></a>MTProto transport</h3>
<p>Before being sent using the selected transport protocol, the payload has to be wrapped in a secondary protocol header, defined by the appropriate MTProto transport protocol. </p>
<ul>
<li><a href="mtproto/mtproto-transports#abridged">Abridged</a></li>
@ -149,17 +123,15 @@ MTProto v1.0 (<a href="/mtproto_v1">described here</a> for reference) is depreca
<li><a href="mtproto/mtproto-transports#padded-intermediate">Padded intermediate</a></li>
<li><a href="mtproto/mtproto-transports#full">Full</a></li>
</ul>
<p>The server recognizes these different protocols (and distinguishes them from HTTP, too) by the header.
Additionally, the following transport features can be used: </p>
<p>The server recognizes these different protocols (and distinguishes them from HTTP, too) by the header.<br>Additionally, the following transport features can be used: </p>
<ul>
<li><a href="mtproto/mtproto-transports#quick-ack">Quick ack</a></li>
<li><a href="mtproto/mtproto-transports#transport-errors">Transport errors</a></li>
<li><a href="mtproto/mtproto-transports#transport-obfuscation">Transport obfuscation</a></li>
</ul>
<p>Example implementations for these protocols can be seen in <a href="https://github.com/tdlib/td/blob/master/td/mtproto/TcpTransport.cpp">tdlib</a> and <a href="https://github.com/danog/MadelineProto/tree/master/src/danog/MadelineProto/Stream/MTProtoTransport">MadelineProto</a>.</p>
<h3><a class="anchor" href="#transport" id="transport" name="transport"><i class="anchor-icon"></i></a>Transport</h3>
<p>Enables the delivery of encrypted containers together with the external header (hereinafter, <em>Payload</em>) from client to server and back.
Multiple transport protocols are defined:</p>
<h3><a class="anchor" name="transport" href="#transport"><i class="anchor-icon"></i></a>Transport</h3>
<p>Enables the delivery of encrypted containers together with the external header (hereinafter, <em>Payload</em>) from client to server and back.<br>Multiple transport protocols are defined:</p>
<ul>
<li><a href="/mtproto/transports#tcp">TCP</a></li>
<li><a href="/mtproto/transports#websocket">Websocket</a></li>
@ -169,7 +141,7 @@ Multiple transport protocols are defined:</p>
<li>UDP</li>
</ul>
<p>(We shall examine only the first five types.)</p>
<h3><a class="anchor" href="#recap" id="recap" name="recap"><i class="anchor-icon"></i></a>Recap</h3>
<h3><a class="anchor" name="recap" href="#recap"><i class="anchor-icon"></i></a>Recap</h3>
<p>To recap, using the <a href="https://en.wikipedia.org/wiki/OSI_model#Layer_architecture">ISO/OSI stack as comparison</a>: </p>
<ul>
<li>Layer 7 (Application): <a href="#high-level-component-rpc-query-languageapi">High-level RPC API</a></li>
@ -183,8 +155,9 @@ Multiple transport protocols are defined:</p>
</li>
<li>Layer 3 (Network): IP</li>
<li>Layer 2 (Data link): MAC/LLC</li>
<li>Layer 1 (Physical): IEEE 802.3, IEEE 802.11, etc...</li>
</ul></div>
<li>Layer 1 (Physical): IEEE 802.3, IEEE 802.11, etc…</li>
</ul>
</div>
</div>