python-telegram-bot-raw (#2324)

* POC

* Remove decorator dependency

* Rework setup.py & build, add separate readme

* Move utils -> ext.utils

* Move pytz dep to ext

* Try fixing timing stuff

* Add 'Typed' classifier

* Update README_RAW.rst

Co-authored-by: Harshil <37377066+harshil21@users.noreply.github.com>

* Some wording

* Deprecation warnings for moved tg.utils

* Tests for Promise

* Test time-helpers without pytz

* Try fixing time-helper tests

* Merge master

Co-authored-by: Harshil <37377066+harshil21@users.noreply.github.com>

.github/pull_request_template.md

@@ -27,3 +27,4 @@ Hey! You're PRing? Cool! Please have a look at the below checklist. It's here to
     - [ ] Added new handlers for new update types
     - [ ] Added new filters for new message (sub)types
     - [ ] Added or updated documentation for the changed class(es) and/or method(s)
+    - [ ] Updated the Bot API version number in all places in `README.rst` and `README_RAW.rst`, including the badge

.github/workflows/readme_notifier.yml

@@ -0,0 +1,16 @@
+name: Warning maintainers
+on:
+  pull_request:
+    paths:
+      - README.rst
+      - README_RAW.rst
+jobs:
+  job:
+    runs-on: ubuntu-latest
+    name: about readme change
+    steps:
+      - name: running the check
+        uses: Poolitzer/notifier-action@master
+        with:
+          notify-message: Hey! Looks like you edited README.rst or README_RAW.rst. I'm just a friendly reminder to apply relevant changes to both of those files :)
+          repo-token: ${{ secrets.GITHUB_TOKEN }}

MANIFEST.in

@@ -1 +1 @@
-include LICENSE LICENSE.lesser Makefile requirements.txt py.typed
+include LICENSE LICENSE.lesser Makefile requirements.txt README_RAW.rst telegram/py.typed

README.rst

@@ -1,3 +1,6 @@
+..
+    Make user to apply any changes to this file to README_RAW.rst as well!
+
 .. image:: https://github.com/python-telegram-bot/logos/blob/master/logo-text/png/ptb-logo-text_768.png?raw=true
    :align: center
    :target: https://python-telegram-bot.org
@@ -17,6 +20,10 @@ We have a vibrant community of developers helping each other in our `Telegram gr
    :target: https://pypi.org/project/python-telegram-bot/
    :alt: Supported Python versions
 
+.. image:: https://img.shields.io/badge/Bot%20API-5.0-blue?logo=telegram
+   :target: https://core.telegram.org/bots/api-changelog
+   :alt: Supported Bot API versions
+
 .. image:: https://img.shields.io/pypi/dm/python-telegram-bot
    :target: https://pypistats.org/packages/python-telegram-bot
    :alt: PyPi Package Monthly Download
@@ -48,7 +55,7 @@ We have a vibrant community of developers helping each other in our `Telegram gr
 .. image:: https://img.shields.io/badge/code%20style-black-000000.svg
     :target: https://github.com/psf/black
 
-.. image:: https://img.shields.io/badge/Telegram-Group-blue.svg
+.. image:: https://img.shields.io/badge/Telegram-Group-blue.svg?logo=telegram
    :target: https://telegram.me/pythontelegrambotgroup
    :alt: Telegram Group
 
@@ -92,6 +99,14 @@ In addition to the pure API implementation, this library features a number of hi
 make the development of bots easy and straightforward. These classes are contained in the
 ``telegram.ext`` submodule.
 
+A pure API implementation *without* ``telegram.ext`` is available as the standalone package ``python-telegram-bot-raw``.  `See here for details. <https://github.com/python-telegram-bot/python-telegram-bot/blob/master/README_RAW.rst>`_
+
+----
+Note
+----
+
+Installing both ``python-telegram-bot`` and ``python-telegram-bot-raw`` in conjunction will result in undesired side-effects, so only install *one* of both.
+
 ====================
 Telegram API support
 ====================
@@ -199,7 +214,6 @@ You can get help in several ways:
 5. You can even ask for help on Stack Overflow using the `python-telegram-bot tag <https://stackoverflow.com/questions/tagged/python-telegram-bot>`_.
 
 
-
 ============
 Contributing
 ============

README_RAW.rst

@@ -0,0 +1,210 @@
+..
+    Make user to apply any changes to this file to README.rst as well!
+
+.. image:: https://github.com/python-telegram-bot/logos/blob/master/logo-text/png/ptb-raw-logo-text_768.png?raw=true
+   :align: center
+   :target: https://python-telegram-bot.org
+   :alt: python-telegram-bot-raw Logo
+
+We have made you a wrapper you can't refuse
+
+We have a vibrant community of developers helping each other in our `Telegram group <https://telegram.me/pythontelegrambotgroup>`_. Join us!
+
+*Stay tuned for library updates and new releases on our* `Telegram Channel <https://telegram.me/pythontelegrambotchannel>`_.
+
+.. image:: https://img.shields.io/pypi/v/python-telegram-bot-raw.svg
+   :target: https://pypi.org/project/python-telegram-bot/
+   :alt: PyPi Package Version
+
+.. image:: https://img.shields.io/pypi/pyversions/python-telegram-bot-raw.svg
+   :target: https://pypi.org/project/python-telegram-bot/
+   :alt: Supported Python versions
+
+.. image:: https://img.shields.io/badge/Bot%20API-5.0-blue?logo=telegram
+   :target: https://core.telegram.org/bots/api-changelog
+   :alt: Supported Bot API versions
+
+.. image:: https://img.shields.io/pypi/dm/python-telegram-bot-raw
+   :target: https://pypistats.org/packages/python-telegram-bot
+   :alt: PyPi Package Monthly Download
+
+.. image:: https://img.shields.io/badge/docs-latest-af1a97.svg
+   :target: https://python-telegram-bot.readthedocs.io/
+   :alt: Documentation Status
+
+.. image:: https://img.shields.io/pypi/l/python-telegram-bot-raw.svg
+   :target: https://www.gnu.org/licenses/lgpl-3.0.html
+   :alt: LGPLv3 License
+
+.. image:: https://github.com/python-telegram-bot/python-telegram-bot/workflows/GitHub%20Actions/badge.svg
+   :target: https://github.com/python-telegram-bot/python-telegram-bot/
+   :alt: Github Actions workflow
+
+.. image:: https://codecov.io/gh/python-telegram-bot/python-telegram-bot/branch/master/graph/badge.svg
+   :target: https://codecov.io/gh/python-telegram-bot/python-telegram-bot
+   :alt: Code coverage
+
+.. image:: http://isitmaintained.com/badge/resolution/python-telegram-bot/python-telegram-bot.svg
+   :target: http://isitmaintained.com/project/python-telegram-bot/python-telegram-bot
+   :alt: Median time to resolve an issue
+
+.. image:: https://api.codacy.com/project/badge/Grade/99d901eaa09b44b4819aec05c330c968
+   :target: https://www.codacy.com/app/python-telegram-bot/python-telegram-bot?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=python-telegram-bot/python-telegram-bot&amp;utm_campaign=Badge_Grade
+   :alt: Code quality
+
+.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
+    :target: https://github.com/psf/black
+
+.. image:: https://img.shields.io/badge/Telegram-Group-blue.svg?logo=telegram
+   :target: https://telegram.me/pythontelegrambotgroup
+   :alt: Telegram Group
+
+.. image:: https://img.shields.io/badge/IRC-Channel-blue.svg
+   :target: https://webchat.freenode.net/?channels=##python-telegram-bot
+   :alt: IRC Bridge
+
+=================
+Table of contents
+=================
+
+- `Introduction`_
+
+- `Telegram API support`_
+
+- `Installing`_
+
+- `Getting started`_
+
+  #. `Logging`_
+
+  #. `Documentation`_
+
+- `Getting help`_
+
+- `Contributing`_
+
+- `License`_
+
+============
+Introduction
+============
+
+This library provides a pure Python, lightweight interface for the
+`Telegram Bot API <https://core.telegram.org/bots/api>`_.
+It's compatible with Python versions 3.6+. PTB-Raw might also work on `PyPy <http://pypy.org/>`_, though there have been a lot of issues before. Hence, PyPy is not officially supported.
+
+``python-telegram-bot-raw`` is part of the `python-telegram-bot <https://python-telegram-bot.org>`_ ecosystem and provides the pure API functionality extracted from PTB. It therefore does *not* have independent release schedules, changelogs or documentation. Please consult the PTB resources.
+
+----
+Note
+----
+
+Installing both ``python-telegram-bot`` and ``python-telegram-bot-raw`` in conjunction will result in undesired side-effects, so only install *one* of both.
+
+====================
+Telegram API support
+====================
+
+All types and methods of the Telegram Bot API **5.0** are supported.
+
+==========
+Installing
+==========
+
+You can install or upgrade python-telegram-bot-raw with:
+
+.. code:: shell
+
+    $ pip install python-telegram-bot-raw --upgrade
+
+Or you can install from source with:
+
+.. code:: shell
+
+    $ git clone https://github.com/python-telegram-bot/python-telegram-bot --recursive
+    $ cd python-telegram-bot
+    $ python setup-raw.py install
+
+In case you have a previously cloned local repository already, you should initialize the added urllib3 submodule before installing with:
+
+.. code:: shell
+
+    $ git submodule update --init --recursive
+
+----
+Note
+----
+
+Installing the `.tar.gz` archive available on PyPi directly via `pip` will *not* work as expected, as `pip` does not recognize that it should use `setup-raw.py` instead of `setup.py`.
+
+===============
+Getting started
+===============
+
+Our Wiki contains an `Introduction to the API <https://github.com/python-telegram-bot/python-telegram-bot/wiki/Introduction-to-the-API>`_. Other references are:
+
+- the `Telegram API documentation <https://core.telegram.org/bots/api>`_
+- the `python-telegram-bot documentation <https://python-telegram-bot.readthedocs.io/>`_
+
+-------
+Logging
+-------
+
+This library uses the ``logging`` module. To set up logging to standard output, put:
+
+.. code:: python
+
+    import logging
+    logging.basicConfig(level=logging.DEBUG,
+                        format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+
+at the beginning of your script.
+
+You can also use logs in your application by calling ``logging.getLogger()`` and setting the log level you want:
+
+.. code:: python
+
+    logger = logging.getLogger()
+    logger.setLevel(logging.INFO)
+
+If you want DEBUG logs instead:
+
+.. code:: python
+
+    logger.setLevel(logging.DEBUG)
+
+
+=============
+Documentation
+=============
+
+``python-telegram-bot``'s documentation lives at `readthedocs.io <https://python-telegram-bot.readthedocs.io/>`_, which
+includes the relevant documentation for ``python-telegram-bot-raw``.
+
+============
+Getting help
+============
+
+You can get help in several ways:
+
+1. We have a vibrant community of developers helping each other in our `Telegram group <https://telegram.me/pythontelegrambotgroup>`_. Join us!
+
+2. In case you are unable to join our group due to Telegram restrictions, you can use our `IRC channel <https://webchat.freenode.net/?channels=##python-telegram-bot>`_.
+
+3. Report bugs, request new features or ask questions by `creating an issue <https://github.com/python-telegram-bot/python-telegram-bot/issues/new/choose>`_ or `a discussion <https://github.com/python-telegram-bot/python-telegram-bot/discussions/new>`_.
+
+4. Our `Wiki pages <https://github.com/python-telegram-bot/python-telegram-bot/wiki/>`_ offer a growing amount of resources.
+
+5. You can even ask for help on Stack Overflow using the `python-telegram-bot tag <https://stackoverflow.com/questions/tagged/python-telegram-bot>`_.
+
+============
+Contributing
+============
+
+Contributions of all sizes are welcome. Please review our `contribution guidelines <https://github.com/python-telegram-bot/python-telegram-bot/blob/master/.github/CONTRIBUTING.rst>`_ to get started. You can also help by `reporting bugs <https://github.com/python-telegram-bot/python-telegram-bot/issues/new>`_.
+
+=======
+License
+=======
+
+You may copy, distribute and modify the software provided that modifications are described and licensed for free under `LGPL-3 <https://www.gnu.org/licenses/lgpl-3.0.html>`_. Derivatives works (including modifications or anything statically linked to the library) can only be redistributed under LGPL-3, but applications that use the library don't have to be.

docs/source/telegram.ext.rst

@@ -6,13 +6,12 @@ telegram.ext package
     telegram.ext.updater
     telegram.ext.dispatcher
     telegram.ext.dispatcherhandlerstop
-    telegram.ext.filters
+    telegram.ext.callbackcontext
+    telegram.ext.defaults
     telegram.ext.job
     telegram.ext.jobqueue
     telegram.ext.messagequeue
     telegram.ext.delayqueue
-    telegram.ext.callbackcontext
-    telegram.ext.defaults
 
 Handlers
 --------
@@ -22,10 +21,11 @@ Handlers
     telegram.ext.handler
     telegram.ext.callbackqueryhandler
     telegram.ext.choseninlineresulthandler
-    telegram.ext.conversationhandler
     telegram.ext.commandhandler
+    telegram.ext.conversationhandler
     telegram.ext.inlinequeryhandler
     telegram.ext.messagehandler
+    telegram.ext.filters
     telegram.ext.pollanswerhandler
     telegram.ext.pollhandler
     telegram.ext.precheckoutqueryhandler
@@ -44,3 +44,10 @@ Persistence
     telegram.ext.basepersistence
     telegram.ext.picklepersistence
     telegram.ext.dictpersistence
+
+utils
+-----
+
+.. toctree::
+
+    telegram.ext.utils.promise

docs/source/telegram.ext.utils.promise.rst

@@ -0,0 +1,6 @@
+telegram.ext.utils.promise.Promise
+==================================
+
+.. autoclass:: telegram.ext.utils.promise.Promise
+    :members:
+    :show-inheritance:

docs/source/telegram.utils.promise.rst

@@ -1,6 +1,9 @@
 telegram.utils.promise.Promise
 ==============================
 
-.. autoclass:: telegram.utils.promise.Promise
+.. py:class:: telegram.utils.promise.Promise
-    :members:
+
-    :show-inheritance:
+   Shortcut for :class:`telegram.ext.utils.promise.Promise`.
+
+   .. deprecated:: 13.2
+      Use :class:`telegram.ext.utils.promise.Promise` instead.

requirements.txt

@@ -1,6 +1,6 @@
 certifi
-tornado>=5.1
 cryptography
-decorator>=4.4.0
+# only telegram.ext: # Keep this line here; used in setup(-raw).py
+tornado>=5.1
 APScheduler==3.6.3
 pytz>=2018.6

setup-raw.py

@@ -0,0 +1,7 @@
+#!/usr/bin/env python
+"""The setup and build script for the python-telegram-bot-raw library."""
+
+from setuptools import setup
+from setup import get_setup_kwargs
+
+setup(**get_setup_kwargs(raw=True))

setup.cfg

@@ -13,7 +13,7 @@ upload-dir = docs/build/html
 max-line-length = 99
 ignore = W503, W605
 extend-ignore = E203
-exclude = setup.py, docs/source/conf.py, telegram/vendor
+exclude = setup.py, setup-raw.py docs/source/conf.py, telegram/vendor
 
 [pylint]
 ignore=vendor

setup.py

@@ -3,49 +3,84 @@
 
 import codecs
 import os
+import subprocess
 import sys
 
 from setuptools import setup, find_packages
 
+UPSTREAM_URLLIB3_FLAG = '--with-upstream-urllib3'
 
-def requirements():
+
+def get_requirements(raw=False):
     """Build the requirements list for this project"""
     requirements_list = []
 
-    with open('requirements.txt') as requirements:
+    with open('requirements.txt') as reqs:
-        for install in requirements:
+        for install in reqs:
+            if install.startswith('# only telegram.ext:'):
+                if raw:
+                    break
+                continue
             requirements_list.append(install.strip())
 
     return requirements_list
 
 
-packages = find_packages(exclude=['tests*'])
+def get_packages_requirements(raw=False):
-requirements = requirements()
+    """Build the package & requirements list for this project"""
+    reqs = get_requirements(raw=raw)
 
+    exclude = ['tests*']
+    if raw:
+        exclude.append('telegram.ext*')
+
+    packs = find_packages(exclude=exclude)
     # Allow for a package install to not use the vendored urllib3
-UPSTREAM_URLLIB3_FLAG = '--with-upstream-urllib3'
     if UPSTREAM_URLLIB3_FLAG in sys.argv:
         sys.argv.remove(UPSTREAM_URLLIB3_FLAG)
-    requirements.append('urllib3 >= 1.19.1')
+        reqs.append('urllib3 >= 1.19.1')
-    packages = [x for x in packages if not x.startswith('telegram.vendor.ptb_urllib3')]
+        packs = [x for x in packs if not x.startswith('telegram.vendor.ptb_urllib3')]
+
+    return packs, reqs
+
+
+def get_setup_kwargs(raw=False):
+    """Builds a dictionary of kwargs for the setup function"""
+    packages, requirements = get_packages_requirements(raw=raw)
+
+    raw_ext = "-raw" if raw else ""
+    readme = f'README{"_RAW" if raw else ""}.rst'
 
-with codecs.open('README.rst', 'r', 'utf-8') as fd:
     fn = os.path.join('telegram', 'version.py')
     with open(fn) as fh:
         code = compile(fh.read(), fn, 'exec')
         exec(code)
 
-    setup(name='python-telegram-bot',
+    with open(readme, 'r', encoding='utf-8') as fd:
-          version=__version__,
+
+        kwargs = dict(
+            script_name=f'setup{raw_ext}.py',
+            name=f'python-telegram-bot{raw_ext}',
+            version=locals()['__version__'],
             author='Leandro Toledo',
             author_email='devs@python-telegram-bot.org',
             license='LGPLv3',
             url='https://python-telegram-bot.org/',
+            # Keywords supported by PyPI can be found at https://git.io/JtLIZ
+            project_urls={
+                "Documentation": "https://python-telegram-bot.readthedocs.io",
+                "Bug Tracker": "https://github.com/python-telegram-bot/python-telegram-bot/issues",
+                "Source Code": "https://github.com/python-telegram-bot/python-telegram-bot",
+                "News": "https://t.me/pythontelegrambotchannel",
+                "Changelog": "https://python-telegram-bot.readthedocs.io/en/stable/changelog.html",
+            },
+            download_url=f'https://pypi.org/project/python-telegram-bot{raw_ext}/',
             keywords='python telegram bot api wrapper',
             description="We have made you a wrapper you can't refuse",
             long_description=fd.read(),
+            long_description_content_type='text/x-rst',
             packages=packages,
-          package_data={'telegram': ['py.typed']},
+
             install_requires=requirements,
             extras_require={
                 'json': 'ujson',
@@ -66,4 +101,23 @@ with codecs.open('README.rst', 'r', 'utf-8') as fd:
                 'Programming Language :: Python :: 3.7',
                 'Programming Language :: Python :: 3.8',
                 'Programming Language :: Python :: 3.9',
-          ],)
+                'Typing:: Typed',
+            ],
+            python_requires='>=3.6'
+        )
+
+    return kwargs
+
+
+def main():
+    # If we're building, build ptb-raw as well
+    if set(sys.argv[1:]) in [{'bdist_wheel'}, {'sdist'}, {'sdist', 'bdist_wheel'}]:
+        args = ['python', 'setup-raw.py']
+        args.extend(sys.argv[1:])
+        subprocess.run(args, check=True, capture_output=True)
+
+    setup(**get_setup_kwargs(raw=False))
+
+
+if __name__ == '__main__':
+    main()

telegram/bot.py

@@ -36,8 +36,6 @@ from typing import (
     no_type_check,
 )
 
-from decorator import decorate
-
 try:
     import ujson as json
 except ImportError:
@@ -113,14 +111,15 @@ def log(
 ) -> Callable[..., RT]:
     logger = logging.getLogger(func.__module__)
 
-    def decorator(self: 'Bot', *args: object, **kwargs: object) -> RT:  # pylint: disable=W0613
+    @functools.wraps(func)
+    def decorator(*args: object, **kwargs: object) -> RT:  # pylint: disable=W0613
         logger.debug('Entering: %s', func.__name__)
         result = func(*args, **kwargs)
         logger.debug(result)
         logger.debug('Exiting: %s', func.__name__)
         return result
 
-    return decorate(func, decorator)
+    return decorator
 
 
 class Bot(TelegramObject):
@@ -162,12 +161,16 @@ class Bot(TelegramObject):
         # For each method ...
         for method_name, method in inspect.getmembers(instance, predicate=inspect.ismethod):
             # ... get kwargs
-            argspec = inspect.getfullargspec(method)
+            signature = inspect.signature(method, follow_wrapped=True)
-            kwarg_names = argspec.args[-len(argspec.defaults or []) :]
+            kwarg_names = (
+                p.name
+                for p in signature.parameters.values()
+                if p.default != inspect.Signature.empty
+            )
             # ... check if Defaults has a attribute that matches the kwarg name
-            needs_default = [
+            needs_default = (
                 kwarg_name for kwarg_name in kwarg_names if hasattr(defaults, kwarg_name)
-            ]
+            )
             # ... make a dict of kwarg name and the default value
             default_kwargs = {
                 kwarg_name: getattr(defaults, kwarg_name)

telegram/ext/conversationhandler.py

@@ -34,7 +34,7 @@ from telegram.ext import (
     Handler,
     InlineQueryHandler,
 )
-from telegram.utils.promise import Promise
+from telegram.ext.utils.promise import Promise
 from telegram.utils.types import ConversationDict
 
 if TYPE_CHECKING:

telegram/ext/dispatcher.py

@@ -34,7 +34,7 @@ from telegram.ext import BasePersistence
 from telegram.ext.callbackcontext import CallbackContext
 from telegram.ext.handler import Handler
 from telegram.utils.deprecate import TelegramDeprecationWarning
-from telegram.utils.promise import Promise
+from telegram.ext.utils.promise import Promise
 from telegram.utils.helpers import DefaultValue, DEFAULT_FALSE
 
 if TYPE_CHECKING:

telegram/ext/handler.py

@@ -22,7 +22,7 @@ from abc import ABC, abstractmethod
 from typing import TYPE_CHECKING, Any, Callable, Dict, Optional, TypeVar, Union, Generic
 
 from telegram import Update
-from telegram.utils.promise import Promise
+from telegram.ext.utils.promise import Promise
 from telegram.utils.helpers import DefaultValue, DEFAULT_FALSE
 
 if TYPE_CHECKING:

telegram/ext/messagequeue.py

@@ -26,7 +26,7 @@ import threading
 import time
 from typing import TYPE_CHECKING, Callable, List, NoReturn
 
-from telegram.utils.promise import Promise
+from telegram.ext.utils.promise import Promise
 
 if TYPE_CHECKING:
     from telegram import Bot

telegram/ext/updater.py

@@ -33,7 +33,7 @@ from telegram.ext import Dispatcher, JobQueue
 from telegram.utils.deprecate import TelegramDeprecationWarning
 from telegram.utils.helpers import get_signal_name
 from telegram.utils.request import Request
-from telegram.utils.webhookhandler import WebhookAppClass, WebhookServer
+from telegram.ext.utils.webhookhandler import WebhookAppClass, WebhookServer
 
 if TYPE_CHECKING:
     from telegram.ext import BasePersistence, Defaults

telegram/ext/utils/__init__.py

@@ -0,0 +1,17 @@
+#
+#  A library that provides a Python interface to the Telegram Bot API
+#  Copyright (C) 2015-2021
+#  Leandro Toledo de Souza <devs@python-telegram-bot.org>
+#
+#  This program is free software: you can redistribute it and/or modify
+#  it under the terms of the GNU Lesser Public License as published by
+#  the Free Software Foundation, either version 3 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU Lesser Public License for more details.
+#
+#  You should have received a copy of the GNU Lesser Public License
+#  along with this program.  If not, see [http://www.gnu.org/licenses/].

telegram/ext/utils/promise.py

@@ -0,0 +1,113 @@
+#!/usr/bin/env python
+#
+# A library that provides a Python interface to the Telegram Bot API
+# Copyright (C) 2015-2021
+# Leandro Toledo de Souza <devs@python-telegram-bot.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser Public License for more details.
+#
+# You should have received a copy of the GNU Lesser Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/].
+"""This module contains the Promise class."""
+
+import logging
+from threading import Event
+from typing import Callable, List, Optional, Tuple, TypeVar, Union
+
+from telegram.utils.types import JSONDict
+
+RT = TypeVar('RT')
+
+
+logger = logging.getLogger(__name__)
+
+
+class Promise:
+    """A simple Promise implementation for use with the run_async decorator, DelayQueue etc.
+
+    Args:
+        pooled_function (:obj:`callable`): The callable that will be called concurrently.
+        args (:obj:`list` | :obj:`tuple`): Positional arguments for :attr:`pooled_function`.
+        kwargs (:obj:`dict`): Keyword arguments for :attr:`pooled_function`.
+        update (:class:`telegram.Update` | :obj:`object`, optional): The update this promise is
+            associated with.
+        error_handling (:obj:`bool`, optional): Whether exceptions raised by :attr:`func`
+            may be handled by error handlers. Defaults to :obj:`True`.
+
+    Attributes:
+        pooled_function (:obj:`callable`): The callable that will be called concurrently.
+        args (:obj:`list` | :obj:`tuple`): Positional arguments for :attr:`pooled_function`.
+        kwargs (:obj:`dict`): Keyword arguments for :attr:`pooled_function`.
+        done (:obj:`threading.Event`): Is set when the result is available.
+        update (:class:`telegram.Update` | :obj:`object`): Optional. The update this promise is
+            associated with.
+        error_handling (:obj:`bool`): Optional. Whether exceptions raised by :attr:`func`
+            may be handled by error handlers. Defaults to :obj:`True`.
+
+    """
+
+    # TODO: Remove error_handling parameter once we drop the @run_async decorator
+    def __init__(
+        self,
+        pooled_function: Callable[..., RT],
+        args: Union[List, Tuple],
+        kwargs: JSONDict,
+        update: object = None,
+        error_handling: bool = True,
+    ):
+        self.pooled_function = pooled_function
+        self.args = args
+        self.kwargs = kwargs
+        self.update = update
+        self.error_handling = error_handling
+        self.done = Event()
+        self._result: Optional[RT] = None
+        self._exception: Optional[Exception] = None
+
+    def run(self) -> None:
+        """Calls the :attr:`pooled_function` callable."""
+
+        try:
+            self._result = self.pooled_function(*self.args, **self.kwargs)
+
+        except Exception as exc:
+            self._exception = exc
+
+        finally:
+            self.done.set()
+
+    def __call__(self) -> None:
+        self.run()
+
+    def result(self, timeout: float = None) -> Optional[RT]:
+        """Return the result of the ``Promise``.
+
+        Args:
+            timeout (:obj:`float`, optional): Maximum time in seconds to wait for the result to be
+                calculated. ``None`` means indefinite. Default is ``None``.
+
+        Returns:
+            Returns the return value of :attr:`pooled_function` or ``None`` if the ``timeout``
+            expires.
+
+        Raises:
+            object exception raised by :attr:`pooled_function`.
+        """
+        self.done.wait(timeout=timeout)
+        if self._exception is not None:
+            raise self._exception  # pylint: disable=raising-bad-type
+        return self._result
+
+    @property
+    def exception(self) -> Optional[Exception]:
+        """The exception raised by :attr:`pooled_function` or ``None`` if no exception has been
+        raised (yet)."""
+        return self._exception

telegram/ext/utils/webhookhandler.py

@@ -0,0 +1,208 @@
+#!/usr/bin/env python
+#
+# A library that provides a Python interface to the Telegram Bot API
+# Copyright (C) 2015-2021
+# Leandro Toledo de Souza <devs@python-telegram-bot.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser Public License for more details.
+#
+# You should have received a copy of the GNU Lesser Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/].
+# pylint: disable=E0401, C0114
+
+import asyncio
+import logging
+import os
+import sys
+from queue import Queue
+from ssl import SSLContext
+from threading import Event, Lock
+from typing import TYPE_CHECKING, Any, Optional
+
+import tornado.web
+from tornado import httputil
+from tornado.httpserver import HTTPServer
+from tornado.ioloop import IOLoop
+
+from telegram import Update
+from telegram.utils.types import JSONDict
+
+if TYPE_CHECKING:
+    from telegram import Bot
+
+try:
+    import ujson as json
+except ImportError:
+    import json  # type: ignore[no-redef]
+
+
+class WebhookServer:
+    def __init__(
+        self, listen: str, port: int, webhook_app: 'WebhookAppClass', ssl_ctx: SSLContext
+    ):
+        self.http_server = HTTPServer(webhook_app, ssl_options=ssl_ctx)
+        self.listen = listen
+        self.port = port
+        self.loop: Optional[IOLoop] = None
+        self.logger = logging.getLogger(__name__)
+        self.is_running = False
+        self.server_lock = Lock()
+        self.shutdown_lock = Lock()
+
+    def serve_forever(self, force_event_loop: bool = False, ready: Event = None) -> None:
+        with self.server_lock:
+            self.is_running = True
+            self.logger.debug('Webhook Server started.')
+            self._ensure_event_loop(force_event_loop=force_event_loop)
+            self.loop = IOLoop.current()
+            self.http_server.listen(self.port, address=self.listen)
+
+            if ready is not None:
+                ready.set()
+
+            self.loop.start()
+            self.logger.debug('Webhook Server stopped.')
+            self.is_running = False
+
+    def shutdown(self) -> None:
+        with self.shutdown_lock:
+            if not self.is_running:
+                self.logger.warning('Webhook Server already stopped.')
+                return
+            self.loop.add_callback(self.loop.stop)  # type: ignore
+
+    def handle_error(self, request: object, client_address: str) -> None:  # pylint: disable=W0613
+        """Handle an error gracefully."""
+        self.logger.debug(
+            'Exception happened during processing of request from %s',
+            client_address,
+            exc_info=True,
+        )
+
+    def _ensure_event_loop(self, force_event_loop: bool = False) -> None:
+        """If there's no asyncio event loop set for the current thread - create one."""
+        try:
+            loop = asyncio.get_event_loop()
+            if (
+                not force_event_loop
+                and os.name == 'nt'
+                and sys.version_info >= (3, 8)
+                and isinstance(loop, asyncio.ProactorEventLoop)
+            ):
+                raise TypeError(
+                    '`ProactorEventLoop` is incompatible with '
+                    'Tornado. Please switch to `SelectorEventLoop`.'
+                )
+        except RuntimeError:
+            # Python 3.8 changed default asyncio event loop implementation on windows
+            # from SelectorEventLoop to ProactorEventLoop. At the time of this writing
+            # Tornado doesn't support ProactorEventLoop and suggests that end users
+            # change asyncio event loop policy to WindowsSelectorEventLoopPolicy.
+            # https://github.com/tornadoweb/tornado/issues/2608
+            # To avoid changing the global event loop policy, we manually construct
+            # a SelectorEventLoop instance instead of using asyncio.new_event_loop().
+            # Note that the fix is not applied in the main thread, as that can break
+            # user code in even more ways than changing the global event loop policy can,
+            # and because Updater always starts its webhook server in a separate thread.
+            # Ideally, we would want to check that Tornado actually raises the expected
+            # NotImplementedError, but it's not possible to cleanly recover from that
+            # exception in current Tornado version.
+            if (
+                os.name == 'nt'
+                and sys.version_info >= (3, 8)
+                # OS+version check makes hasattr check redundant, but just to be sure
+                and hasattr(asyncio, 'WindowsProactorEventLoopPolicy')
+                and (
+                    isinstance(
+                        asyncio.get_event_loop_policy(),
+                        asyncio.WindowsProactorEventLoopPolicy,  # pylint: disable=E1101
+                    )
+                )
+            ):  # pylint: disable=E1101
+                self.logger.debug(
+                    'Applying Tornado asyncio event loop fix for Python 3.8+ on Windows'
+                )
+                loop = asyncio.SelectorEventLoop()
+            else:
+                loop = asyncio.new_event_loop()
+            asyncio.set_event_loop(loop)
+
+
+class WebhookAppClass(tornado.web.Application):
+    def __init__(self, webhook_path: str, bot: 'Bot', update_queue: Queue):
+        self.shared_objects = {"bot": bot, "update_queue": update_queue}
+        handlers = [(rf"{webhook_path}/?", WebhookHandler, self.shared_objects)]  # noqa
+        tornado.web.Application.__init__(self, handlers)
+
+    def log_request(self, handler: tornado.web.RequestHandler) -> None:
+        pass
+
+
+# WebhookHandler, process webhook calls
+# pylint: disable=W0223
+class WebhookHandler(tornado.web.RequestHandler):
+    SUPPORTED_METHODS = ["POST"]
+
+    def __init__(
+        self,
+        application: tornado.web.Application,
+        request: httputil.HTTPServerRequest,
+        **kwargs: JSONDict,
+    ):
+        super().__init__(application, request, **kwargs)
+        self.logger = logging.getLogger(__name__)
+
+    def initialize(self, bot: 'Bot', update_queue: Queue) -> None:
+        # pylint: disable=W0201
+        self.bot = bot
+        self.update_queue = update_queue
+
+    def set_default_headers(self) -> None:
+        self.set_header("Content-Type", 'application/json; charset="utf-8"')
+
+    def post(self) -> None:
+        self.logger.debug('Webhook triggered')
+        self._validate_post()
+        json_string = self.request.body.decode()
+        data = json.loads(json_string)
+        self.set_status(200)
+        self.logger.debug('Webhook received data: %s', json_string)
+        update = Update.de_json(data, self.bot)
+        if update:
+            self.logger.debug('Received Update with ID %d on Webhook', update.update_id)
+            self.update_queue.put(update)
+
+    def _validate_post(self) -> None:
+        ct_header = self.request.headers.get("Content-Type", None)
+        if ct_header != 'application/json':
+            raise tornado.web.HTTPError(403)
+
+    def write_error(self, status_code: int, **kwargs: Any) -> None:
+        """Log an arbitrary message.
+
+        This is used by all other logging functions.
+
+        It overrides ``BaseHTTPRequestHandler.log_message``, which logs to ``sys.stderr``.
+
+        The first argument, FORMAT, is a format string for the message to be logged.  If the format
+        string contains any % escapes requiring parameters, they should be specified as subsequent
+        arguments (it's just like printf!).
+
+        The client ip is prefixed to every message.
+
+        """
+        super().write_error(status_code, **kwargs)
+        self.logger.debug(
+            "%s - - %s",
+            self.request.remote_ip,
+            "Exception in WebhookHandler",
+            exc_info=kwargs['exc_info'],
+        )

telegram/utils/helpers.py

@@ -25,7 +25,6 @@ import time
 
 from collections import defaultdict
 from html import escape
-from numbers import Number
 from pathlib import Path
 
 from typing import (
@@ -41,13 +40,20 @@ from typing import (
     IO,
 )
 
-import pytz  # pylint: disable=E0401
-
 from telegram.utils.types import JSONDict, FileInput
 
 if TYPE_CHECKING:
     from telegram import Message, Update, TelegramObject, InputFile
 
+# in PTB-Raw we don't have pytz, so we make a little workaround here
+DTM_UTC = dtm.timezone.utc
+try:
+    import pytz  # pylint: disable=E0401
+
+    UTC = pytz.utc
+except ImportError:
+    UTC = DTM_UTC  # type: ignore[assignment]
+
 try:
     import ujson as json
 except ImportError:
@@ -176,10 +182,19 @@ def _datetime_to_float_timestamp(dt_obj: dtm.datetime) -> float:
     return dt_obj.timestamp()
 
 
+def _localize(datetime: dtm.datetime, tzinfo: dtm.tzinfo) -> dtm.datetime:
+    """
+    Localize the datetime, where UTC is handled depending on whether pytz is available or not
+    """
+    if tzinfo is DTM_UTC:
+        return datetime.replace(tzinfo=DTM_UTC)
+    return tzinfo.localize(datetime)  # type: ignore[attr-defined]
+
+
 def to_float_timestamp(
     time_object: Union[int, float, dtm.timedelta, dtm.datetime, dtm.time],
     reference_timestamp: float = None,
-    tzinfo: pytz.BaseTzInfo = None,
+    tzinfo: dtm.tzinfo = None,
 ) -> float:
     """
     Converts a given time object to a float POSIX timestamp.
@@ -206,10 +221,14 @@ def to_float_timestamp(
             If ``t`` is given as an absolute representation of date & time (i.e. a
             ``datetime.datetime`` object), ``reference_timestamp`` is not relevant and so its
             value should be :obj:`None`. If this is not the case, a ``ValueError`` will be raised.
-        tzinfo (:obj:`datetime.tzinfo`, optional): If ``t`` is a naive object from the
+        tzinfo (:obj:`pytz.BaseTzInfo`, optional): If ``t`` is a naive object from the
             :class:`datetime` module, it will be interpreted as this timezone. Defaults to
             ``pytz.utc``.
 
+            Note:
+                Only to be used by ``telegram.ext``.
+
+
     Returns:
         (float | None) The return value depends on the type of argument ``t``. If ``t`` is
             given as a time increment (i.e. as a obj:`int`, :obj:`float` or
@@ -236,7 +255,7 @@ def to_float_timestamp(
         return reference_timestamp + time_object
 
     if tzinfo is None:
-        tzinfo = pytz.utc
+        tzinfo = UTC
 
     if isinstance(time_object, dtm.time):
         reference_dt = dtm.datetime.fromtimestamp(
@@ -247,7 +266,7 @@ def to_float_timestamp(
 
         aware_datetime = dtm.datetime.combine(reference_date, time_object)
         if aware_datetime.tzinfo is None:
-            aware_datetime = tzinfo.localize(aware_datetime)
+            aware_datetime = _localize(aware_datetime, tzinfo)
 
         # if the time of day has passed today, use tomorrow
         if reference_time > aware_datetime.timetz():
@@ -255,10 +274,8 @@ def to_float_timestamp(
         return _datetime_to_float_timestamp(aware_datetime)
     if isinstance(time_object, dtm.datetime):
         if time_object.tzinfo is None:
-            time_object = tzinfo.localize(time_object)
+            time_object = _localize(time_object, tzinfo)
         return _datetime_to_float_timestamp(time_object)
-    if isinstance(time_object, Number):
-        return reference_timestamp + time_object
 
     raise TypeError(f'Unable to convert {type(time_object).__name__} object to timestamp')
 
@@ -266,7 +283,7 @@ def to_float_timestamp(
 def to_timestamp(
     dt_obj: Union[int, float, dtm.timedelta, dtm.datetime, dtm.time, None],
     reference_timestamp: float = None,
-    tzinfo: pytz.BaseTzInfo = None,
+    tzinfo: dtm.tzinfo = None,
 ) -> Optional[int]:
     """
     Wrapper over :func:`to_float_timestamp` which returns an integer (the float value truncated
@@ -281,9 +298,7 @@ def to_timestamp(
     )
 
 
-def from_timestamp(
+def from_timestamp(unixtime: Optional[int], tzinfo: dtm.tzinfo = UTC) -> Optional[dtm.datetime]:
-    unixtime: Optional[int], tzinfo: dtm.tzinfo = pytz.utc
-) -> Optional[dtm.datetime]:
     """
     Converts an (integer) unix timestamp to a timezone aware datetime object.
     :obj:`None`s are left alone (i.e. ``from_timestamp(None)`` is :obj:`None`).

telegram/utils/promise.py

@@ -16,98 +16,22 @@
 #
 # You should have received a copy of the GNU Lesser Public License
 # along with this program.  If not, see [http://www.gnu.org/licenses/].
-"""This module contains the Promise class."""
+"""This module contains the :class:`telegram.ext.utils.promise.Promise` class for backwards
+compatibility."""
+import warnings
 
-import logging
+import telegram.ext.utils.promise as promise
-from threading import Event
+from telegram.utils.deprecate import TelegramDeprecationWarning
-from typing import Callable, List, Optional, Tuple, TypeVar, Union
 
-from telegram.utils.types import JSONDict
+warnings.warn(
-
+    'telegram.utils.promise is deprecated. Please use telegram.ext.utils.promise instead.',
-RT = TypeVar('RT')
+    TelegramDeprecationWarning,
-
+)
-
-logger = logging.getLogger(__name__)
-
-
-class Promise:
-    """A simple Promise implementation for use with the run_async decorator, DelayQueue etc.
-
-    Args:
-        pooled_function (:obj:`callable`): The callable that will be called concurrently.
-        args (:obj:`list` | :obj:`tuple`): Positional arguments for :attr:`pooled_function`.
-        kwargs (:obj:`dict`): Keyword arguments for :attr:`pooled_function`.
-        update (:class:`telegram.Update` | :obj:`object`, optional): The update this promise is
-            associated with.
-        error_handling (:obj:`bool`, optional): Whether exceptions raised by :attr:`func`
-            may be handled by error handlers. Defaults to :obj:`True`.
-
-    Attributes:
-        pooled_function (:obj:`callable`): The callable that will be called concurrently.
-        args (:obj:`list` | :obj:`tuple`): Positional arguments for :attr:`pooled_function`.
-        kwargs (:obj:`dict`): Keyword arguments for :attr:`pooled_function`.
-        done (:obj:`threading.Event`): Is set when the result is available.
-        update (:class:`telegram.Update` | :obj:`object`): Optional. The update this promise is
-            associated with.
-        error_handling (:obj:`bool`): Optional. Whether exceptions raised by :attr:`func`
-            may be handled by error handlers. Defaults to :obj:`True`.
 
+Promise = promise.Promise
 """
+:class:`telegram.ext.utils.promise.Promise`
 
-    # TODO: Remove error_handling parameter once we drop the @run_async decorator
+.. deprecated:: v13.2
-    def __init__(
+   Use :class:`telegram.ext.utils.promise.Promise` instead.
-        self,
-        pooled_function: Callable[..., RT],
-        args: Union[List, Tuple],
-        kwargs: JSONDict,
-        update: object = None,
-        error_handling: bool = True,
-    ):
-        self.pooled_function = pooled_function
-        self.args = args
-        self.kwargs = kwargs
-        self.update = update
-        self.error_handling = error_handling
-        self.done = Event()
-        self._result: Optional[RT] = None
-        self._exception: Optional[Exception] = None
-
-    def run(self) -> None:
-        """Calls the :attr:`pooled_function` callable."""
-
-        try:
-            self._result = self.pooled_function(*self.args, **self.kwargs)
-
-        except Exception as exc:
-            self._exception = exc
-
-        finally:
-            self.done.set()
-
-    def __call__(self) -> None:
-        self.run()
-
-    def result(self, timeout: float = None) -> Optional[RT]:
-        """Return the result of the ``Promise``.
-
-        Args:
-            timeout (:obj:`float`, optional): Maximum time in seconds to wait for the result to be
-                calculated. ``None`` means indefinite. Default is ``None``.
-
-        Returns:
-            Returns the return value of :attr:`pooled_function` or ``None`` if the ``timeout``
-            expires.
-
-        Raises:
-            object exception raised by :attr:`pooled_function`.
 """
-        self.done.wait(timeout=timeout)
-        if self._exception is not None:
-            raise self._exception  # pylint: disable=raising-bad-type
-        return self._result
-
-    @property
-    def exception(self) -> Optional[Exception]:
-        """The exception raised by :attr:`pooled_function` or ``None`` if no exception has been
-        raised (yet)."""
-        return self._exception

telegram/utils/webhookhandler.py

@@ -16,193 +16,19 @@
 #
 # You should have received a copy of the GNU Lesser Public License
 # along with this program.  If not, see [http://www.gnu.org/licenses/].
-# pylint: disable=E0401, C0114
+"""This module contains the :class:`telegram.ext.utils.promise.Promise` class for backwards
+compatibility."""
+import warnings
 
-import asyncio
+import telegram.ext.utils.webhookhandler as webhook_handler
-import logging
+from telegram.utils.deprecate import TelegramDeprecationWarning
-import os
-import sys
-from queue import Queue
-from ssl import SSLContext
-from threading import Event, Lock
-from typing import TYPE_CHECKING, Any, Optional
 
-import tornado.web
+warnings.warn(
-from tornado import httputil
+    'telegram.utils.webhookhandler is deprecated. Please use telegram.ext.utils.webhookhandler '
-from tornado.httpserver import HTTPServer
+    'instead.',
-from tornado.ioloop import IOLoop
+    TelegramDeprecationWarning,
-
-from telegram import Update
-from telegram.utils.types import JSONDict
-
-if TYPE_CHECKING:
-    from telegram import Bot
-
-try:
-    import ujson as json
-except ImportError:
-    import json  # type: ignore[no-redef]
-
-
-class WebhookServer:
-    def __init__(
-        self, listen: str, port: int, webhook_app: 'WebhookAppClass', ssl_ctx: SSLContext
-    ):
-        self.http_server = HTTPServer(webhook_app, ssl_options=ssl_ctx)
-        self.listen = listen
-        self.port = port
-        self.loop: Optional[IOLoop] = None
-        self.logger = logging.getLogger(__name__)
-        self.is_running = False
-        self.server_lock = Lock()
-        self.shutdown_lock = Lock()
-
-    def serve_forever(self, force_event_loop: bool = False, ready: Event = None) -> None:
-        with self.server_lock:
-            self.is_running = True
-            self.logger.debug('Webhook Server started.')
-            self._ensure_event_loop(force_event_loop=force_event_loop)
-            self.loop = IOLoop.current()
-            self.http_server.listen(self.port, address=self.listen)
-
-            if ready is not None:
-                ready.set()
-
-            self.loop.start()
-            self.logger.debug('Webhook Server stopped.')
-            self.is_running = False
-
-    def shutdown(self) -> None:
-        with self.shutdown_lock:
-            if not self.is_running:
-                self.logger.warning('Webhook Server already stopped.')
-                return
-            self.loop.add_callback(self.loop.stop)  # type: ignore
-
-    def handle_error(self, request: object, client_address: str) -> None:  # pylint: disable=W0613
-        """Handle an error gracefully."""
-        self.logger.debug(
-            'Exception happened during processing of request from %s',
-            client_address,
-            exc_info=True,
 )
 
-    def _ensure_event_loop(self, force_event_loop: bool = False) -> None:
+WebhookHandler = webhook_handler.WebhookHandler
-        """If there's no asyncio event loop set for the current thread - create one."""
+WebhookServer = webhook_handler.WebhookServer
-        try:
+WebhookAppClass = webhook_handler.WebhookAppClass
-            loop = asyncio.get_event_loop()
-            if (
-                not force_event_loop
-                and os.name == 'nt'
-                and sys.version_info >= (3, 8)
-                and isinstance(loop, asyncio.ProactorEventLoop)
-            ):
-                raise TypeError(
-                    '`ProactorEventLoop` is incompatible with '
-                    'Tornado. Please switch to `SelectorEventLoop`.'
-                )
-        except RuntimeError:
-            # Python 3.8 changed default asyncio event loop implementation on windows
-            # from SelectorEventLoop to ProactorEventLoop. At the time of this writing
-            # Tornado doesn't support ProactorEventLoop and suggests that end users
-            # change asyncio event loop policy to WindowsSelectorEventLoopPolicy.
-            # https://github.com/tornadoweb/tornado/issues/2608
-            # To avoid changing the global event loop policy, we manually construct
-            # a SelectorEventLoop instance instead of using asyncio.new_event_loop().
-            # Note that the fix is not applied in the main thread, as that can break
-            # user code in even more ways than changing the global event loop policy can,
-            # and because Updater always starts its webhook server in a separate thread.
-            # Ideally, we would want to check that Tornado actually raises the expected
-            # NotImplementedError, but it's not possible to cleanly recover from that
-            # exception in current Tornado version.
-            if (
-                os.name == 'nt'
-                and sys.version_info >= (3, 8)
-                # OS+version check makes hasattr check redundant, but just to be sure
-                and hasattr(asyncio, 'WindowsProactorEventLoopPolicy')
-                and (
-                    isinstance(
-                        asyncio.get_event_loop_policy(),
-                        asyncio.WindowsProactorEventLoopPolicy,  # pylint: disable=E1101
-                    )
-                )
-            ):  # pylint: disable=E1101
-                self.logger.debug(
-                    'Applying Tornado asyncio event loop fix for Python 3.8+ on Windows'
-                )
-                loop = asyncio.SelectorEventLoop()
-            else:
-                loop = asyncio.new_event_loop()
-            asyncio.set_event_loop(loop)
-
-
-class WebhookAppClass(tornado.web.Application):
-    def __init__(self, webhook_path: str, bot: 'Bot', update_queue: Queue):
-        self.shared_objects = {"bot": bot, "update_queue": update_queue}
-        handlers = [(rf"{webhook_path}/?", WebhookHandler, self.shared_objects)]  # noqa
-        tornado.web.Application.__init__(self, handlers)
-
-    def log_request(self, handler: tornado.web.RequestHandler) -> None:
-        pass
-
-
-# WebhookHandler, process webhook calls
-# pylint: disable=W0223
-class WebhookHandler(tornado.web.RequestHandler):
-    SUPPORTED_METHODS = ["POST"]
-
-    def __init__(
-        self,
-        application: tornado.web.Application,
-        request: httputil.HTTPServerRequest,
-        **kwargs: JSONDict,
-    ):
-        super().__init__(application, request, **kwargs)
-        self.logger = logging.getLogger(__name__)
-
-    def initialize(self, bot: 'Bot', update_queue: Queue) -> None:
-        # pylint: disable=W0201
-        self.bot = bot
-        self.update_queue = update_queue
-
-    def set_default_headers(self) -> None:
-        self.set_header("Content-Type", 'application/json; charset="utf-8"')
-
-    def post(self) -> None:
-        self.logger.debug('Webhook triggered')
-        self._validate_post()
-        json_string = self.request.body.decode()
-        data = json.loads(json_string)
-        self.set_status(200)
-        self.logger.debug('Webhook received data: %s', json_string)
-        update = Update.de_json(data, self.bot)
-        if update:
-            self.logger.debug('Received Update with ID %d on Webhook', update.update_id)
-            self.update_queue.put(update)
-
-    def _validate_post(self) -> None:
-        ct_header = self.request.headers.get("Content-Type", None)
-        if ct_header != 'application/json':
-            raise tornado.web.HTTPError(403)
-
-    def write_error(self, status_code: int, **kwargs: Any) -> None:
-        """Log an arbitrary message.
-
-        This is used by all other logging functions.
-
-        It overrides ``BaseHTTPRequestHandler.log_message``, which logs to ``sys.stderr``.
-
-        The first argument, FORMAT, is a format string for the message to be logged.  If the format
-        string contains any % escapes requiring parameters, they should be specified as subsequent
-        arguments (it's just like printf!).
-
-        The client ip is prefixed to every message.
-
-        """
-        super().write_error(status_code, **kwargs)
-        self.logger.debug(
-            "%s - - %s",
-            self.request.remote_ip,
-            "Exception in WebhookHandler",
-            exc_info=kwargs['exc_info'],
-        )

tests/conftest.py

@@ -361,12 +361,12 @@ def check_shortcut_signature(
     Returns:
         :obj:`bool`: Whether or not the signature matches.
     """
-    shortcut_arg_spec = inspect.getfullargspec(shortcut)
+    shortcut_sig = inspect.signature(shortcut)
-    effective_shortcut_args = set(shortcut_arg_spec.args).difference(additional_kwargs)
+    effective_shortcut_args = set(shortcut_sig.parameters.keys()).difference(additional_kwargs)
     effective_shortcut_args.discard('self')
 
-    bot_arg_spec = inspect.getfullargspec(bot_method)
+    bot_sig = inspect.signature(bot_method)
-    expected_args = set(bot_arg_spec.args).difference(shortcut_kwargs)
+    expected_args = set(bot_sig.parameters.keys()).difference(shortcut_kwargs)
     expected_args.discard('self')
 
     args_check = expected_args == effective_shortcut_args
@@ -377,29 +377,29 @@ def check_shortcut_signature(
     # all
     annotation_check = True
     for kwarg in effective_shortcut_args:
-        if bot_arg_spec.annotations[kwarg] != shortcut_arg_spec.annotations[kwarg]:
+        if bot_sig.parameters[kwarg].annotation != shortcut_sig.parameters[kwarg].annotation:
-            if isinstance(bot_arg_spec.annotations[kwarg], type):
+            if isinstance(bot_sig.parameters[kwarg].annotation, type):
-                if bot_arg_spec.annotations[kwarg].__name__ != str(
+                if bot_sig.parameters[kwarg].annotation.__name__ != str(
-                    shortcut_arg_spec.annotations[kwarg]
+                    shortcut_sig.parameters[kwarg].annotation
                 ):
                     print(
-                        f'Expected {bot_arg_spec.annotations[kwarg]}, but '
+                        f'Expected {bot_sig.parameters[kwarg].annotation}, but '
-                        f'got {shortcut_arg_spec.annotations[kwarg]}'
+                        f'got {shortcut_sig.parameters[kwarg].annotation}'
                     )
                     annotation_check = False
                     break
             else:
                 print(
-                    f'Expected {bot_arg_spec.annotations[kwarg]}, but '
+                    f'Expected {bot_sig.parameters[kwarg].annotation}, but '
-                    f'got {shortcut_arg_spec.annotations[kwarg]}'
+                    f'got {shortcut_sig.parameters[kwarg].annotation}'
                 )
                 annotation_check = False
                 break
 
-    bot_method_signature = inspect.signature(bot_method)
+    bot_method_sig = inspect.signature(bot_method)
-    shortcut_signature = inspect.signature(shortcut)
+    shortcut_sig = inspect.signature(shortcut)
     default_check = all(
-        shortcut_signature.parameters[arg].default == bot_method_signature.parameters[arg].default
+        shortcut_sig.parameters[arg].default == bot_method_sig.parameters[arg].default
         for arg in expected_args
     )
 
@@ -429,7 +429,7 @@ def check_shortcut_call(
     Returns:
         :obj:`bool`
     """
-    bot_arg_spec = inspect.getfullargspec(bot_method)
+    bot_signature = inspect.signature(bot_method)
-    expected_args = set(bot_arg_spec.args).difference(['self'])
+    expected_args = set(bot_signature.parameters.keys()).difference(['self'])
 
     return expected_args == set(kwargs.keys())

tests/test_helpers.py

@@ -16,8 +16,12 @@
 #
 # You should have received a copy of the GNU Lesser Public License
 # along with this program.  If not, see [http://www.gnu.org/licenses/].
+import os
+import subprocess
+import sys
 import time
 import datetime as dtm
+from importlib import reload
 from pathlib import Path
 
 import pytest
@@ -46,6 +50,26 @@ RELATIVE_TIME_SPECS = DELTA_TIME_SPECS + TIME_OF_DAY_TIME_SPECS
 TIME_SPECS = ABSOLUTE_TIME_SPECS + RELATIVE_TIME_SPECS
 
 
+# This is here for ptb-raw, where we don't have pytz (unless the user installs it)
+@pytest.fixture(scope='function', params=[True, False])
+def pytz_install(request):
+    skip = not os.getenv('GITHUB_ACTIONS', False)
+    reason = 'Un/installing pytz slows tests down, so we just do that in CI'
+
+    if not request.param:
+        if skip:
+            pytest.skip(reason)
+        subprocess.check_call([sys.executable, "-m", "pip", "uninstall", "pytz", "-y"])
+        del sys.modules['pytz']
+        reload(helpers)
+    yield
+    if not request.param:
+        if skip:
+            pytest.skip(reason)
+        subprocess.check_call([sys.executable, "-m", "pip", "install", "pytz"])
+        reload(helpers)
+
+
 class TestHelpers:
     def test_escape_markdown(self):
         test_str = '*bold*, _italic_, `code`, [text_link](http://github.com/)'
@@ -84,13 +108,21 @@ class TestHelpers:
         with pytest.raises(ValueError):
             helpers.escape_markdown('abc', version=-1)
 
-    def test_to_float_timestamp_absolute_naive(self):
+    def test_to_float_timestamp_absolute_naive(self, pytz_install):
         """Conversion from timezone-naive datetime to timestamp.
         Naive datetimes should be assumed to be in UTC.
         """
         datetime = dtm.datetime(2019, 11, 11, 0, 26, 16, 10 ** 5)
         assert helpers.to_float_timestamp(datetime) == 1573431976.1
 
+    def test_to_float_timestamp_absolute_naive_no_pytz(self, monkeypatch, pytz_install):
+        """Conversion from timezone-naive datetime to timestamp.
+        Naive datetimes should be assumed to be in UTC.
+        """
+        monkeypatch.setattr(helpers, 'UTC', helpers.DTM_UTC)
+        datetime = dtm.datetime(2019, 11, 11, 0, 26, 16, 10 ** 5)
+        assert helpers.to_float_timestamp(datetime) == 1573431976.1
+
     def test_to_float_timestamp_absolute_aware(self, timezone):
         """Conversion from timezone-aware datetime to timestamp"""
         # we're parametrizing this with two different UTC offsets to exclude the possibility
@@ -114,7 +146,7 @@ class TestHelpers:
         delta = time_spec.total_seconds() if hasattr(time_spec, 'total_seconds') else time_spec
         assert helpers.to_float_timestamp(time_spec, reference_t) == reference_t + delta
 
-    def test_to_float_timestamp_time_of_day(self):
+    def test_to_float_timestamp_time_of_day(self, pytz_install):
         """Conversion from time-of-day specification to timestamp"""
         hour, hour_delta = 12, 1
         ref_t = _datetime_to_float_timestamp(dtm.datetime(1970, 1, 1, hour=hour))
@@ -141,7 +173,7 @@ class TestHelpers:
         )
 
     @pytest.mark.parametrize('time_spec', RELATIVE_TIME_SPECS, ids=str)
-    def test_to_float_timestamp_default_reference(self, time_spec):
+    def test_to_float_timestamp_default_reference(self, time_spec, pytz_install):
         """The reference timestamp for relative time specifications should default to now"""
         now = time.time()
         assert helpers.to_float_timestamp(time_spec) == pytest.approx(
@@ -153,7 +185,7 @@ class TestHelpers:
             helpers.to_float_timestamp(Defaults())
 
     @pytest.mark.parametrize('time_spec', TIME_SPECS, ids=str)
-    def test_to_timestamp(self, time_spec):
+    def test_to_timestamp(self, time_spec, pytz_install):
         # delegate tests to `to_float_timestamp`
         assert helpers.to_timestamp(time_spec) == int(helpers.to_float_timestamp(time_spec))
 
@@ -164,7 +196,7 @@ class TestHelpers:
     def test_from_timestamp_none(self):
         assert helpers.from_timestamp(None) is None
 
-    def test_from_timestamp_naive(self):
+    def test_from_timestamp_naive(self, pytz_install):
         datetime = dtm.datetime(2019, 11, 11, 0, 26, 16, tzinfo=None)
         assert helpers.from_timestamp(1573431976, tzinfo=None) == datetime
 

tests/test_meta.py

@@ -23,11 +23,11 @@ import pytest
 
 def call_pre_commit_hook(hook_id):
     __tracebackhide__ = True
-    return os.system(' '.join(['pre-commit', 'run', '--all-files', hook_id]))  # pragma: no cover
+    return os.system(' '.join(['pre-commit', 'run', hook_id, '--all-files']))  # pragma: no cover
 
 
 @pytest.mark.nocoverage
-@pytest.mark.parametrize('hook_id', argvalues=('yapf', 'flake8', 'pylint'))
+@pytest.mark.parametrize('hook_id', ('black', 'flake8', 'pylint', 'mypy'))
 @pytest.mark.skipif(not os.getenv('TEST_PRE_COMMIT', False), reason='TEST_PRE_COMMIT not enabled')
 def test_pre_commit_hook(hook_id):
     assert call_pre_commit_hook(hook_id) == 0  # pragma: no cover
@@ -37,3 +37,9 @@ def test_pre_commit_hook(hook_id):
 @pytest.mark.skipif(not os.getenv('TEST_BUILD', False), reason='TEST_BUILD not enabled')
 def test_build():
     assert os.system('python setup.py bdist_dumb') == 0  # pragma: no cover
+
+
+@pytest.mark.nocoverage
+@pytest.mark.skipif(not os.getenv('TEST_BUILD', False), reason='TEST_BUILD not enabled')
+def test_build_raw():
+    assert os.system('python setup-raw.py bdist_dumb') == 0  # pragma: no cover

tests/test_promise.py

@@ -0,0 +1,65 @@
+#!/usr/bin/env python
+#
+# A library that provides a Python interface to the Telegram Bot API
+# Copyright (C) 2015-2021
+# Leandro Toledo de Souza <devs@python-telegram-bot.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser Public License for more details.
+#
+# You should have received a copy of the GNU Lesser Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/].
+import pytest
+
+from telegram import TelegramError
+from telegram.ext.utils.promise import Promise
+
+
+class TestPromise:
+    """
+    Here we just test the things that are not covered by the other tests anyway
+    """
+
+    test_flag = False
+
+    @pytest.fixture(autouse=True)
+    def reset(self):
+        self.test_flag = False
+
+    def test_call(self):
+        def callback():
+            self.test_flag = True
+
+        promise = Promise(callback, [], {})
+        promise()
+
+        assert promise.done
+        assert self.test_flag
+
+    def test_run_with_exception(self):
+        def callback():
+            raise TelegramError('Error')
+
+        promise = Promise(callback, [], {})
+        promise.run()
+
+        assert promise.done
+        assert not self.test_flag
+        assert isinstance(promise.exception, TelegramError)
+
+    def test_wait_for_exception(self):
+        def callback():
+            raise TelegramError('Error')
+
+        promise = Promise(callback, [], {})
+        promise.run()
+
+        with pytest.raises(TelegramError, match='Error'):
+            promise.result()

tests/test_updater.py

@@ -40,7 +40,7 @@ from telegram import TelegramError, Message, User, Chat, Update, Bot
 from telegram.error import Unauthorized, InvalidToken, TimedOut, RetryAfter
 from telegram.ext import Updater, Dispatcher, DictPersistence, Defaults
 from telegram.utils.deprecate import TelegramDeprecationWarning
-from telegram.utils.webhookhandler import WebhookServer
+from telegram.ext.utils.webhookhandler import WebhookServer
 
 signalskip = pytest.mark.skipif(
     sys.platform == 'win32',

tests/test_utils.py

@@ -0,0 +1,37 @@
+#!/usr/bin/env python
+#
+# A library that provides a Python interface to the Telegram Bot API
+# Copyright (C) 2015-2021
+# Leandro Toledo de Souza <devs@python-telegram-bot.org>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser Public License for more details.
+#
+# You should have received a copy of the GNU Lesser Public License
+# along with this program.  If not, see [http://www.gnu.org/licenses/].
+
+
+class TestUtils:
+    def test_promise_deprecation(self, recwarn):
+        import telegram.utils.promise  # noqa: F401
+
+        assert len(recwarn) == 1
+        assert str(recwarn[0].message) == (
+            'telegram.utils.promise is deprecated. Please use telegram.ext.utils.promise instead.'
+        )
+
+    def test_webhookhandler_deprecation(self, recwarn):
+        import telegram.utils.webhookhandler  # noqa: F401
+
+        assert len(recwarn) == 1
+        assert str(recwarn[0].message) == (
+            'telegram.utils.webhookhandler is deprecated. Please use '
+            'telegram.ext.utils.webhookhandler instead.'
+        )