Error handlers now handle all errors (#1483)

* python3.4 is no longer supported

* Prepare CHANGES.RST & README.rst for v12.0.0 release

* CHANGES.rst: small fix

* Add Bibo-Joshi to Credits

* improving error_handler

* fixing affected tests

.travis.yml

@@ -2,7 +2,6 @@ language: python
 matrix:
   include:
     - python: 2.7
-    - python: 3.4
     - python: 3.5
     - python: 3.6
     - python: 3.7

AUTHORS.rst

@@ -4,7 +4,7 @@ Credits
 ``python-telegram-bot`` was originally created by
 `Leandro Toledo <https://github.com/leandrotoledo>`_ and is now maintained by
 `Jannes Höke <https://github.com/jh0ker>`_ (`@jh0ker <https://t.me/jh0ker>`_ on Telegram),
-`Noam Meltzer <https://github.com/tsnoam>`_, `Pieter Schutz <https://github.com/eldinnie>`_ and `Jasmin Bom <https://github.com/jsmnbom>`_.
+`Noam Meltzer <https://github.com/tsnoam>`_, `Pieter Schutz <https://github.com/eldinnie>`_, `Jasmin Bom <https://github.com/jsmnbom>`_ and `Hinrich Mahler <https://github.com/Bibo-Joshi>`_.
 
 We're vendoring urllib3 as part of ``python-telegram-bot`` which is distributed under the MIT
 license. For more info, full credits & license terms, the sources can be found here:
@@ -65,6 +65,7 @@ The following wonderful people contributed directly or indirectly to this projec
 - `Patrick Hofmann <https://github.com/PH89>`_
 - `Paul Larsen <https://github.com/PaulSonOfLars>`_
 - `Pieter Schutz <https://github.com/eldinnie>`_
+- `Poolitzer <https://github.com/Poolitzer>`_
 - `Rahiel Kasim <https://github.com/rahiel>`_
 - `Sascha <https://github.com/saschalalala>`_
 - `Shelomentsev D <https://github.com/shelomentsevd>`_

CHANGES.rst

@@ -2,14 +2,13 @@
 Changelog
 =========
 
-Version 12.0.0b1
+Version 12.0.0
 ================
-*Released 2019-02-13*
+*Released 2019-08-24*
 
-First beta release ever.
+Well... This felt like decades. But here we are with a new release.
-It has been so long since last release that we would like to test the impact before a final release.
 
-*We do NOT recommend using this beta release in production.*
+Expect minor releases soon (mainly complete Bot API 4.4 support)
 
 **Major changes:**
 
@@ -18,6 +17,9 @@ It has been so long since last release that we would like to test the impact bef
 - PrefixHandler added (Handler overhaul)
 - Deprecation of RegexHandler and edited_messages, channel_post, etc. arguments (Filter overhaul)
 - Various ConversationHandler changes and fixes
+- Bot API 4.1, 4.2, 4.3 support
+- Initial Bot API 4.4 support
+- Python 3.4 is no longer supported
 
 **See the wiki page at https://git.io/fxJuV for a detailed guide on how to migrate from version 11 to version 12.**
 
@@ -82,6 +84,13 @@ Bug fixes & improvements
 - Allow SOCKSConnection to parse username and password from URL (`#1211`_)
 - Fix for arguments in passport/data.py (`#1213`_)
 - Improve message entity parsing by adding text_mention (`#1206`_)
+- Documentation fixes (`#1348`_, `#1397_`, `#1436`_)
+- Merged filters short-circuit (`#1350`_)
+- Fix webhook listen with tornado (`#1383`_)
+- Call task_done() on update queue after update processing finished (`#1428`_)
+- Fix send_location() - latitude may be 0 (`#1437`_)
+- Make MessageEntity objects comparable (`#1465`_)
+- Add prefix to thread names (`#1358`_)
 
 .. _`#1100`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1100
 .. _`#1283`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1283
@@ -110,6 +119,15 @@ Bug fixes & improvements
 .. _`#1319`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1319
 .. _`#1343`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1343
 .. _`#1270`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1270
+.. _`#1348`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1348
+.. _`#1350`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1350
+.. _`#1383`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1383
+.. _`#1397`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1397
+.. _`#1428`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1428
+.. _`#1436`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1436
+.. _`#1437`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1437
+.. _`#1465`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1465
+.. _`#1358`: https://github.com/python-telegram-bot/python-telegram-bot/pull/1358
 
 Internal improvements
 ---------------------

README.rst

@@ -104,21 +104,7 @@ All types and methods of the Telegram Bot API **4.1** are supported.
 Installing
 ==========
 
-**Beta note**
+You can install or upgrade python-telegram-bot with:
-
-The newest stable release is currently version 11.1.0.
-
-The newest release is a beta release for version 12.
-Install or upgrade with:
-
-.. code:: shell
-
-    $ pip install python-telegram-bot==12.0.0b1 --upgrade
-
-
-See CHANGES.rst for the changelog and make sure to report any bugs you find!
-
-You can install or upgrade the stable python-telegram-bot with:
 
 .. code:: shell
 

telegram/ext/dispatcher.py

@@ -328,15 +328,29 @@ class Dispatcher(object):
                 if self.persistence.store_chat_data and update.effective_chat:
                     chat_id = update.effective_chat.id
                     try:
-                        self.persistence.update_chat_data(chat_id, self.chat_data[chat_id])
+                        self.persistence.update_chat_data(chat_id,
-                    except Exception:
+                                                          self.chat_data[chat_id])
-                        self.logger.exception('Saving chat data raised an error')
+                    except Exception as e:
+                        try:
+                            self.dispatch_error(update, e)
+                        except Exception:
+                            message = 'Saving chat data raised an error and an ' \
+                                      'uncaught error was raised while handling ' \
+                                      'the error with an error_handler'
+                            self.logger.exception(message)
                 if self.persistence.store_user_data and update.effective_user:
                     user_id = update.effective_user.id
                     try:
-                        self.persistence.update_user_data(user_id, self.user_data[user_id])
+                        self.persistence.update_user_data(user_id,
-                    except Exception:
+                                                          self.user_data[user_id])
-                        self.logger.exception('Saving user data raised an error')
+                    except Exception as e:
+                        try:
+                            self.dispatch_error(update, e)
+                        except Exception:
+                            message = 'Saving user data raised an error and an ' \
+                                      'uncaught error was raised while handling ' \
+                                      'the error with an error_handler'
+                            self.logger.exception(message)
 
         # An error happened while polling
         if isinstance(update, TelegramError):
@@ -366,20 +380,17 @@ class Dispatcher(object):
                 break
 
             # Dispatch any error.
-            except TelegramError as te:
+            except Exception as e:
-                self.logger.warning('A TelegramError was raised while processing the Update')
-
                 try:
-                    self.dispatch_error(update, te)
+                    self.dispatch_error(update, e)
                 except DispatcherHandlerStop:
                     self.logger.debug('Error handler stopped further handlers')
                     break
+                # Errors should not stop the thread.
                 except Exception:
-                    self.logger.exception('An uncaught error was raised while handling the error')
+                    self.logger.exception('An error was raised while processing the update and an '
-
+                                          'uncaught error was raised while handling the error '
-            # Errors should not stop the thread.
+                                          'with an error_handler')
-            except Exception:
-                self.logger.exception('An uncaught error was raised while processing the update')
 
     def add_handler(self, handler, group=DEFAULT_GROUP):
         """Register a handler.
@@ -453,11 +464,15 @@ class Dispatcher(object):
                     self.persistence.update_user_data(user_id, self.user_data[user_id])
 
     def add_error_handler(self, callback):
-        """Registers an error handler in the Dispatcher.
+        """Registers an error handler in the Dispatcher. This handler will receive every error
+        which happens in your bot.
+
+        Warning: The errors handled within these handlers won't show up in the logger, so you
+        need to make sure that you reraise the error.
 
         Args:
             callback (:obj:`callable`): The callback function for this error handler. Will be
-                called when an error is raised Callback signature for context based API:
+                called when an error is raised. Callback signature for context based API:
 
                 ``def callback(update: Update, context: CallbackContext)``
 
@@ -483,7 +498,7 @@ class Dispatcher(object):
 
         Args:
             update (:obj:`str` | :class:`telegram.Update` | None): The update that caused the error
-            error (:class:`telegram.TelegramError`): The Telegram error that was raised.
+            error (:obj:`Exception`): The error that was raised.
 
         """
         if self.error_handlers:

tests/test_dispatcher.py

@@ -24,10 +24,12 @@ from time import sleep
 import pytest
 
 from telegram import TelegramError, Message, User, Chat, Update, Bot, MessageEntity
-from telegram.ext import MessageHandler, Filters, CommandHandler, CallbackContext, JobQueue
+from telegram.ext import (MessageHandler, Filters, CommandHandler, CallbackContext,
+                          JobQueue, BasePersistence)
 from telegram.ext.dispatcher import run_async, Dispatcher, DispatcherHandlerStop
 from telegram.utils.deprecate import TelegramDeprecationWarning
 from tests.conftest import create_dp
+from collections import defaultdict
 
 
 @pytest.fixture(scope='function')
@@ -276,10 +278,11 @@ class TestDispatcher(object):
 
     def test_exception_in_handler(self, dp, bot):
         passed = []
+        err = Exception('General exception')
 
         def start1(b, u):
             passed.append('start1')
-            raise Exception('General exception')
+            raise err
 
         def start2(b, u):
             passed.append('start2')
@@ -298,14 +301,14 @@ class TestDispatcher(object):
                                            bot=bot))
 
         # If an unhandled exception was caught, no further handlers from the same group should be
-        # called.
+        # called. Also, the error handler should be called and receive the exception
         passed = []
         dp.add_handler(CommandHandler('start', start1), 1)
         dp.add_handler(CommandHandler('start', start2), 1)
         dp.add_handler(CommandHandler('start', start3), 2)
         dp.add_error_handler(error)
         dp.process_update(update)
-        assert passed == ['start1', 'start3']
+        assert passed == ['start1', 'error', err, 'start3']
 
     def test_telegram_error_in_handler(self, dp, bot):
         passed = []
@@ -341,6 +344,49 @@ class TestDispatcher(object):
         assert passed == ['start1', 'error', err, 'start3']
         assert passed[2] is err
 
+    def test_error_while_saving_chat_data(self, dp, bot):
+        increment = []
+
+        class OwnPersistence(BasePersistence):
+            def __init__(self):
+                super(BasePersistence, self).__init__()
+                self.store_user_data = True
+                self.store_chat_data = True
+
+            def get_chat_data(self):
+                return defaultdict(dict)
+
+            def update_chat_data(self, chat_id, data):
+                raise Exception
+
+            def get_user_data(self):
+                return defaultdict(dict)
+
+            def update_user_data(self, user_id, data):
+                raise Exception
+
+        def start1(b, u):
+            pass
+
+        def error(b, u, e):
+            increment.append("error")
+
+        # If updating a user_data or chat_data from a persistence object throws an error,
+        # the error handler should catch it
+
+        update = Update(1, message=Message(1, User(1, "Test", False), None, Chat(1, "lala"),
+                                           text='/start',
+                                           entities=[MessageEntity(type=MessageEntity.BOT_COMMAND,
+                                                                   offset=0,
+                                                                   length=len('/start'))],
+                                           bot=bot))
+        my_persistence = OwnPersistence()
+        dp = Dispatcher(bot, None, persistence=my_persistence)
+        dp.add_handler(CommandHandler('start', start1))
+        dp.add_error_handler(error)
+        dp.process_update(update)
+        assert increment == ["error", "error"]
+
     def test_flow_stop_in_error_handler(self, dp, bot):
         passed = []
         err = TelegramError('Telegram error')

tests/test_persistence.py

@@ -176,10 +176,10 @@ class TestBasePersistence(object):
         with caplog.at_level(logging.ERROR):
             dp.process_update(u)
         rec = caplog.records[-1]
-        assert rec.msg == 'Saving user data raised an error'
+        assert rec.msg == 'No error handlers are registered, logging exception.'
         assert rec.levelname == 'ERROR'
         rec = caplog.records[-2]
-        assert rec.msg == 'Saving chat data raised an error'
+        assert rec.msg == 'No error handlers are registered, logging exception.'
         assert rec.levelname == 'ERROR'
         m.from_user = user2
         m.chat = chat1