45 commits

Author	SHA1	Message	Date
Il'ya (Marshal)	7aee3658e3	resend requests with 5XX status code	2022-02-24 21:58:17 +01:00
Il'ya (Marshal)	d74e0551d9	addd sticker related links	2022-01-31 19:22:23 +01:00
Il'ya (Marshal)	3fda615c53	add video stickers guidelines	2022-01-29 12:02:16 +01:00
Il'ya (Marshal)	344e550bd9	add 304 as success response; ... add handling of xml; add sparkle file of macOS client; fix request retrying with 500 status code; fix encoding.	2021-12-30 14:22:01 +01:00
Il'ya (Marshal)	c3a66d3397	return convertToGigagroup on corefork	2021-11-29 19:21:16 +01:00
Il'ya (Marshal)	2f84611c90	add td_api of tdlib	2021-11-11 10:18:01 +01:00
Il'ya (Marshal)	57ace577fa	add mtproto tl	2021-11-04 09:24:24 +01:00
Il'ya (Marshal)	45c7211f67	fix tracking of ad panel	2021-10-26 17:55:38 +02:00
Il'ya (Marshal)	e5e09b7156	new link for ads	2021-10-26 17:40:07 +02:00
Il'ya (Marshal)	6262ec0372	fix bug with tracking of plain text urls	2021-10-01 19:47:42 +02:00
Il'ya (Marshal)	d17a2978f7	add getProxyConfig	2021-10-01 19:18:27 +02:00
Il'ya (Marshal)	9e3ac83140	exclude strange url	2021-09-26 21:02:35 +02:00
Il'ya (Marshal)	58e3417baa	return tracking of corefork	2021-09-09 20:43:53 +02:00
Il'ya	addc1f66e2	waiting for ads	2021-09-01 14:55:53 +02:00
Alisa Sireneva	22d07f21f7	Added TDLib schema link	2021-08-31 15:18:25 +03:00
Il'ya (Marshal)	1961b658a8	add comparing output to github ci logs	2021-06-27 23:06:29 +02:00
Il'ya (Marshal)	12672aba45	resend requests with server error response	2021-06-21 18:39:20 +02:00
Il'ya (Marshal)	0b86273326	add logging of skips	2021-06-21 16:09:58 +02:00
Il'ya (Marshal)	4b84e48098	add contest.com	2021-06-01 09:03:44 +02:00
Il'ya (Marshal)	1b8af50843	add additional urls section; ... add tracking of mtproto schema.	2021-06-01 08:57:18 +02:00
Il'ya (Marshal)	83fbc4b4a3	add apple privacy page	2021-05-28 20:43:54 +02:00
Il'ya (Marshal)	f9cddfdb47	add CIDR	2021-05-24 13:52:40 +02:00
Il'ya (Marshal)	fe44e713b7	exclude dynamic list of users	2021-05-05 12:59:01 +02:00
Il'ya (Marshal)	002c2c61d9	temp exclude strange dynamic page	2021-05-01 16:28:12 +02:00
Il'ya (Marshal)	ede1ee0888	add mtproto api to crawler	2021-05-01 15:39:24 +02:00
Il'ya (Marshal)	e55ff9fdbc	exclude frequently updated pages	2021-05-01 15:10:20 +02:00
Il'ya (Marshal)	0aa464ac96	Revert "test alert system" ... This reverts commit fad0585d	2021-04-25 18:59:03 +02:00
Il'ya (Marshal)	fad0585daf	test alert system	2021-04-25 16:37:52 +02:00
Il'ya (Marshal)	ae43e9e442	cleanup readme file; disable tests of link crawler.	2021-04-25 14:13:52 +02:00
Il'ya (Marshal)	2dffddeaae	fix resending of requests; ... fix timeout settings; fix handling of invalid urls; edit schedule of generation list to test.	2021-04-25 08:18:06 +02:00
Il'ya (Marshal)	9a4a6e55ac	temporary remove /apps page with easter egg from Telegram (random order web clients); update links list as often as possible for tests	2021-04-24 23:06:56 +02:00
Il'ya (Marshal)	dcd5adc08a	fix, improve and cleanup links crawler; add diff or files	2021-04-24 22:40:39 +02:00
Il'ya (Marshal)	e927373229	disable crawling of bugs.telegram.org	2021-04-24 14:43:01 +02:00
Il'ya (Marshal)	6ceb897e33	move tracked links list to main branch; ... add generation of content tree; add workflow for tree generation; add new tg hidden link.	2021-04-24 14:19:01 +02:00
Il'ya (Marshal)	5f29beda73	handle timeout and retry	2021-04-24 11:38:33 +02:00
Il'ya (Marshal)	cd5653605f	rewrite logic of rules system (not its uses regex); ... fix trailing slash; many attempt to send request;	2021-04-24 11:29:19 +02:00
Il'ya (Marshal)	b2afbd72fc	rework exclude system; add exclude by count of slashes; add translations.telegram.org	2021-04-24 00:06:50 +02:00
Il'ya (Marshal)	e385f45da6	print lead time	2021-04-23 22:49:21 +02:00
Il'ya (Marshal)	246a0087e1	fix performance and memory issues	2021-04-23 22:38:54 +02:00
Il'ya (Marshal)	d449290dfb	add ability to set hidden urls	2021-04-23 22:31:23 +02:00
Il'ya (Marshal)	fba0bd1d64	return api section of tg core	2021-04-23 22:05:00 +02:00
Il'ya (Marshal)	c118a32a2a	test correct diff	2021-04-23 21:52:14 +02:00
Il'ya (Marshal)	4a422db809	the end.	2021-04-23 21:47:15 +02:00
Il'ya (Marshal)	b053015289	fix set operation	2021-04-23 20:44:17 +02:00
Il'ya (Marshal)	80934fecaa	Initial commit	2021-04-23 20:34:42 +02:00