Commit graph

299 commits

Author SHA1 Message Date
Eugen Rochko
6f78500d4f
Do not remove "dead" domains in tootctl accounts cull (#9108)
Leave `tootctl accounts cull` to simply check removed accounts from
live domains, and skip temporarily unavailable domains, while listing
them in the final output for further action.

Add `tootctl domains purge DOMAIN` to be able to purge a domain from
that list manually
2018-10-27 22:56:16 +02:00
Sascha
b9d7021c1b cli: set exit_on_failure for all CLI classes (#9094) 2018-10-25 16:05:33 +02:00
Eugen Rochko
2f0797bdbd
Bump version to 2.6.0rc2 (#9087)
* Bump version to 2.6.0rc2

* Update CHANGELOG.md
2018-10-25 04:42:25 +02:00
Eugen Rochko
f5b8bd4392
Fix cull tripping on nil in last_webfingered_at (#9051)
Fix #8741
2018-10-22 16:58:08 +02:00
Jeong Arm
2e18ad74dc Fix tootctl cull on dead servers (#9041)
* Delete first 9 accounts on dead servers

* Clean up code by moving dead server culling to the end
2018-10-21 22:52:27 +02:00
Jeong Arm
68f0e4d54e Handle if username is not found on tootctl feeds build (#9040) 2018-10-21 16:42:22 +02:00
Eugen Rochko
f468bfb830
Bump version to 2.6.0rc1 (#9025)
* Bump version to 2.6.0rc1

* Update AUTHORS.md

* Update CHANGELOG.md
2018-10-21 00:49:36 +02:00
Eugen Rochko
ef43f1d2ca
Bump version to 2.5.2 (#8960) 2018-10-12 00:24:09 +02:00
Eugen Rochko
e6c01171de
Bump version to 2.5.1 (#8953) 2018-10-11 02:29:32 +02:00
ashleyhull-versent
f194857ac9 rubocop issues - Cleaning up (#8912)
* cleanup pass

* undo mistakes

* fixed.

* revert
2018-10-08 04:50:11 +02:00
Eugen Rochko
186024a058
Add tootctl settings registrations open (#8829) 2018-10-04 16:05:54 +02:00
Eugen Rochko
ac07bfb018 Fix tootctl accounts reattaching not unsuspending deleted account (#8812) 2018-09-29 18:40:40 +09:00
Eugen Rochko
38f5f9cf5b
Add tootctl accounts backup (#8811) 2018-09-28 03:34:24 +02:00
ThibG
9b32898e3c Unconditionally re-encode locally-uploaded images to strip metadata (#8714)
This strips metadata on file upload by re-encoding the files, at the cost
of possible slight image quality decrease and processing resources.
2018-09-17 20:24:46 +02:00
Eugen Rochko
6a3f9b7e53
Move more tasks to tootctl (#8675)
* Move more tasks to tootctl

- tootctl feeds build
- tootctl feeds clear
- tootctl accounts refresh

Clean up exit codes and help messages

* Move user modifying to tootctl

* Improve user modification through CLI, rename commands

add -> create
mod -> modify
del -> delete

To remove ambiguity

* Fix code style issues

* Fix not being able to unset admin/mod role
2018-09-14 17:42:22 +02:00
luzpaz
40dd19be37 Misc. typos (#8694)
Found via `codespell -q 3 --skip="./app/javascript/mastodon/locales,./config/locales"`
2018-09-14 00:53:09 +02:00
rinsuki
25dd523887 using mailto scheme in AUTHORS.md (#8663) 2018-09-10 02:01:03 +02:00
nightpool
b0b484ba18 Add rake task for generating AUTHORS.md (#8661)
* add rake task for generating AUTHORS.md

* update AUTHORS.md

* rubocop
2018-09-10 01:31:42 +02:00
Eugen Rochko
47ea318479
tootctl accounts [add|del|cull] (#8642)
* CLI interface for creating/deleting local users

- tootctl accounts add USERNAME
- tootctl accounts del USERNAME

* Add CLI interface for culling remote users that no longer exist

- tootctl accounts cull
2018-09-09 13:33:36 +02:00
Eugen Rochko
b65a9b7243
Bump version to 2.5.0 2018-09-02 22:32:27 +02:00
Eugen Rochko
08721170da
Bump version to 2.5.0rc2 2018-09-01 03:37:43 +02:00
Eugen Rochko
50f226348f
Fix wrong string being used on login failure when using LDAP (#8534)
Fix #8527
2018-09-01 03:35:37 +02:00
mayaeh
18eb565755 Add --verbose and --dry-run option to tootctl media remove (#8519)
* Add --verbose and --dry-run options to CLI interface for removing remote media.

* Fix coding style problem.
2018-08-31 03:46:13 +02:00
mayaeh
cc26fd71ac Fix CLI interface for removing remote media (#8506)
* Fix that can't delete media files even if "tootctl media remove" execute when "--background" not attached.

* Revert

This reverts commit 5aa7e09645.

* Change to obtain and pass all columns when "--background" option is false.
2018-08-29 18:35:09 +02:00
Eugen Rochko
cabdbb7f9c
Add CLI task for rotating keys (#8466)
* If an Update is signed with known key, skip re-following procedure

Because it means the remote actor did *not* lose their database

* Add CLI method for rotating keys

    bin/tootctl accounts rotate [USERNAME]

Generates a new RSA key per account and sends out an Update activity
signed with the old key.

* Key rotation: Space out Update fan-outs every 5 minutes per 1000 accounts

* Skip suspended accounts in key rotation
2018-08-26 20:21:03 +02:00
Quint Guvernator
da13fa5021 Fix low-hanging rubocop gripes (#8458)
* rubocop: quit being so picky

* rubocop: miscellany

* rubocop: prefer present to blank
2018-08-26 19:22:46 +02:00
Eugen Rochko
b9c3788485
Fix NUM_DAYS in rake mastodon:media:remove_remote and print warning (#8462)
Remove old deprecated rake tasks
2018-08-26 17:19:47 +02:00
Eugen Rochko
b378b4c5c7
Add CLI interface for importing custom emoji (#8437)
bin/tootctl emoji import PATH_TO_TAR

Fix #8435
2018-08-26 16:53:06 +02:00
M Somerville
2bba6e582d Rename S3_CLOUDFRONT_HOST to S3_ALIAS_HOST. (#8423)
Still check for S3_CLOUDFRONT_HOST for existing installs.
2018-08-25 13:27:08 +02:00
Eugen Rochko
793eea2982
Add improved CLI interface for removing remote media (#8411)
./bin/tootctl media remove --days 7 --background

Make the old rake task point to it
2018-08-25 13:25:39 +02:00
Eugen Rochko
23752639b2
Bump version to 2.5.0rc1 (#8356) 2018-08-23 23:30:57 +02:00
Eugen Rochko
d98de8ada7
Get rid of all batch order warnings (#8334) 2018-08-21 12:25:50 +02:00
Immae
b0f4fe456b Add ldap search filter (#8151) 2018-08-15 18:12:44 +02:00
Eugen Rochko
018a9e4e7f
Add post-deployment migration system (#8182)
Adopted from GitLab CE. Generate new migration with:

    rails g post_deployment_migration name_of_migration_here

By default they are run together with db:migrate. To not run them,
the env variable SKIP_POST_DEPLOYMENT_MIGRATIONS must be set

Code by Yorick Peterse <yorickpeterse@gmail.com>, see also:

83c8241160
2018-08-13 13:40:01 +02:00
ThibG
110b3f6335 Add some feedback to maintenance rake tasks (#8173) 2018-08-11 22:02:55 +02:00
Eugen Rochko
bb71538bb5
Redesign public profiles and toots (#8068) 2018-07-28 19:25:33 +02:00
Eugen Rochko
0fb0037ca7
Resize images by area instead of fixed dimensions (#8083)
To improve the way super tall or super ride images are treated, the
numbers remain the same, 1280x1280 and 400x400, but if an image
is less in one dimension than the other, the other can become larger

Thanks to @WAHa_06x36@mastodon.social for the tip
2018-07-28 03:33:00 +02:00
Eugen Rochko
5a6645c922
Fix static GIFs being saved as empty files when using local storage (#8012)
Fix #7997
Fix #6237
2018-07-14 03:56:52 +02:00
Eugen Rochko
7ac5151b74
Bump version to 2.4.3 2018-07-11 11:21:27 +02:00
Eugen Rochko
89dc0c98ee
Bump version to 2.4.3rc3 2018-07-11 01:31:23 +02:00
Eugen Rochko
451e585b97
Bump version to 2.4.3rc2 2018-07-07 22:23:52 +02:00
Eugen Rochko
cb36ab9a10
Bump version to 2.4.3rc1 2018-07-05 21:08:03 +02:00
Eugen Rochko
d017f240f9
Bump version to 2.4.2 2018-06-19 01:45:00 +02:00
Eugen Rochko
86cce466b6
Bump version to 2.4.2rc3 2018-06-19 00:19:54 +02:00
Eugen Rochko
a04dca36a3
Bump version to 2.4.2rc2 2018-06-18 23:01:52 +02:00
Eugen Rochko
3b8023f9f9
Bump version to 2.4.2rc1 2018-06-15 20:38:41 +02:00
Eugen Rochko
0979d4275a
Bump version to 2.4.1 2018-06-10 17:20:56 +02:00
Eugen Rochko
35ae960122
Bump version to 2.4.1rc4 2018-06-10 00:39:56 +02:00
Eugen Rochko
158cd7ee74
Bump version to 2.4.1rc3 2018-06-06 21:27:06 +02:00
Eugen Rochko
fab65fb7e5
Bump version to 2.4.1rc2 2018-06-05 11:53:30 +00:00
Eugen Rochko
c7405fda11
Fix missing column in select in mastodon:feeds:build task (#7720) 2018-06-04 04:21:15 +02:00
Eugen Rochko
ad40bf5e0c
Speed up some rake tasks by moving execution to Sidekiq (#7678)
* Speed up some rake tasks by moving execution to Sidekiq

mastodon:media:remove_silenced
mastodon:media:remove_remote
mastodon:media:redownload_avatars
mastodon:feeds:build

* Fix code style issue
2018-05-30 21:09:30 +02:00
Eugen Rochko
a29f196f95
Bump version to 2.4.1rc1 2018-05-30 19:16:41 +02:00
Eugen Rochko
7745a22ec7
Bump version to 2.4.0 2018-05-22 19:39:35 +02:00
Eugen Rochko
292c987522
Bump version to 2.4.0rc5 2018-05-21 16:05:31 +02:00
Eugen Rochko
bcbb6aa46f
Bump version to 2.4.0rc4 2018-05-19 18:38:10 +02:00
MIYAGI Hikaru
919eef3098 User agent for WebFinger (#7531)
* User agent for WebFinger

* local_domain → web_domain

* 'http' is away accidentally...
2018-05-18 01:47:22 +02:00
Eugen Rochko
8756fd1e82
Bump version to 2.4.0rc3 2018-05-14 17:46:44 +02:00
Eugen Rochko
d75b63e4fb
Bump version to 2.4.0rc2 2018-05-14 14:49:38 +02:00
Eugen Rochko
50ed1e83ac
Bump version to 2.4.0rc1 (#7283) 2018-05-06 11:50:00 +02:00
ThibG
a24605961a Fixes/do not override timestamps (#7336)
* Revert "Fixes/do not override timestamps (#7331)"

This reverts commit 581a5c9d29.

* Document Snowflake ID corner-case a bit more

Snowflake IDs are used for two purposes: making object identifiers harder to
guess and ensuring they are in chronological order. For this reason, they
are based on the `created_at` attribute of the object.

Unfortunately, inserting items with older snowflakes IDs will break the
assumption of consumers of the paging APIs that new items will always have
a greater identifier than the last seen one.

* Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at
2018-05-03 23:02:46 +02:00
Eugen Rochko
cb5b5cb5f7
Slightly reduce RAM usage (#7301)
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
Eugen Rochko
63553c6b5c
Add support for separate Redis for cache (#7272)
* Add support for separate Redis for cache

CACHE_REDIS_URL to allow using a different Redis server for cache
purposes, with cache-specific configuration such as key eviction

* Fix code style issues
2018-04-27 01:37:59 +02:00
Alejandro Martinez Ruiz
06817b3c1f tasks/mastodon: fix prompt for Redis password (#7241) 2018-04-23 16:03:58 +02:00
David Baucum
ca9192d9ba Ability to specify Redis passwd on mastodon:setup (#7222)
Closes #7221
2018-04-22 11:49:16 +02:00
Yamagishi Kazutoshi
084cf0babf Add extract_foreign_key_action to Mastodon::MigrationHelpers (#7195) 2018-04-20 12:21:28 +02:00
Eugen Rochko
36eac8ba90
Do not default SMTP verify mode to "peer", default to "none" (#6996) 2018-04-02 19:19:51 +02:00
Eugen Rochko
2a90da1837
Fix UniqueUsernameValidator comparison (#6926)
Comparison was downcasing only one side, therefore if previously
existing account had a non-lowercase spelling, it would be ignored
when checking for duplicates.

New rake task `mastodon:maintenance:find_duplicate_usernames` will
help find constraint violations that might have occured from the
presence of this bug.

Bump version to 2.3.3
2018-03-27 04:33:57 +02:00
Akihiko Odaki
54b273bf99 Close http connection in perform method of Request class (#6889)
HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.
2018-03-24 12:49:54 +01:00
Eugen Rochko
6cc432bbc4
Bump version to 2.3.2 2018-03-22 14:13:46 +01:00
Eugen Rochko
dafae9818d
Bump version to 2.3.2rc5 2018-03-22 11:31:52 +01:00
Eugen Rochko
f64af6473f
Bump version to 2.3.2rc4 2018-03-20 23:49:24 +01:00
Eugen Rochko
ac49c7932d
Add LDAP_TLS_NO_VERIFY option, don't require LDAP_ENABLED outside .env (#6845)
Fix #6816, fix #6790
2018-03-20 19:41:51 +01:00
Eugen Rochko
74c39fada0
Bump version to 2.3.2rc3 2018-03-19 12:20:57 +01:00
Eugen Rochko
092f1df9d0
Bump version to 2.3.2rc2 2018-03-17 15:28:52 +01:00
Eugen Rochko
cdf8b92fea
Bump version to 2.3.2rc1 2018-03-17 14:07:00 +01:00
Akihiko Odaki
338bff8b93 Correct the reference to user's password in mastodon:add_user task (#6800) 2018-03-17 13:27:19 +01:00
Eugen Rochko
2ef1ce1182
Bump version to 2.3.1 2018-03-13 15:50:15 +01:00
Eugen Rochko
79d3a8553f
Bump version to 2.3.1rc3 2018-03-13 09:21:21 +01:00
Ushitora Anqou
051b649628 Detailed SMTP setup (#6759)
* add detailed SMTP settings setup in mastodon:setup

* add localhost SMTP settings setup in mastodon:setup

* SMTP settings setup should exit after successful delivery of test mail
2018-03-12 21:41:26 +01:00
艮 鮟鱇
f5f165a5eb set SAFETY_ASSURED=1 of db:setup in mastodon:setup (#6758) 2018-03-12 16:21:48 +01:00
Eugen Rochko
f89ff65ec7
Bump version to 2.3.1rc2 2018-03-12 12:48:49 +01:00
Eugen Rochko
ab27dccba5
Bump version to 2.3.1rc1 2018-03-11 15:13:13 +01:00
Eugen Rochko
6e309f5e45
Bump version to 2.3.0 2018-03-09 12:59:58 +01:00
Eugen Rochko
e5f18ace2a
When inside Docker, output saved configuration during mastodon:setup (#6711) 2018-03-09 11:52:18 +01:00
Eugen Rochko
bd077ad7d9
Bump version to 2.3.0rc3 2018-03-08 11:19:02 +01:00
Eugen Rochko
64db9ed5f6
After blocking domain with reject_media, invalidate cache (#6679)
Media attachments are part of the association cache of statuses,
since they are presumed to be immutable. Unless this cache is
cleared manually, the statuses will continue to look like they
have media embedded.
2018-03-08 06:59:42 +01:00
Eugen Rochko
4ca60c665e
Bump version to 2.3.0rc2 2018-03-07 12:06:23 +01:00
Eugen Rochko
78d772af86
Fix #3807: Increase avatars to 400x400 max (#6651)
But do not upscale when they are smaller
2018-03-06 06:29:01 +01:00
Akihiko Odaki
778b37790b Do not fetch environment variables to determine default locale (#6618)
The default locale is now set by config.
2018-03-04 10:00:46 +01:00
Thomas Leister
ecd36c1ede Fixes #6584 (#6585) 2018-03-01 23:30:06 +01:00
Eugen Rochko
64f2ada5d4
Bump version to 2.3.0rc1 2018-03-01 20:50:23 +01:00
Eugen Rochko
47bdb9b33b
Fix #942: Seamless LDAP login (#6556) 2018-02-28 19:04:53 +01:00
Paul Woolcock
76198c63b6 Some images can cause convert to fail, which crashes this whole task (#6565)
* Some images can cause `convert` to fail, which crashes this whole task

* Add more specific exception
2018-02-26 22:01:49 +01:00
Eugen Rochko
5cc716688a
Ensure the app does not even start if OTP_SECRET is not set (#6557)
* Ensure the app does not even start if OTP_SECRET is not set

* Remove PAPERCLIP_SECRET (it's not used by anything, actually)

Imports are for internal consumption and the url option isn't even
used correctly, so we can remove the hash stuff from them
2018-02-26 01:31:44 +01:00
Eugen Rochko
d3a62d2637
Fix #6525: Make sure file is opened in LazyThumbnail processor (#6529) 2018-02-22 00:28:19 +01:00
Eugen Rochko
61ed133fea
Account archive download (#6460)
* Fix #201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue
2018-02-21 23:21:32 +01:00
takayamaki
f69d7cb43b fix purge_removed_accounts task should suspend account before delete it (#6521) 2018-02-21 16:30:46 +01:00
Eugen Rochko
a7171af0a3
Fix avatar and header issues by using custom geometry detector (#6515)
* Fix avatar and header issues by using custom geometry detector

Revert a part of #6508. The file passed to dynamic styles method
was not actually a file, but an instance of Paperclip::Attachment,
which broke all styles by always returning {} from the method.

One problem with GIF avatars was that Paperclip::GeometryDetector
reported wrong dimensions for them, e.g. 120x120 GIF avatar would
for some reason be detected as 120x53. By writing our own geometry
parser, we can use FastImage, which also happens to be faster than
ImageMagick, to detect image dimensions, which are also correct.

Unfortunately, this PR does not implement skipping a `convert`
entirely if the dimensions are already correct, as I found no easy
way to write that behaviour into Paperclip without rewriting the
Paperclip::Thumbnail class.

* Only invoke convert if dimension or format needs to be changed
2018-02-21 03:40:12 +01:00
Eugen Rochko
cf36d184f4
Interactive rake mastodon:setup task (#6451)
* Add better CLI prompt

* Add rake mastodon:setup interactive wizard

* Test db/redis/smtp configurations and add admin user at the end

* Test database connection even when database does not exist yet
2018-02-11 18:40:57 +01:00