Commit graph

31 commits

Author SHA1 Message Date
Automated
ee7e5cc4cb [Auto] Updated Upstream (Spigot)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Spigot Changes:
2474d93d SPIGOT-4462: Catch async chunk regenerate
2018-11-03 05:31:30 -04:00
Aikar
908300e2c0 Many major improvements to Async Chunk Loading
Fixes some bugs with urgent priority, improves priority all
around to optimize blocking chunk requests as much as possible.

fixes casing on the -Dpaper.maxchunkthreads to now be -Dpaper.maxChunkThreads

adds -Dpaper.genThreadPriority=3 -Dpaper.loadThreadPriority=4

lowering thread priorities will help ensure main has more
priority over chunk threads
2018-11-02 22:48:33 -04:00
Shane Freeder
d2ff9c662c Updated Upstream (Bukkit/CraftBukkit/Spigot)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
9d0221aa API to get client side view distance.
9be7f0ea SPIGOT-4395: Additions to PlayerBedEnterEvent.
01e534c6 Minor cosmetic cleanups to imports etc

CraftBukkit Changes:
96c461b3 API to get client side view distance.
e2785f4e Remove note about development build
a8000588 SPIGOT-4395: Additions to PlayerBedEnterEvent.

Spigot Changes:
117d4f7e Rebuild patches
2018-11-03 00:23:27 +00:00
Aikar
4b59bce95d Revert "Many major improvements to Async Chunk Loading"
This reverts commit aee814c4b3.
2018-11-02 09:58:16 -04:00
Aikar
aee814c4b3 Many major improvements to Async Chunk Loading
Fixes some bugs with urgent priority, improves priority all
around to optimize blocking chunk requests as much as possible.

fixes casing on the -Dpaper.maxchunkthreads to now be -Dpaper.maxChunkThreads

adds -Dpaper.genThreadPriority=3 -Dpaper.loadThreadPriority=4

lowering thread priorities will help ensure main has more
priority over chunk threads
2018-10-31 23:57:03 -04:00
Automated
bb35e6164c [Auto] Updated Upstream (CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

CraftBukkit Changes:
aed3aecb Make natural item dropping mimic Vanilla
2018-10-31 05:21:34 -04:00
Automated
07874655b3 [Auto] Updated Upstream (Bukkit/CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
98d3f031 SPIGOT-4376: Add draft BlockDropItemEvent

CraftBukkit Changes:
1057710a SPIGOT-4376: Add draft BlockDropItemEvent
2018-10-31 05:01:59 -04:00
Shane Freeder
7941af6fec Process chunks blocking main urgently (#1627)
A chunk load on the main thread will be added to the high priority queue, however, due to existing work on this queue, there was no guarantee that the load would occur within a reasonable amount of time, potentially causing a server to crash while waiting for a chunk in the queue

In order to counteract this, a new urgent priority has been added, allowing us to prioritize these tasks over standard chunk gen/loading

(#1625 #1615 #1575 #1558 (and probably more))
2018-10-28 14:40:30 +00:00
Zach Brown
f29440f891 Update upstream B/CB/S 2018-10-27 00:02:22 -04:00
Aikar
af53c8c59e Remove debug text
resolves #1606
2018-10-21 19:16:54 -04:00
Caleb Bassham
0c7c2af037 Fix NPE when setting a player's spectator target (#1596)
Fix NPE when setting a player's spectator target
2018-10-21 14:27:53 -05:00
Aikar
6ef36f5d24 Remove MOST Synchronization from Chunk Map
This will provide quite a major performance boost by avoiding
synchronizing on EVERY chunk lookup.

Synchronize, even without contention, incurs processor cache flushes.

Considering this is the 2nd hottest method in the code base, lets
avoid doing that...

Additionally, chunk conversion operations were occuring while
under synchronization which lead to deadlocks.

Now the conversion will occur outside of the lock, and fix
that issue, resolving #1586

Note, that the chunk map is still thread safe for get operations!

The chunk map was never intended to be modified async with our
changes, as we post to main to modify the map, however
we do still synchronize for write operations (put, remove)

We also synchronize for async get operations, ensuring that
async gets are safe.

We do not need to synchronize main thread gets as the processor
cache will be insync since the map is only updated on the main thread.

However, if someone does try to delete or put concurrently, we
will force their operation back to the main thread.
2018-10-18 22:41:23 -04:00
Andrew Steinborn
344d299d69 Add Velocity IP forwarding support (#1557)
While Velocity supports BungeeCord-style IP forwarding, it is not secure. Users
have a lot of problems setting up firewalls or setting up plugins like IPWhitelist.
Further, the BungeeCord IP forwarding protocol still retains essentially its original
form, when there is brand new support for custom login plugin messages in 1.13.

Velocity's modern IP forwarding uses an HMAC-SHA256 code to ensure authenticity
of messages, is packed into a binary format that is smaller than BungeeCord's
forwarding, and is integrated into the Minecraft login process by using the 1.13
login plugin message packet.
2018-10-18 20:44:59 -04:00
Shane Freeder
442538c84f Share the main thread queue for AsyncChunkProvider
fixes an issue in which thread requests are only processed
for the current provider which can cause a deadlock should
multiple requests exist across providers
2018-10-12 15:41:15 +01:00
Aikar
3fe4a2dd2f Improvements to Async Chunks
If a chunk load comes in on a chunk load or gen thread,
execute it synchronously on that thread instead of enqueueing it.

It doesn't make sense to enqueue it as that thread is then
going to future.join() it and block until it's ready anyways.

This opens risk to a deadlock if every load or gen thread is
waiting on a recursive chunk but it will never finish because
all of the threads are waiting.
2018-10-10 22:48:35 -04:00
Aikar
ed0d75e389 Identify corrupt region file data and self recover
If we identify an invalid offset (negative, or the header sectors),
then back up the region file and erase that specific chunks offset
data.

This will avoid crashing the server with AIOBB errors and also avoids
server owners having to consider the entire region file 'lost'.

I'm not sure what leads to this state, I can only assume write cut
off mid bits.

In this scenario, there is absolutely no way to know where the chunk
actually is in the data file without loading every
single chunk in the file. And even to do that, would be quite extreme
due to the fact the file isn't in some orderly fashion.

Since the file is backed up, the user can use a region fixer tool
externally to try to restore that single chunk. We could even
add a command to restore a chunk from a backup file in a different
commit later on. But this at least prevents the server from crashing.

The server will just generate a new chunk and move on,
after printing an error to the console about it.

Also fixed the case reported in this issue about the server
hanging when a corrupt chunk is encountered, so this issue
is now fully closed.

Resolves #1541
2018-10-10 21:28:12 -04:00
Aikar
3e600507c8 Complete future when chunk fails to load 2018-10-08 21:14:13 -04:00
Aikar
efb4b6ef8b Improve synchronization on chunk gen to not block main - Fixes #1550
Chunk Generation was occuring while inside of the progressCache lock
this caused the progressCache to stay blocked for a long period of time
which then blocked main when main needed to clean the expiring map.

We now maintain a separate map for pending scheduler entries, that
we can join on if a 2nd request comes in while one is starting.

This strategy keeps the lock only for a fraction of time but
maintains thread safety.

So now the chunk is generated without holding a lock and wont
block the main thread on the expiring map as we will insert it
once ready.
2018-10-08 00:45:04 -04:00
Aikar
9f3623d30d Improvements to Logging Warnings/Dupe Entities - Resolves #1544
1) Removed "Regen" mode of Dupe UUID resolver, forced safe.
Some servers who updated before we had safe mode added still had this value.

There's really no reason to keep this mode, as we've seen that vanilla
triggers this often and 99.9999999% of cases will be an actual duplicate
that needs to be deleted.

2) Made Vanilla Debug messages about dupe UUIDs and dupe uuid resolve messages
only show up if the debug.entities flag is on. This will stop server owners
from panicing from seeing these logs, and stop opening bug reports on this,
only for us to tell you "don't worry about it".

3) Avoid adding entities to world that are already added to world.

This can be triggered by anything that causes an entity to be added
to the world during the chunk load process, such as chunk conversions.

Issue #1544 was a case of this.

4) Removed debug warning about ExpiringMap.

Nothing more I know to do about this anyways. We recover from it,
stop warning to reduce noise of issues to us.
2018-10-07 14:58:53 -04:00
Aikar
add0f4d4de Fix many light bugs by removing recheckGaps change
Fixes #1539
Fixes #1483
2018-10-06 00:56:20 -04:00
Aikar
fd85b785ea Don't stop chunk executors, cancel pending chunks instead
Fixes #1532
2018-10-06 00:15:40 -04:00
Aikar
43535ccedc Cancel pending chunk loads when they are no longer needed
This will improve queue times by canceling chunks when a player
leaves the radius of an async loading/generating chunk.

This matches behavior we had pre 1.13 for loading too.
2018-10-06 00:02:09 -04:00
Aikar
3058a042a5 Add config options to paper.yml for Async Chunks and Shared Host Overrides
Shared Hosts have been concerned over the increased CPU usage that Async Chunks
has introduced. We understand that shared hosts can't allow 1 client
to abuse the systme resources depriving other servers of cpu time, so we've
added some configs to allow hosts to force some settings.

Hosts, you may set the following SYSTEM ENVIRONMENT VARIABLES (NOT JVM FLAGS)
to change some behavior:

PAPER_ASYNC_CHUNKS_SHARED_HOST_GEN=1   - Enable Single Threaded World Generation (capping CPU Usage
PAPER_ASYNC_CHUNKS_SHARED_HOST_GEN=2   - Disable Async Chunk Generation - Please do not do this!!!
  - Minecraft is moving towards asynchronousm per-world chunk generation itself. You need to prepare
  - for this, so please do not disable async chunk generation.

PAPER_ASYNC_CHUNKS_SHARED_HOST_LOAD=#  - Where # is Maximum number of threads to use on Chunk Loading
  - This is a max, so your end user can reduce it more too. I suggest no lower than 2. Minimum is 1.
  - These are usually much faster request, so outside of loading entire worlds, They usually do not
  - get as much load. Paperby default uses CPU Core count * 1.5, so this may default to a higher value
  - on shared host and you should override it with this value to say 2-4.

These settings will override your clients paper.yml options. Set as a system wide environment
variable to ensure all Paper clients respect your resource usage desires.

Resolves #1520
2018-10-02 23:42:29 -04:00
Aikar
4899a1eb93 Disable use of Locks on main thread for DataPaletteBlock
This should greatly improve performance by using a No Op lock
while on the main thread.

Vanilla always had a write lock on write operations, but we added
a Read Lock during Async Chunks to make concurrent writes non fatal
for Async Chunks.

This means we added on a bunch of over head to all chunk read operations.

This corrects that, as well as disabling the write lock while on main thread.

It is a general rule that you do not touch a chunk async once it is loaded
into the world, as we never had locks on the chunk before 1.13 even.

So once we are on main, we don't expect concurrent access to begin with,
so we don't need the write locks either.
2018-10-02 22:29:31 -04:00
Shane Freeder
fc0af24140 Updated Upstream (Bukkit/CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
c71bb9ca Add PlayerRecipeDiscoverEvent and methods to (un/)discover recipes

CraftBukkit Changes:
7a2f4867 Implement PlayerRecipeDiscoverEvent and methods to (un/)discover recipes
2018-10-02 11:01:56 +01:00
Aikar
e7ff2c2126 Fix Async Chunk Load Callback if chunk was already loaded
Fixes ACL API mainly, but might fix other smaller things too

Also improved the high priority check if the chunk was already marked done
2018-10-01 01:00:09 -04:00
Aikar
065868036c Optimize Spare Chunk loads to be removed faster
this has technically been a longer standing problem, but if an async
chunk loads after a chunk has been removed from the chunk map, it would
be treated as any other spare chunk and kept loaded until Chunk GC kicks in.

This fixes that, but also obsoletes ChunkGC in that anytime we load a spare
chunk (a chunk outside of any players view distance), we will immediately
mark it for unload.

This should reduce the amount of spare chunks loaded on a server.
2018-09-30 20:34:05 -04:00
Aikar
1b331d8534 Fix bug in async load order priority
accidently had priority inverted!

This will greatly improve visual chunk load speeds
2018-09-28 23:12:16 -04:00
Aikar
d10ea572de Updated Upstream (Bukkit/CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
39ce5d3a SPIGOT-4399: ItemMeta.equals broken with AttributeModifiers

CraftBukkit Changes:
1cf8b5dc SPIGOT-4400: Populators running on existing chunks
116cb9a1 SPIGOT-4399: Add attribute modifier equality test
5ee1c18a SPIGOT-4398: Set ASM7_EXPERIMENTAL flag
2018-09-28 19:31:59 -04:00
Aikar
16846b782c Updated Upstream (Bukkit/CraftBukkit/Spigot)
Upstream has released updates that appears to apply and compile correctly.

This update has been tested to ensure that World Conversion still occurs correctly.

Bukkit Changes:
0812ce2c SPIGOT-4397: isChunkGenerated API

CraftBukkit Changes:
4824655c SPIGOT-4398: Upgrade to ASM 6.2.1 for better Java 11 support
eea43870 MC-134115: Fix issues converting tile entities
1a7f2d10 SPIGOT-4397: isChunkGenerated API
40aed54d SPIGOT-4396: Improve vehicle movement

Spigot Changes:
f6a273b1 Rebuild patches
2018-09-26 22:35:42 -04:00
Daniel Ennis
58a9ea2926 Async Chunk Loading and Generation (#1397)
This restores Asynchronous Chunk Loading to 1.13, as well as adds Asynchronous Chunk Generation..

This provides a massive boost to server performance and should bring 1.13 back closer to 1.12 performance levels for pre generated worlds.

World Generation will now be faster overall than 1.12 since 1.12 did not have async generation.


**DONATE:**
Lots of effort has went into making this happen. Want to say thanks and donate? 

https://donate.emc.gs/AsyncChunks
2018-09-26 18:21:44 -04:00