Commit graph

2700 commits

Author SHA1 Message Date
Aikar
5201efc608 Add more information to Timing Reports
Still needs front end changes to see it yet though.

1) Adds Game Rules per world
2) Adds View distances per world
3) Removes extra garbage on lambda task names
4) Adds more memory information such as native load
5) Adds load average for non crap operating systems.
6) Fixes online mode showing false when privacy=true
7) Adds Data packs loaded
2020-06-02 23:26:29 -04:00
Aikar
9ad7e2fe15 Improve Thread Pool usage to allow single threads for single cpu servers
Switch to a standard fixed size ThreadPoolExecutor as we don't use the
advanced capabilities of a ForkJoinPool.

ForkJoinPool does not allow single threads, and really rather not use
2 different executor types based on core count.

Also, change thread priorities so that main thread is prioritized by
the OS at a higher priority than the other threads. May not help too much
but it at least signals the OS the information to know main is more important.
2020-06-02 02:19:07 -04:00
Aikar
55e3e3d0f8 Fix sounds when using worldedit regen command
Locks dimension manager to the first world its used with.

WE is creating a temp world and the world ref on that manager
is getting changed to the temp world.

This would of also caused a memory leak of that temp world too.
2020-06-01 23:51:37 -04:00
Aikar
eb4300d9e1 Updated Upstream (Bukkit/CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

My recent work on serialization is now in CraftBukkit so was able to drop the patch and Paper
is now consistent with upstream.

Bukkit Changes:
e2699636 Move API notes to more obvious location

CraftBukkit Changes:
1b2830a3 SPIGOT-4441: Fix serializing Components to and from Legacy
2020-06-01 23:15:47 -04:00
Aikar
4c4b8f4d3e Improve legacy format serialization more
This should now complete legacy serialization to avoid ever
changing the output content.

This removes the concept of "Default Color" from the method as
that entire concept was flawed and broke the intent of chat components.

Going to actually PR this patch to Spigot soon.

This now puts us back at a point where any data saved pre Spigot
breaking things will still save back the exact same way as before,

but new component -> legacy will now be fixed to not insert undesirable
default colors (such as black) into the legacy string, and instead use
the proper reset code.

This means you can now safety get the text from a book and
put it in chat or an entity display name without worry about black
color codes or other undesired color codes leaking into the new
context where that color doesn't make sense.
2020-05-31 22:10:49 -04:00
Aikar
9edaa0fc34 Restore Serialization Improvements again, wasn't an issue.
I was a little quick to jump to blame this patch as cause of an issue,
but appears it was not the source for what I saw that made me revert this.
2020-05-31 05:01:26 -04:00
Aikar
1545cbf338 Revert "Improve Chat Component Legacy Serialization more"
This reverts commit 8e7099030e.

Seems to be having undesired impact, will need to polish more.
2020-05-31 04:08:03 -04:00
Aikar
8e7099030e Improve Chat Component Legacy Serialization more
This brings chat componenent serialization to 100% accuracy so
that any text input in the legacy format, converting to comps and
then back to legacy will result in identical results.

If the user explicitly sets a color as prefix to a string, it is retained,
even if that color matches the default.

This also helps improve dealing with the empty string wrappers Bukkit creates.

A unit test has been added to verify this behavior.
2020-05-31 03:20:31 -04:00
wea_ondara
c404f38837 Fix serialization of colors from components
This patch fixes the serialization of display names, item lores and
other things which use strings with color codes. The old implementation
deleted the color codes at the beginning of the resulting string if it
matched the default color passed to the conversion function. This
resulted in items having a black display name losing the black color
code in the beginning of the text when the item was serialized (e.g.
saving an ItemStack in a Yaml config).

Spigot has now made the issue worse and expanded the scope to more places.
2019-11-02 22:27:02 +01:00
Aikar
09e2cbb497 Add Villager Tasks to EAR inactive tick to keep behavior
Stuff like restocking trades and reputation changes can happen when
a Villager is out of Activation Range now.
2020-05-30 04:54:40 -04:00
Aikar
80de5ede6f Improve Chunk Prioritization / Load Order
1) Improve frustum to look more at the near chunks and frontal chunks only instead of 1 large single look up.
2) Delay adding 33 tickets based on view distance and lower their task priority. This will slower roll out the spiral
3) Chunks behind the player have additional delay on loading, favoring chunks in front of the player.

This has benefit that if faster traveling, some of the chunks will be cancelled / not loaded.

This should reduce pressure on chunk loading, as well as reduce loading/unloading unnecessary chunks while moving.
2020-05-30 02:53:47 -04:00
Aikar
b7fcf47d0d Improve Chunk Status Transition Speed
When a chunk is loaded from disk that has already been generated,
the server has to promote the chunk through the system to reach
it's current desired status level.

This results in every single status transition going from the main thread
to the world gen threads, only to discover it has no work it actually
needs to do.... and then it returns back to main.

This back and forth costs a lot of time and can really delay chunk loads
when the server is under high TPS due to their being a lot of time in
between chunk load times, as well as hogs up the chunk threads from doing
actual generation and light work.

Additionally, the whole task system uses a lot of CPU on the server threads anyways.

So by optimizing status transitions for status's that are already complete,
we can run them to the desired level while on main thread (where it has
to happen anyways) instead of ever jumping to world gen thread.

This will improve chunk loading effeciency to be reduced down to the following
scenario / path:

1) MAIN: Chunk Requested, Load Request sent to ChunkTaskManager / IO Queue
2) IO: Once position in queue comes, submit read IO data and schedule to chunk task thread
3) CHUNK: Once IO is loaded and position in queue comes, deserialize the chunk data, process conversions, submit to main queue
4) MAIN: next Chunk Task process (Mid Tick or End Of Tick), load chunk data into world (POI, main thread tasks)
5) MAIN: process status transitions all the way to LIGHT, light schedules Threaded task
6) SERVER: Light tasks register light enablement for chunk and any lighting needing to be done
7) MAIN: Task returns to main, finish processing to FULL/TICKING status

Previously would have hopped to SERVER around 12+ times there extra.
2020-05-30 01:12:18 -04:00
Aikar
7ffde5cb24 Synchronize DataPaletteBlock instead of ReentrantLock
Mojang has flaws in their logic about chunks being concurrently
wrote to. So we constantly see crashes around multiple threads writing.

Additionally, java has optimized synchronization so well that its
in many times faster than trying to manage read wrote locks for low
contention situations.

And this is extremely a low contention situation.

Fixes #3293
Fixes #2493
2020-05-29 20:36:42 -04:00
Aikar
47af5acd7a Drop remove leaked chunk patch - causing many issues
I'm hoping the other fix in 324 for the level map getting corrupted
fixes the real issue and this isn't needed anymore, but i suspect it is

will wait until more study can be done though.

Fixes #3469
2020-05-29 03:30:58 -04:00
Aikar
50557cebdb Fix premature unloading of neighbor chunks for world gen
We must check the level tracker as ticket levels add "virtual"
tickets to neighbors.

Also added neighbor tracking during generation to be extra safe.

Fixes #3465
Fixes #3451
Fixes #3459
2020-05-28 22:59:40 -04:00
Aikar
0ef7812813 Fix Chunks unloading immediately for priority access - Fixes #3461
Adding a priority ticket was apparently causing the level tracker
to immediately unload the chunk.

That code is scary
2020-05-28 22:57:27 -04:00
Aikar
8e9b42b127 Fix some logic on villager poi max attempts
Hopefully will help #3462
2020-05-28 17:04:20 -04:00
Aikar
6bff219f15 Fix concurrency issue in light engine (Vanilla bug)
Mojang implemented a cache like chunks have, but this cache
is accessed by multiple threads and is totally not safe.

So just remove it

Fixes #3466

Also missed a pooled nibble release, so slid that in there too.
2020-05-28 16:59:38 -04:00
Spottedleaf
db127b6f9c Cleanup async chunks (#3456)
Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com>
2020-05-28 16:34:58 -07:00
Aikar
131b67c38a Optimize Villager Pathfinding - Massive Villager Improvement
This change reimplements the entire BehaviorFindPosition method to
get rid of all of the streams, and implement the logic in a more sane way.

We keep vanilla behavior 100% the same with this change, just wrote more
optimal, as we can abort iterating POI's as soon as we find a match....

One slight change is that Minecraft adds a random delay before a POI is
attempted again. I've increased the amount of that delay based on the distance
to said POI, so farther POI's will not be attempted as often.

Additionally, we spiral out, so we favor local POI's before we ever favor farther POI's.

We also try to pathfind 1 POI at a time instead of collecting multiple POI's then tossing them
all to the pathfinder, so that once we get a match we can return before even looking at other
POI's.

This benefits us in that ideally, a villager will constantly find the near POI's and
not even try to pathfind to the farther POI. Trying to pathfind to distant POI's is
what causes significant lag.

Other improvements here is to stop spamming the POI manager with empty nullables.
Vanilla used them to represent if they needed to load POI data off disk or not.

Well, we load POI data async on chunk load, so we have it, and we surely do not ever
want to load POI data sync either for unloaded chunks!

So this massively reduces object count in the POI hashmaps, resulting in less hash collions,
and also less memory use.

Additionally, unemployed villagers were using significant time due to major ineffeciency in
the code rebuilding data that is static every single invocation for every POI type...

So we cache that and only rebuild it if professions change, which should be never unless
a plugin manipulates and adds custom professions, which it will handle by rebuilding.
2020-05-27 01:23:19 -04:00
Aikar
f4452c380d Workaround for really evil plugins breaking sounds
Some plugins are doing really really bad things to worlds breaking the
ability to send sounds to some users.

So creating another reference to the player chunk map that plugins wont be breaking, and
print a stack trace at world creation if we ever get an expected world state to identify
who is doing it!

If we encounter this illegal state, we fall back to the old method of sending sounds, so
sending sounds will still work, just less effecient.
2020-05-27 00:47:20 -04:00
Aikar
21d2cda885 Don't unregister the primary dimensions on world unload 2020-05-27 00:15:57 -04:00
Aikar
a73ee9f448 Improve implementation of chunk leak clean to not run as often 2020-05-25 21:42:53 -04:00
Aikar
1b23cf5819 Add missing null check for structure start chunk access
Spigot made structure start not load chunks, but forgot to null check
the result...

This likely never blew up before due to the chunk leak issue, but now
that leaky chunks are cleaned up, it was identified.
2020-05-25 14:19:11 -04:00
Aikar
db9534bfdf One more to seal the deal to improve unloading inactive chunks
While last was mostly there, still had some slight risk of unloading
before it was fully finished.

So just going to bump the delay to 3 minutes to be safe. Better than
forever at least.

Was really hoping we could unload them as soon as they were done to
any memory prematurely promoting to old generation, but guess we can't.
2020-05-25 13:09:56 -04:00
Aikar
cb1f733303 Restore a condition I accidently removed in last build 2020-05-25 12:53:40 -04:00
Aikar
6480eba1cc Fix unloading inaccessible chunks too fast regressing gen speed
A chunk was loaded but not yet finished in use and was unloaded too early.

This caused it to be reloaded again or caused crashes.

Now also check if the chunk pops out of the unload queue that it also
doesn't now have a ticket either.
2020-05-25 12:34:03 -04:00
Aikar
07ab0369ac Unload leaked Cached Chunks
Due to some complexity in mojangs complicated chain of juggling
whether or not a chunk should be unloaded when the last ticket is
removed, many chunks are remaining around in the cache.

These chunks are never being targetted for unload because they are
vastly out of view distance range and have no reason to be looked at.

This is a huge issue for performance because we have to iterate these
chunks EVERY TICK... This is what's been leading to high SELF time in
Ticking Chunks timings/profiler results.

We will now detect these chunks in that iteration, and automatically
add it to the unload queue when the chunk is found without any tickets.
2020-05-25 11:12:22 -04:00
Aikar
3cf6f06a21 Fix Spigot bug with chunk unloading
Spigot inserted their Slack Activity Accountant in the wrong location
resulting in a chunk being removed from the unload queue, inserted into
the unload map, but never calling the function to finish the removal....

This caused the chunk to become stuck in the unload map if ever hit, because
the unload map was meant to be a TEMPORARY location while it was saving.

Fix this by abort iteration AFTER the current chunk is finisehd processing

Also, improve how aggressive we are at unloading chunks, targetting 10% per tick instead.
These saves are asynchronous so there should be less of a hit here.
2020-05-25 11:08:11 -04:00
Aikar
8833b1ba15 Preload important classes such as Logger and JLine
This is for 2 reasons:
1) Ensuring our log4j is mostly loaded at OUR version.
   I've seen stack traces with line numbers that do not match our version. This means that some
   plugin has shaded in log4j and their loaded version is mixing with ours....
   So by at least trying to load a bunch of log4j classes before we load plugins, we can be
   more sure mixed versions are not loading.

2) If the jar file is replaced while the server is runnimg class not found errors galore
   This will preloaod a bunch of classes commonly seen to error during shutdown due to this.

   The goal here is to help let the server shutdown gracefully as possible. Some plugins will
   still blow up here if they access a class that hadn't been loaded yet, but goal is to at least
   stop freezing the shutdown process as it does with JLine and Log4j errors requiring an external kill.

   Ideally you should not replace jars while the server is running, but it is something that happens in
   development for testing.

Updated test server to do a copy though to avoid this happening in Paper development.
2020-05-24 13:09:02 -04:00
Aikar
4657db9347 Fix netty version to 50 and restore epoll native support
Accidently used the snapshot, as well as needed to exclude the
existing old 25 build of libnetty so that we load the newer build of
epoll instead.

Thanks to 56738 for the catch.
2020-05-24 13:07:53 -04:00
Aikar
41d5f3cc86 Workaround for some hacky environments that mess up things
2 people had issues where some plugin is doing some reallly insane NMS hackery
that created invalid worlds, which caused some errors...

Really don't understand what in the world they did, but putting in a dumb guard that
shouldn't even be necessary to just not send the sound effect rather than erroring.
2020-05-23 22:27:37 -04:00
Aikar
18bd266258 Fix plugins calling getChunkAtAsync asynchronously
While this method has async in it's name, it's not actually meant
to be called asynchronously.... It just means IT will load the chunk
asynchronously without blocking main.

So fix this so that if a plugin calls it async, it forces the request back to main thread.
2020-05-23 18:38:29 -04:00
Aikar
238ab72fce Update Netty to 4.1.51
Minecraft's Netty version was severely out of date. There has been
numerous security fixes, bug fixes, and likely performance fixes
since the version Minecraft uses (4.1.25).

This fixes some known issues with "Closed Channel" spam.

Fixes #3388
2020-05-23 17:35:27 -04:00
Aikar
bfb789c862 Optimize sending packets to nearby locations (sounds/effects)
Instead of using the entire world or player list, use the distance
maps to only iterate players who are even seeing the chunk the packet
is originating from.

This will drastically cut down on packet sending cost for worlds with
lots of players in them.

Closes #3437
2020-05-23 17:15:40 -04:00
Aikar
4ec22a4736 Fix Non Full Status Chunk NBT Memory Leak
Any full status chunk that was requested for any status less than full
would hold onto their entire nbt tree and every variable in that function.

This was due to use of a lambda that persists on the Chunk object
until that chunk reaches FULL status.

With introduction of no tick, we greatly increased the number of non
full chunks so this was really starting to hurt.

We further improve it by making a copy of the nbt tag with only the memory
it needs, so that we dont have to hold a copy to the entire compound.
2020-05-23 01:40:27 -04:00
Aikar
78d38c0d3c Update Log4J Configuration file to stop truncating stack traces
This should help greatly (as long as this change works...) in
understanding an exception when it doesn't get truncated with
"... and 14 more" at a vital point of the stack trace.
2020-05-22 23:49:10 -04:00
Aikar
5978d3cdd8 More Improvements to Chunks
Fixed issues where urgent and prioritized chunks didn't actually
always get their priority boosted correctly....

Properly deprioritize non ticking chunks.
Limit recursion on watchdog prints to stop flooding as much
Remove neighbor priorities from watchdog to reduce information
reduce synchronization duration so that watch dog won't block main should main actually wake up
probably fixed a deadlock risk in watchdog printing also that was leading to crashes
fixed chunk holder enqueues not being processed correctly
added async catchers in some locations that should not be ran async

Fixed upstream bug where VITAL callbacks that must run on main actually could
sometimes run on the server thread pool causing alot of these nasty bugs we've seen lately!

This build will provide massive improvements to stability as well as even faster
sync chunk load/gens now that priority is correctly set.

Fixes #3435
2020-05-22 19:03:48 -04:00
Aikar
09a640626a Fix high memory use of non ticking chunks
The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks
as we store the NBT Tag of Tile Entities in a Chunk, but don't process
them and remove them until chunk reaches Entity Ticking status....

This caused some phantom references to persist causing high memory use
of these chunks.

So I just got rid of pooling from NBT deserialization and we'll have to
take the hit on memory allocations there because too many cascading concerns
with anyone using NBT Tag Byte Arrays.

Fixes #3431
2020-05-22 18:58:26 -04:00
Aikar
f3e13bf42c Fix another case of breaking blocks causing sync chunk loads 2020-05-22 18:57:48 -04:00
Aikar
12ecaf1372 Drop AABB limit patch until it can be tested more
Fixes #3428
2020-05-22 02:21:32 -04:00
Aikar
4ca2ccc422 Fix log spam about Hanging entities bounding boxes
Nothing really "Broke" them, just a lot of log spam. This will clean that up
2020-05-22 02:04:00 -04:00
Aikar
0110ef1aa1 Fix conversion for deserializing raw nbt itemstacks - Fixes #3424 2020-05-22 01:08:18 -04:00
Aikar
ad73c0eb38 Many fixes and improvements to chunk prioritization
I believe this brings us back to stable. A lot of complexity was
learned about juggling priorities.

We were essentially promoting more chunks to urgent than really
needed to be urgent.

So this commit adds a lot more logic to juggle neighbor priorities
and demote their priority once they meet the requirements needed of
them.

This greatly improves the performance of "urgent" chunks".

Fixes #3410
Fixes #3426
Fixes #3425
Fixes #3416
2020-05-22 00:46:44 -04:00
Aikar
f04209258d Use saner Entity bounding box limits
CB only protected from > 64 but there's no reason an entity should ever
be more than 2x its width or 1x height as the BB is supposed to represent
the entity size.

BB is / 2 to calculate position.
2020-05-22 00:40:56 -04:00
Aikar
690f01f604 Protect the visible chunk map from plugins touching it, trim Timing Errors
Blow up if a plugin tries to mutate visibleChunks directly and prevent them
from doing so.

Also provide a safe get call if any plugins directly call get on it so
that it uses the special logic to check pending.

Also restores ABI for the visibleChunks field back to what it was too.

Additionally, remove the stack trace from Timings Stack Corruption for any
error thrown on Minecraft Timings, and tell them to get the error ABOVE this
instead, so people stop giving us useless error reports.

Also fixes a memory leak when the source map down sizes but dest map didn't,
which resulted in lingering references to old chunk holders.

Fixes #3414
2020-05-20 22:22:47 -04:00
Aikar
d4d21be955 Optimize performance of object pool
synchronized arraydeque ends up still being way faster.
Kinda shocked how much that strategy was using, it wasn't really
that complicated... but oh well, this is even simpler and not
seeing blocked threads show up at all in profiling because
the lock is held for such a short amount of time.

also because most uses are on either server thread pool or chunk load pool.

Also optimize the pooling of nibbles to not register Cleaner's
for Light Engine directed usages, as we know we are properly
controlling clean up there, so we don't need to rely on GC.

This will return them to the pool manually, saving a lot of Cleaners.

Closes #3417
2020-05-20 21:45:43 -04:00
Aikar
33246e0ba0 Many improvements to chunk prioritization and bug fixes
Fixed a few bugs, and made numerous improvements.
Fixed issue where a sync chunk load could have its ticket removed and the
priority ticket could expire...
Still not perfect there but better than before.

Also fixed few other misc issues such as watchdog cpu usage, chunk queue update
had risk of double enqueue due to it no longer being a set.

Added much more information about chunk state to watchdog prints.

I see some more room for improvement even, but this is much better than before.

Fixes #3407
Fixes #3411
Fixes #3395
Fixes #3389
2020-05-20 05:11:57 -04:00
Aikar
ade297307f Fix pooled buffer leak resulting in dynmap black spots - Fixes #3386
Dynmap accessed the raw bytes because it utilized NBT locally, but the
NBTTagcompound was garbage collected while the bytes were still being used.

This will return getBytes() back to being safe, and add a new PoolSafe method
that will prevent the additional allocations for general chunk loading.

Also fixed applyPatches for people with paths in their working directory
if they have mcdev sources built.
2020-05-20 00:51:28 -04:00
Aikar
e14f7e171f Implement Chunk Priority / Urgency System for Chunks
Mark chunks that are blocking main thread for world generation as urgent

Implements a general priority system so that chunks that are sorted in
the generator queues can prioritize certain chunks over another.

Urgent chunks will jump to the front of the line, ensuring that a
sync chunk load on an ungenerated chunk does not lag the server for
a long period of time if the servers generator queues are filled with
lots of chunks already.

This massively reduces the lag spikes from sync chunk gens.

Then we further prioritize loading order so nearby chunks have higher
priority than distant chunks, reducing the pressure a high no tick
view distance holds on you.

Chunks in front of the player have higher priority, to help with
fast traveling players keep up with their movement.

This commit also improves single core cpu scenarios in that we will
now automatically disable Async Chunks as well as Minecrafts thread
pool.

It is never recommended to use async chunks on a single CPU as context
switching will be slower than just running it all on main.

This also bumps the number of server worker threads by default too.
Mojang does not utilize the workers in an effecient manner, resulting
in them using barely any sustained CPU.

So give it more workers so more chunks can be processed concurrently

This change also improves urgent chunk loading, so players flying into
unloaded chunks will hurt a little bit less (but still hurt)

Ping #3395 #3363 (Not marking as closed, we need to make prevent moving work)
2020-05-19 04:01:53 -04:00
Mariell
b4f1f6fc59 Fix data version check for ItemStack serialization (#3394)
The expected version should be equal to or newer than the one stored.

Although Aikar claims he did this on accident (and NOT my ligatures!), I
claim this is all a big conspiracy by followers of the Taco cult.
2020-05-18 19:24:06 +02:00
Mariell Hoversholm
96bb9740fa Add Raw Byte ItemStack Serialization
Serializes using NBT which is safer for server data migrations than bukkits format.
2020-04-30 16:59:24 +02:00
Spottedleaf
aba7b389ab Allow server startup for those poor people running <1G Xmx
Clamp logic for poolsize so we dont pass zero
2020-05-16 21:06:19 -07:00
Spottedleaf
ab609796e6 Fix cases where no-tick < tick view distance 2020-05-16 20:53:09 -07:00
MeFisto94
0ceb05a8fc Workaround for Client Lag Spikes (MC-162253)
When crossing certain chunk boundaries, the client needlessly
calculates light maps for chunk neighbours. In some specific map
configurations, these calculations cause a 500ms+ freeze on the Client.

This patch basically serves as a workaround by sending light maps
to the client, so that it doesn't attempt to calculate them.
This mitigates the frametime impact to a minimum (but it's still there).
2020-05-12 23:11:30 +02:00
Mariell Hoversholm
d994e4651c Add option for console having all permissions
Overrides permissions set to default false (which op does not get)
2020-05-16 10:13:01 +02:00
Mariell Hoversholm
dcd9ddf8e0 Add permission for command blocks 2020-05-16 10:06:05 +02:00
kickash32
0de6c6f86a Check portal restrictions when entering end gateways 2020-05-16 04:31:26 -04:00
Josh Roy
0219f4acd4 Fix NPE when temp ip bans expire (#3373) 2020-05-16 23:26:49 -04:00
Max Lee
acbe459f57 Add missing mob goals for API (#3367) 2020-05-17 04:19:24 +01:00
Mariell Hoversholm
ee99320810 Ensure no-tick view is not smaller than ticking VD
Fixes #3372.
2020-05-16 10:38:48 +02:00
MiniDigger | Martin
792993faca Expand Pathfinding API with more options 2020-05-16 15:15:11 +02:00
MiniDigger | Martin
8743a66ca2 Fix usage of vanilla goals 2020-05-16 15:25:40 +02:00
kickash32
77323d14ca Drop Leads from nether portals - Fixes #3226 2020-05-16 04:31:53 -04:00
Spottedleaf
459f6194e8 Reduce MutableInt and Vec3d allocations, use ArrayDeque 2020-05-16 22:05:14 -04:00
Aikar
378ac3e7f3 Optimize NibbleArray to use pooled buffers
Massively reduces memory allocation of 2048 byte buffers by using
an object pool for these.

Uses lots of advanced new capabilities of the Paper codebase :)

Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration
of 6GB of heap (any more over 6GB won't give more nibble pool)

You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048

Remember this number is * 8 then * heap memory in GB

That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB

You may also control max number of pooled objects directly instead of any
dynamic calculation using -DPaper.maxNibblePoolSize=1024000

While this will use more old generation by a tad bit, allocation rate will drop
significantly, causing less young generation GC's.

This commit has gone through extensive testing for over a day and confident
it no longer has any issues with light corruption.
2020-05-16 21:41:56 -04:00
Aikar
90072b811c Foundational work for Future Memory usage improvements
This commit doesn't do much on its own, but adds a new Java Cleaner API
that lets us hook into Garbage Collector events to reclaim pooled objects and
return them to the pool.

Adds framework for Network Packets to know when a packet has finished dispatching
to get an idea when a packet is done sending to players.

Rewrites PooledObjects impl to properly respect max pool size and remove
almost all risk of contention.

Bumps the Paper Async Task Queue to use 2 threads, and properly shuts it down on shutdown.
2020-05-16 21:38:19 -04:00
Spottedleaf
4e7644ce04 Handle CraftPlayer#setSpectatorTarget better
Use a proper teleport for teleporting to entities in different
worlds.

Validate that the target entity is valid and deny spectate
requests from frozen players.

Also, make sure the entity is spawned to the client before
sending the camera packet. If the entity isn't spawned clientside
when it receives the camera packet, then the client will not
spectate the target entity.
2020-05-14 18:05:00 -05:00
Aikar
7ead67800e Fix collision checks on spawning hanging entities and null on async chunk loads
getCubes will now always load chunks
getChunk with gen false will now not throw error

Fixes #3368
Fixes #3364
2020-05-14 04:58:47 -04:00
Aikar
49aa9c42db Protect Bedrock and End Portal/Frames from being destroyed
This fixes exploits that let players destroy bedrock by Pistons, explosions
and Mushrooom/Tree generation.

These blocks are designed to not be broken except by creative players/commands.
So protect them from a multitude of methods of destroying them.

A config is provided if you rather let players use these exploits, and let
them destroy the worlds End Portals and get on top of the nether easy.
2020-05-14 03:16:31 -04:00
Aikar
47f11f19a5 Updated Upstream (Bukkit/CraftBukkit/Spigot)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
ffc8e4ca SPIGOT-5716: Clarify documentation of MultipleFacing

CraftBukkit Changes:
d07a78b1 SPIGOT-5716: Clarify documentation of MultipleFacing
46a13860 SPIGOT-5718: Block.BreakBlockNaturally does not reflect tool used
214ffea9 SPIGOT-5727: GameRule doImmediateRespawn cannot be set per-world

Spigot Changes:
2f5d615f SPIGOT-5730: Modernise inventory patch
a2bdb119 SPIGOT-5679: Add config option for end portal activation sound

Closes #3352
2020-05-12 01:27:03 -04:00
Aikar
92497a1b5d Fix Pathfinding and obscure glitchy buggy 0 tick farms
I swear the crap that stuff will abuse to make stuff happen is insane.

Hash codes apparently changing behavior of stuff based on its value, so
reverting e9fcee1190

Fixes #3346
Fixes #3341
2020-05-11 21:18:11 -04:00
Aikar
c0441b6262 Optimize Pathfinder - Remove Streams / Optimized collections
I utilized the IDE to convert streams to non streams code, so shouldn't
be any risk of behavior change. Only did minor optimization of the
generated code set to remove unnecessary things.

I expect us to just drop this patch on next major update and re-apply
it with the IDE again and re-apply the collections optimization.

Optimize collection by creating a list instead of a set of the key and value.

This lets us get faster foreach iteration, as well as avoids map lookups on
the values when needed.
2020-05-11 04:24:38 -04:00
Aikar
f3def69920 Optimize Hoppers - Major performance improvement
Removed streams from hoppers and also fixed a mistake in the logic.
When this patch was ported to 1.14/1.15, a line of code was put in
the wrong place which disabled a significant portion of the improvement.

Replaced usages of streams in isEmpty and itemstack checks
Replaced usage of streams in pulling loop
Replaced usage of streams in Lootable Inventory isEmpty() check
Only check for refilling Lootable Inventory when accessing first slot, not all

All of these in general were pretty significant hits, so this single commit
is going to cause tacos to magically appear in front of you every day.

🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮🌮

Nom Nom Nom
If you hate taco's, you're not allowed to use this improvement.

Also ignore the renames, pulled a lot of PR's.
2020-05-11 03:42:32 -04:00
Omer Uddin
7506508641 Fix PotionEffect API Ignoring Icon bug
Fixes #3142
2020-05-11 00:44:27 -04:00
Omer Uddin
d116dbd6dc Fix EntityRaider picking up items when they shouldn't be able
Fixes #2205
2020-05-11 00:41:15 -04:00
Aikar
924d46ea01 Add a TELEPORT ticket when changing dimensions
This should fix any remaining issues around collision issues on
chunk edges when changing dimensions sending to Y 256
2020-05-10 23:58:04 -04:00
Aikar
80a570d89f Fix missing flag pass for isUrgent 2020-05-10 23:31:36 -04:00
Josh Roy
6df55d9c05 Potential bed api (#3339) 2020-05-11 00:19:04 -04:00
Spottedleaf
aff206ef81 Optimize WorldBorder collision checks and air
Port part of leafs changes from Tuinity, will worry about rest later with larger changes coming
2020-05-10 22:49:58 -04:00
Aikar
02dcaa0af7 Wait for Async Tasks during shutdown
Server.reload() had this logic to give time for tasks to shutdown,
however shutdown did not...

Adds a 5 second grace period for any async tasks to finish and warns
if any are still running after that delay just as reload does.

Closes #3337
2020-05-10 22:17:55 -04:00
Aikar
1c0dde5df5 Ensure Entity AABB's are never invalid
If anything used setPositionRaw, it left potential for an AABB
to be left stale at their old location, which could cause massive
AABB boxes if movement ever then got called on the new position.

This guarantees any time we set the entities position, we also
update their AABB.
2020-05-10 22:14:47 -04:00
Aikar
b74f95d9f9 Properly remove Entities from current chunk
We store a reference to the chunk the entity is currently in, so use it
to more accurately unregister it in chunkCheck

Should maybe fix some entity loss issues.
2020-05-10 05:52:31 -04:00
Aikar
a26aabf9d0 Fix teleporting onto a chunk line
Obscure detail in that if you teleport right on a chunk line, it
adds +1 to your collision check and will check the unloaded neighbor.

but the call to load the chunk then returned null if it was pending unload, such
as the load we did in Player List

However we want gen=true for players here anyways, so use getType
2020-05-10 05:36:35 -04:00
Aikar
ee714e312d Send LOGIN protocol packets immediately - Fix disconnect during async prelogin 2020-05-10 03:12:05 -04:00
Aikar
bd2a5095d3 Don't use our modified chunk checks for collision in world gen
Fixes issues with errors in logs on generating chunks
2020-05-10 02:26:21 -04:00
Aikar
5eea746c78 Allow loading entities current chunk if needed to fix collision checks
Having trouble reproducing this myself, but this should do it.
2020-05-10 01:04:45 -04:00
Aikar
56d8694bd9 Add Urgent API for Async Chunks API and use it for Async Teleport
This also cleans up the implementation of Async Chunks to get rid of most
Consumer callbacks and instead return futures.

This lets us propogate errors correctly up the future chain
(barring one isn't lost even deeper in the chain...)

So exceptions can now bubble to plugins using getChunkAtAsync
2020-05-09 22:30:28 -04:00
Aikar
04becc8a1c Ensure chunks loaded on respawn for suffocation check 2020-05-09 21:13:30 -04:00
Aikar
f36b2a2c05 Remove some Streams usage in Entity Collision
While there is more down the collision system, remove some of the wrapping
Spliterator stuff as even this wrapper stream has shown up in profiling.

With other collision optimizations, we might also even avoid inner streams too.
2020-05-09 19:18:26 -04:00
JellySquid
3229702834 Implement JellySquid's Entity Collision optimisations patch
This patch replaces the vanilla collision code for both block and entity collisions with faster implementations by JellySquid, used originally in her Lithium mod.

Optimizes Full Block voxel collisions, and removes streams from Entity collisions

Original code by JellySquid, licensed under GNU Lesser General Public License v3.0
you can find the original code on https://github.com/jellysquid3/lithium-fabric/tree/1.15.x/fabric (Yarn mappings)

Ported by
Co-authored-by: Zoutelande <54509836+Zoutelande@users.noreply.github.com>

Touched up by Aikar to keep previous paper optimizations
2020-04-22 17:11:50 +02:00
Aikar
f0b502d564 Optimize Collision to not load chunks
The collision code takes an AABB and generates a cuboid of checks rather
than a cylinder, so at high velocity this can generate a lot of chunk checks.

Treat an unloaded chunk as a collision for entities, and also for players if
the "prevent moving into unloaded chunks" setting is enabled.

If that setting is not enabled, collisions will be ignored for players, since
movement will load only the chunk the player enters anyways and avoids loading
massive amounts of surrounding chunks due to large AABB lookups.

Fixes #3321
2020-05-09 15:47:08 -04:00
Aikar
7c9097159c Fix ChunkCache .getXIfLoaded to use the local chunks in the cache
It was calling back out to world for it, we already have them.

Also use the nocache method for building the chunk cache too.
2020-05-09 14:58:08 -04:00
Aikar
2c648ff35f Don't toString block unless actually showing the message
This is causing MASSIVE performance issues with hoppers and block states
in general.
2020-05-09 13:55:41 -04:00
Aikar
d816beaced MC-183249: Don't generate Carving Masks BitSet unless needed
This was using SIGNIFICANT amounts of memory allocating many
long[]'s for BitSets for every ProtoChunk in the cache that had
been unloaded and reloaded.

This will result in a nice memory reduction.
2020-05-09 12:13:29 -04:00
Aikar
e9fcee1190 Improve BlockPosition hashCode/equals
Actually showed up in profiling as decent time spent here...

Noticed y/z was missing its final that it use to have, when x had it. some how
must of got messed up on some update. though people suggest this shouldn't of
mattered anyways, but lets put it back for safety.

Added cache of hashcode, as well as optimized the hash code using larger primes.
Also stored the long value of the x/y/z so that for equals we can compare a single long,
as well as have that long value cached for .asLong()
2020-05-09 10:59:29 -04:00
Aikar
c6b756d121 Fix world unload toLowercase - Fixes #3326 2020-05-09 10:59:10 -04:00
virustotalop
61142982e0 Optimize brigadier child sorting performance
Use a TreeMap instead of a LinkedHashMap and then manually sorting

This will significantly reduce CPU usage on logging in and changing worlds.
2020-04-16 20:57:02 -07:00
Aikar
b57f715649 Fix CraftServer.unloadWorld Leak
The dimension manager was still registered which leaked the entire World
2020-05-08 20:40:42 -04:00
Aikar
8c7b991317 Don't create region files for chunk checks - mistake on upstream merge
Accidently handled a conflict wrong in a previous upstream merge and set wrong param.
2020-05-08 18:22:37 -04:00
Aikar
1404a3b655 Fix Light Command
This lets you run /paper fixlight <chunkRadius> (max 5) to automatically
fix all light data in the chunks.

Permission node is same "bukkit.command.paper"
2020-05-07 19:44:33 -04:00
Aikar
e2fb92fbb6 Revert Nibble patch, causing issues still 2020-05-07 05:30:40 -04:00
Aikar
7f8e789cf3 Use a finalizer for light packet instead of onPacketDone
Sadly sharing packets is breaking the ability to do this :(

We can prob still do a little better than this but will save that
for another commit.
2020-05-07 01:58:23 -04:00
Aikar
9204e8c641 Optimize NibbleArray to use pooled buffers
Massively reduces memory allocation of 2048 byte buffers by using
an object pool for these.
2020-05-07 01:32:02 -04:00
Aikar
bb418b5532 Move delayed init down later in tick, improve accuracy of startup time
Now tracks the full startup time for "Done" message at end, as apparently
Vanillas was done in a place that skipped tracking a lot of code too.

This fixes an issue with ViaVersion
2020-05-06 21:24:43 -04:00
Aikar
58c0bbdd11 Force Plugins that use delayed tasks for init back in their place
Will now run those tasks just before we print "Done" so that startup
time is appropriately accounted for a plugin, as well as will no longer
trip the watchdog on startup.

Any plugin that tries to bypass this is just going to then trip watchdog
on Spigot too, so don't you dare.

Stop trying to cheat the delay your plugin added to startup time.

This isn't a behavior change because the first thing the tick does....
was run these tasks....

So it's just moving it slightly a few lines to be before a watchdog tick and
to account for it in "Done" time.

Fixes #3294
2020-05-06 20:18:31 -04:00
Aikar
7ec3cf8948 Don't skip full player connection tick when dead
Still need to send keep alives and kick for idle

Fixes #3299
2020-05-06 19:30:47 -04:00
Spottedleaf
f410ec4423 Revert loaded entity list (#3304) 2020-05-06 09:31:29 -07:00
Spottedleaf
0f2383acc0 Move another NetworkManager util into the inner class (#3303)
Those poor souls running outdated ProtocolLib.
2020-05-06 08:07:51 -07:00
Spottedleaf
f1b980c22f Make loaded entity list logic more consistent (#3301)
When adding/removing to a chunk, we need to also look at
editing the loaded entity list.

Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com>
2020-05-06 07:57:18 -07:00
Aikar
d2a300598c Updated Upstream (Bukkit/CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
da9ef3c5 #496: Add methods to get/set ItemStacks in EquipmentSlots
3abebc9f #492: Let Tameable extend Animals rather than Entity
941111a0 #495: Expose ItemStack and hand used in PlayerShearEntityEvent
4fe19cae #494: InventoryView - Add missing Brewing FUEL_TIME

CraftBukkit Changes:
933e9094 #664: Add methods to get/set ItemStacks in EquipmentSlots
18722312 #662: Expose ItemStack and hand used in PlayerShearEntityEvent
2020-05-06 06:05:22 -04:00
Aikar
1127deb87d Rebuild all patches using the new rebuild pattern 2020-05-06 05:48:49 -04:00
Mariell Hoversholm
45dc9552b1 Add villager reputation API 2020-04-22 23:46:57 +02:00
Aikar
fd5c98a9ef Optimize Network Manager to not need synchronization
Removes synchronization from sending packets
Makes normal packet sends no longer need to be wrapped and queued like it use to work.
Adds more packet queue immunities on top of keep alive to let the following scenarios go out
without delay:
  - Keep Alive
  - Chat
  - Kick
  - All of the packets during the Player Joined World event

Hoping that latter one helps join timeout issues more too for slow connections.

Removes processing packet queue off of main thread
  - for the few cases where it is allowed, order is not necessary nor
    should it even be happening concurrently in first place (handshaking/login/status)

Ensures packets sent asynchronously are dispatched on main thread

This helps ensure safety for ProtocolLib as packet listeners
are commonly accessing world state. This will allow you to schedule
a packet to be sent async, but itll be dispatched sync for packet
listeners to process.

This should solve some deadlock risks

This may provide a decent performance improvement because thread synchronization incurs a cache reset
so by avoiding ever entering a synchronized block, we get to avoid that, and packet sending is a really
hot activity.
2020-05-06 05:22:03 -04:00
stonar96
30d007b163 Anti Xray cleanup
Undo the accidental renaming of a method in e45e15c
Aikar wanted to rename DataPalette#getDataBits(T object) to getOrCreateIdFor
in e45e15c but he also accidentally renamed
ChunkPacketInfo#getDataBitsIndex(int chunkSectionIndex) to
getOrCreateIdForIndex.

Remove chunk-edge-mode and chunk loading entirely from Anti-Xray
The chunk-edge-mode is broken since several versions.
Loading chunk neighbors for chunk edge obfuscation isn't needed anymore.
Unlike in previous versions, these are under normal circumstances already loaded
at the time we need them (plugins for example can bypass this).

Use the modified methods and constructors everywhere
Anti-Xray provides support for the default nms methods and constructors,
which where modified by Anti-Xray to avoid breaking stuff (plugins)
which somehow uses these methods.
However, the modified versions of those methods and constructors should be used
where possible.
2020-05-06 04:35:20 -04:00
Spottedleaf
584c2fbe18 No-Tick view distance implementation - Closes #3196
Implements world view distance getters/setters

Per-Player is absent due to difficulty of maintaining
the diff required to make it happen.
2020-05-06 03:49:52 -04:00
Spottedleaf
fbaa9ae1f3 Stop copy-on-write operations for updating light data
Causes huge memory allocations + gc issues
2020-05-06 03:48:56 -04:00
Spottedleaf
7307189ab2 Optimize isOutsideRange to use distance maps
Use a distance map to find the players in range quickly
2020-05-06 03:48:18 -04:00
Spottedleaf
fbe8958237 Use distance map to optimise entity tracker / Misc Utils
Use the distance map to find candidate players for tracking.

This also ports a few utility changes from Tuinity
2020-05-06 03:44:47 -04:00
Spottedleaf
444dd5a0c6 Optimize Entity Ticking to Loaded Chunks only
We retain a list of loaded entities specifically for this usage
2020-05-06 03:43:11 -04:00
MiniDigger | Martin
d17856519c Mob Goal API 2020-04-30 17:18:45 +02:00
Aikar
3e4fec905c Revert "Don't flush packet queue off main thread"
This reverts commit e3105bc57a.

Sadly not gonna work due to server pings
2020-05-04 04:50:08 -04:00
Aikar
e3105bc57a Don't flush packet queue off main thread
Increases risk of deadlock if a plugin using protocollib sends a packet
async, and then a listener then reads world state, and main thread is then
blocked waiting for the queue to flush.

This will break out of the synchronized block when it jumps to the netty event loop.

See: https://gist.github.com/aikar/e7abb2ba7059149d0a91f7a226e98590
2020-05-04 03:07:04 -04:00
Aikar
5a8b71725f Clean up Direct Memory Region Files Fix for different Java versions
Java 9+ doesn't allow using the exposed cleanup method, but added
a new method on Unsafe to do it.

So have to detect java version and use the appropriate strategy.
2020-05-04 02:19:38 -04:00
Aikar
cc88bc65e1 Set cap on JDK per-thread native byte buffer cache
See: https://www.evanjones.ca/java-bytebuffer-leak.html

This is potentially a source of lots of native memory usage.

We are clearly seeing native usage upwards to 1-4GB which doesn't make sense.

Region File usage fixed in previous patch should of tecnically only been somewhat
temporary until GC finally gets it some time later, but between all the various
plugins doing IO on various threads, this hidden detail of the JDK could be
keeping long lived large direct buffers in cache.

Set system properly at server startup if not set already to help protect from this.
2020-05-04 01:14:10 -04:00
Aikar
8756fc733c Cleanup Region Files Direct Memory on close
Mojang was semi leaking native memory here by relying on finalizers
to clean up the direct memory.

Finalizers have no guarantee on when they will be ran, and since this is
old generation memory, it might be a while before its called.
2020-05-04 01:03:57 -04:00
Aikar
87066b9acb Optimize Voxel Shape Merging
This method shows up as super hot in profiler, and also a high "self" time.

Upon analyzing, it appears most usages of this method fall down to the final
else statement of the nasty ternary.

Upon even further analyzation, it appears then the majority of those have a
consistent list 1.... One with Infinity head and Tails.

First optimization is to detect these infinite states and immediately return that
VoxelShapeMergerList so we can avoid testing the rest for most cases.

Break the method into 2 to help the JVM promote inlining of this fast path.

Then it was also noticed that VoxelShapeMergerList constructor is also a hotspot
with a high self time...

Well, knowing that in most cases our list 1 is actualy the same value, it allows
us to know that with an infinite list1, the result on the merger is essentially
list2 as the final values.

This let us analyze the 2 potential states (Infinite with 2 sources or 4 sources)
and compute a deterministic result for the MergerList values.

Additionally, this lets us avoid even allocating new objects for this too, further
reducing memory usage.
2020-05-03 23:02:52 -04:00
Aikar
02d5ea2199 Sync position on teleportation
We've seen many a cases where the "last good" x/y/z is desynced from
the x/y/z that is checked for moving too fast.

Theory is that when you have multiple movement packets queued up,
and the player is teleported after the first then the 2nd and 3rd come in,
it is triggering a massive movement velocity.

This will ensure that the servers position is synchronized anytime player is te

Fixes #3258
2020-05-03 14:28:45 -04:00
Aikar
f33a326493 Nanothing to see here 2020-05-03 03:38:19 -04:00
Aikar
51f3a1e457 Attempt to fix FastLogin maybe 2020-05-03 02:20:56 -04:00
Aikar
3ff811a50a Rename to AsyncPlayerSendSuggestionsEvent to be consistent in naming
It was still technically read correctly in what it was doing, but
all our Player events begin with Player.

Nothing uses this event yet so safe to rename.

If you are some rapid adopter of this event, sorry :P
2020-05-02 15:25:55 -04:00
Aikar
adc83a5b5c AsyncSendPlayerSuggestionsEvent Brigadier Event
Going to be used for ACF to be able to send Tooltips to client :)

This is after completions are calculated, to control what is sent to the client.
2020-05-02 14:17:17 -04:00
Aikar
bce99d4a94 Fix being kicked in survival for block picking - Fixes #3277 2020-05-02 13:32:03 -04:00
MiniDigger | Martin
4564d6b566 Expose game version (#3274) 2020-05-02 10:56:08 +02:00
Aikar
600d62f2cc Validate PickItem Packet and kick for invalid - Fixes #3256 2020-05-02 03:10:26 -04:00
Aikar
6daf4a783f Special case Keep Alive packets from Anti Xray
If a server enables Anti Xray, packet sending can be delayed until the
chunk has been obfuscated, blocking the entire queue from going out.

On a busy server, considering Anti Xray can only operate on a single
thread, it is quite possible the obfuscation backlog can get quite behind
resulting in a delay of sending packets.

And logging in is a clear area where lots of chunks are going to be queued
for obfuscation....

We should probably special case a few more than this (such as chat),
but this will hopefully help the keep alive issues some people run into.
2020-05-02 00:53:57 -04:00
Spottedleaf
89e22765be Improved oversized chunk data packet handling
Now target all TE data, except for TE's that do not have
update packets.
2020-05-01 23:18:33 -04:00
Aikar
89eb7a7232 Use Vanilla Bed Search for non players (Villagers)
Should help #2959
2020-05-01 21:56:50 -04:00
Aikar
8a3c4f1d42 Fix 2 plugin specific issues with loot drop and pathfinders
Based on what that collection does, won't really cause any real issue for UC.
So let it silently be ignored.

Fixes #3269
Fixes #3255
2020-05-01 21:44:06 -04:00
Aikar
df7e6433ad Update Activation Range 2.0 with more villager controls
Now has separate configs to control Villager immunities a bit.

whether or not they wake up due to panic situations (raids)
and when should they wake up when work is available after being
inactive for so long, and for how long.

This work config may make the 'wake up inactive' feature for villagers
useless in most scenarios, but if there is a situation where the villager
does go without needing to work for a long period of time, it would kick
in then.

This also removes movement based immunities, so now villagers should only move
if they trigger a work immunity, panic immunity, or inactive wake up immunity.

Fixes #3263
2020-05-01 21:40:23 -04:00
Aikar
b06cb423cb Updated Upstream (Bukkit/CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
b999860d SPIGOT-2304: Add LootGenerateEvent

CraftBukkit Changes:
77fd87e4 SPIGOT-2304: Implement LootGenerateEvent
a1a705ee SPIGOT-5566: Doused campfires & fires should call EntityChangeBlockEvent
41712edd SPIGOT-5707: PersistentDataHolder not Persistent on API dropped Item
2020-05-01 18:03:47 -04:00
MiniDigger | Martin
9c8d3fce12 Fix commodore (#3264)
fixes #3262
2020-04-30 19:05:20 +02:00
Aikar
6b3f3d8d8c Improve Async Login to avoid firing in middle of Entity Ticking
If a sync load was triggered, it would process pending join events,
causing them to be added to the world in the middle of the entity ticking
process.

This caused their add to be queued instead of immediate, causing
"Illegal Tracking" errors.

This schedules it to fire at the players next Connection Tick, which
is exactly where this entire process use to run anyways.

Also added missing tab complete and syntax for syncloadinfo debug command
2020-04-29 04:14:47 -04:00
Aikar
3588b51a22 Fix build due to spigot changing the build timestamp process 2020-04-27 04:03:00 -04:00
Aikar
f42b510b51 Updated Upstream (Bukkit/CraftBukkit/Spigot)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
220bc594 #486: Add method to get player's attack cooldown
21853d39 SPIGOT-5681: Increase max plugin channel size
5b972adc Improve build process
b55e58d9 Note which custom generator is missing required method

CraftBukkit Changes:
893ad93b #650: Add method to get player's attack cooldown
ef706b06 #655: Added support for the VM tag jansi.passthrough when processing messages sent to a ColouredConsoleSender.
e0cfb347 SPIGOT-5689: Fireball.setDirection increases velocity too much
94cb030f SPIGOT-5673: swingHand API does not show to self
b331a055 SPIGOT-5680: isChunkGenerated creates empty region files
e1335932 Improve build process
a8ec1d60 Add a couple of method null checks to CraftWorld
ce66f693 Misc checkstyle fixes
8bd0e9ab SPIGOT-5669: Fix Beehive.isSedated

Spigot Changes:
2040c4c4 SPIGOT-5677, MC-114796: Fix portals generating outside world border
ab8f6b5a Rebuild patches
e7dc2f53 Rebuild patches
2020-04-27 03:34:45 -04:00
nick
6f8867b6ae Add getter and setter for villager's numberOfRestocksToday (#3231) 2020-04-27 02:13:41 -04:00
Aikar
e125c7ff12 null check tracker for entity metadata update - Fixes #3070 2020-04-27 01:57:42 -04:00
Aikar
0c2014644b Implement Brigadier Mojang API
This is the start of a new module for Paper to add support for API's
that interface Mojang API's directly.

This allows us to version properly by MC version incase Mojang makes any major breaking changes.

It also lets us separate Mojang API's from Paper-API so our downstream friends at Glowstone
will not have to worry about Mojang code.

Adds AsyncPlayerSendCommandsEvent
  - Allows modifying on a per command basis what command data they see.

Adds CommandRegisteredEvent
  - Allows manipulating the CommandNode to add more children/metadata for the client
2020-04-27 01:37:37 -04:00
Aikar
544cae4eca Entity Activation Range 2.0! Major improvements to restoring behavior
Calling this 2.0 as it's a pretty major improvement with more knobs to twist.

This update fixes many things. The goal here is to restore vanilla behavior to some degree.
Instead of permanent inactive pools of animals, let them show some signs of life some....

Yes this may reduce performance compared to before, but I hope it is minimal. Got to find a balance.
Previous EAR logic really compromised vanilla behavior of mobs. This tries to restore it.

Changes:

1) All monsters are now classed as Monster. Mojang has an interface, we should use it.
   - This now includes Shulker, Slimes, see #2 for Phantom and Ghast
2) Villagers and Flying Monsters now have their own separate activation range configs.
   - Villagers will default to your Animals config
3) Added a bunch of more immunities
   - Brand new entities are immune for a few seconds
   - Entities that recently traveled by portal are immune for few seconds
   - Entities that are leashed to a player are immune
   - Ender Signals are immune
   - Entities that are jumping, climbing, dying (lol) are immune
   - Minecarts are now always immune to the movement restriction
4) Villagers immunity received major overhaul...
   - Now has many immunities for Villager activities to let them
     do their work then go back inactive
   - Such as interacting with doors and workstations should be more normal now
   - Raids will trigger immunities, in that villagers will run and hide when bell rings.
   - Raid should keep the entire village immune during the raid to keep gameplay mechanics
     You can disable raids by game rule if you dont want raids

Then the big one.....

Wake Up Inactive Entities:
One issue plagueing "farms" is that we no longer even let entities move now.
Entities become lifeless.

A new system has been introduced to wake up inactive entities every so often, to let
them stretch their legs, eat some food, play with each other and experience the good entity life.

Animals, Villagers, Monsters (Includes Pillagers), and Flying Monsters will now wake up every
so often after staying inactive for a very long. This grants them a temporary immunity, that
the goal is they will then find "stuff to do" by having a longer activity window.

How many to wake up, how often they wake up, and for how long they wake up are all configurable.

Current EAR Immunities really don't give some entities enough of a window to find work
to then keep them immune for the work to even start. This system should help that.

We will only wake up a few entities per tick on the first wave, restoring 1 per type per world per tick.

So say you have 10 monsters qualify for inactive wake up, all 8 will wake up on the first eligible tick,
and then the 9th will wake up on next tick, 10th on next tick.

If for 5 ticks no more inactive wake up, our buffer will have built back up to 5, and then 5 can go next needed tick.

This basically incrementally wakes them up, preventing too many from waking up in a single tick, to reduce impact to TPS.
2020-04-27 00:34:51 -04:00
Aikar
244841d9a3 Fix Tracking Range mismatch on Vehicle/Passenger checks
This was missing Entity Tracking Range support, creating different
values in this section vs normal section.

Concerned this might of caused some carnage on tracker if this code says
"Yes you should track this player 500 blocks away from you on a horse" and then
the other check uses the normal value.
2020-04-27 00:33:14 -04:00
Aikar
985dbd1710 Add a config to turn off Optimized TickList #3145
Set:

settings:
  - use-optimized-ticklist: false

If you are having issues with block updates and want to see if this fixes it.

Please report confirmations on #3145 ticket
2020-04-26 04:01:03 -04:00
Aikar
a96311595a Improve blocking players from opening inventories while sleeping
This is friendlier to plugins as far as the plugin is concerned,
the inventory did open and immediately closed.

We avoid sending the packet to client so they don't see the window
flash either.

If a plugin wants to avoid wasteful fake opens, they should check
that the player is not sleeping before opening the inventory.
2020-04-26 03:26:21 -04:00
Aikar
0762c51996 Clean up Timings and Async Chunk Configs
Renames a bunch of timings to be more appropriate for the new environment.

Many things dealt with sync loads which wasn't correct anymore.
adjusted timings to be a little bit more accurate here.

Also cleaned up old 1.13 async chunks configs so people won't keep
thinking they can change some of those configs when they can't.
2020-04-26 02:05:41 -04:00
Aikar
879269f221 Improve mid tick chunk loading, Fix Oversleep, other improvements
Process loads outside of any canSleep check. Original intent was to
only apply those restrictions to generations but realized I had some
checks higher up the call chain.

Reworked the back off strategy to just run every 1 millisecond per world,
and to apply the per tick limit to generations only.

This guarantees that your chunk will load with at most around 1ms delay.

Additionally, fire midTick processing in a few more places, notably the
oversleep section so we can keep processing loads here too which has
a large up to 50ms window...

Speaking of oversleep, we had a bug in our implementation changes for
Timings that caused oversleep to not sleep the correct amount.

Because we now moved it into the NEXT tick instead of THIS tick, the
value of nextTick had already been increased to +50ms, resulting in
the risk of sleeping more than it should, but, more importantly, this
caused every task that was trying to NOT run during oversleep to actually
run during oversleep.

This is now fixed.

Another small tweak is to the /tps command, to no longer show the star when
TPS is right at 20.

Due to ineffeciencies in the sleep precision, TPS is commonly 20.02.
This causes the star to show up almost constantly, so now only show it if
we actually hit a real "catchup".

This commit also improves the changes to the CallbackExecutor, in that
it now is also recursion safe.

It was possible that the executor could run tasks out of desired order
if the executor task scheduled more executor tasks.

We solve this by ensuring new additions do not enter the currently iterated queue.

Each depth level will have its own queue.

Fixes #3220
2020-04-25 23:47:29 -04:00
Spottedleaf
0e3875914c Improve random ticking behaviour - Fixes #3181 2020-04-19 06:07:57 -07:00
Aikar
5ea7f7c4e9 Fix numerous item duplication issues and teleport issues
This notably fixes the newest "Donkey Dupe", but also fixes a lot
of dupe bugs in general around nether portals and entity world transfer

We also fix item duplication generically by anytime we clone an item
to drop it on the ground, destroy the source item.

This avoid an itemstack ever existing twice in the world state pre
clean up stage.

So even if something NEW comes up, it would be impossible to drop the
same item twice because the source was destroyed.

This should make us more forward proof on preventing dupes.

These dupes have been in for years at this point, they aren't new...
Everyone knows about them and are mitigating with plugins atm breaking gameplay.

so better to make it clear its fixed in the messaging.

I am submitting this to Mojang.
2020-04-25 06:52:30 -04:00
BillyGalbreath
fe730b5a87 Add phantom creative and insomniac controls (#3222) 2020-04-25 15:33:18 -05:00
Aikar
9c59e219f6 Improve Async Login so pending connections dont get exposed
We still keep vanilla process of waiting for existing session to be removed before logging in
by storing a separate map of pending.

also fire the callback using executor incase further recursion causes any trouble
2020-04-24 05:33:33 -04:00
Aikar
58876facfa Optimize Hoppers - Major Boost - Got2GoFast!
* Don't check for Entities with Inventories if the block above us is also occluding (not just Inventoried)
* Remove Streams from Item Suck In and restore restore 1.12 AABB checks which is simpler and no voxel allocations (was doing TWO Item Suck ins)
* Restore missing application of previous optimization to getEntities for Inventoried Entities from CullanP
* Use getChunkIfLoadedImmediately for getting loaded entities (faster/simpler, no risk of sync loads)
2020-04-24 03:01:17 -04:00
Aikar
71ed03c3ca Improvements to watchdog changes
catch a thread death, fix invalid watchdog timeouts

<= 0 seconds will now change to 5 minutes.
2020-04-24 00:27:28 -04:00
Aikar
d0f606ec78 Fix build for last commit. 5am commits are great 2020-04-23 04:53:20 -04:00
Aikar
a6dbadb9d0 Don't process watchdog until server has fully started and ticked.
I think adjusting watchdog settings may of caused instant crashes.

Fixes #3209
2020-04-23 04:46:53 -04:00
Aikar
cd84dc9e5d Don't fire BlockFade on worldgen threads - Fixes #3208
Caused a deadlock
2020-04-23 01:37:30 -04:00
Aikar
2fb3e5fb88 Fix loading spawn chunks when async chunks is off
I feel sorry for those who need to do this, and now feel sorry more
since back to slow startups again.

There is keep-spawn-loaded-range in paper.yml to reduce the range to
mitigate this if you must keep async chunks off.
2020-04-22 05:47:35 -04:00
Aikar
9e4ba94fa1 Improvements to async login
Bump chunk priority to ensure chunks load fast
Handle case where client disconnects before they even fire PlayerJoinEvent
  - no longer call PlayerQuitEvent or print quit message.
  - don't save the player data file if never joined. Nothing has changed.
    CraftBukkit has a bug here that if you do save it, you will lose
    any horse that the player logged off on because the horse hasn't
    been resummoned yet.
2020-04-22 05:40:06 -04:00
Ytnoos
f7e48582b5 Revert "Re-track players that dismount from other players" 2020-04-22 00:06:30 +02:00
Aikar
6578a2d659 Fix some issues with async login as well another source of sync loads 2020-04-22 02:52:17 -04:00
Aikar
f52c927ae6 Allow multiple callbacks to schedule for Callback Executor
ChunkMapDistance polls multiple entries for pendingChunkUpdates

Each of these have the potential to move a chunk in and out of
"Loaded" state, which will result in multiple callbacks being
needed within a single tick of ChunkMapDistance

Use an ArrayDeque to store this Queue
2020-04-21 03:55:31 -04:00
nossr50
cfacd0543e Add PlayerAttackEntityCooldownResetEvent
This event is called when processing a player's attack on an entity
right before their attack strength cd is reset, there are no existing
events that fire within this period of time so it was impossible to
capture the players attack strength via API prior to this commit.

The event is cancellable, which will just skip over the normal reset of
attack strength cd
2020-03-26 20:04:19 -07:00
Mariell Hoversholm
a530d9d70d Allow sleeping players to float
This change lets players who are in their bed have a position which is above
ground for a longer period of time. This is because of the server not setting
their position to the ground/exit location when entering the bed, resulting in
the server believing they're still in the air.
2020-04-19 12:26:07 +02:00
Omer Uddin
93e0c8c61e Fix Bed respawn deviating too far from vanilla (#3195) 2020-04-20 23:14:28 -04:00
2277
5f31a94a22 Move player to spawn point if spawn in unloaded world
The code following this has better support for null worlds to move
them back to the world spawn.
2020-03-31 10:38:18 +01:00
Spottedleaf
d77bc736a0 Improve async login (#3189)
Add helper functions to ChunkProviderServer to make this easier
for other uses

Co-authored-by: Spottedleaf <Spottedleaf@users.noreply.github.com>
2020-04-19 10:58:02 -07:00
Aikar
909a2454fc Fix Citizens Player NPC tracking issue - Fixes #3186 2020-04-19 13:41:16 -04:00
Aikar
f7e26b47eb Fix Player Tracking issue during join process.
Because we moved entity registration to occur before the PlayerJoinEvent occurs,
We started tracking the entity too early before it was registered to the client.

So delay tracking until after list packets have been sent.
2020-04-19 06:37:10 -04:00
Aikar
9fc8102ef5 Load Spawn Chunks Asynchronously and Spiral Out
Ensures absolute center is first in priority
Speeds up server startup
2020-04-19 06:01:07 -04:00
Aikar
37c7a3af0a Load Chunks for Login Asynchronously
No longer will trigger Synchronous Chunk Loads when a player logs
in to the server.

Will delay PlayerJoinEvent until the chunk has been loaded.

Should have massive performance benefits for larger servers with
lots of players logging in and out.
2020-04-19 04:32:04 -04:00
Aikar
fd9350e9ba Clean up duplicate PlayerInitialSpawnEvent
Confused on this one, as commit history says Spigots version is older
than our version, so i'm not sure how we ended up duplicating this when
the 2 events are 100% identical.

Subclass spigots event and rely on the inheritance system, and clean up
the duplicate event fires.

Fix Spigots setPosition to use setPositionRaw to avoid chunk load prematurely.
2020-04-19 03:11:02 -04:00
Aikar
9cdf9d5502 Fix Longstanding Broken behavior of PlayerJoinEvent
For years, plugin developers have had to delay many things they do
inside of the PlayerJoinEvent by 1 tick to make it actually work.

This all boiled down to 1 reason why: The event fired before the
player was fully ready and joined to the world!

Additionally, if that player logged out on a vehicle, the event
fired before the vehicle was even loaded, so that plugins had no
access to the vehicle during this event either.

This change finally fixes this issue, fully preparing the player
into the world as a fully ready entity, vehicle included.

There should be no plugins that break because of this change, but might
improve consistency with other plugins instead.

For example, if 2 plugins listens to this event, and the first one
teleported the player in the event, then the 2nd plugin actually
would be getting a valid player!

This was very non deterministic. This change will ensure every plugin
receives a deterministic result, and should no longer require 1 tick
delays anymore.
2020-04-19 00:56:43 -04:00
AvrooVulcan
1fb073fb19 Broadcast join messages to console 2020-04-17 00:18:51 +01:00
Aikar
1b04899f1d Don't crash if player is attempted to be removed from untracked chunk.
I suspect it deals with teleporting as it uses players current x/y/z

Fixes #2803
2020-04-18 16:02:37 -04:00
Spottedleaf
dc66fe2c49 Make sure the chunk conversion task is executed immediately
Appending to the tail of the chunk tasks leaves a
window for the chunk to be moved to a
non-ticking status.

Additionally, use CB's callback executor so we
can ensure that we are not incorrectly
scheduling.
2020-04-18 08:41:02 -07:00
Aikar
c293d0a3d8 Fix Chunk Post Processing deadlock risk
See: https://gist.github.com/aikar/dd22bbd2a3d78a2fd3d92e95e9f28dc6

as part of post processing a chunk, we can call ChunkConverter.

ChunkConverter then kicks off major physics updates, and when blocks
that have connections across chunk boundries occur, a recursive risk
can occur where A updates a block that triggers a physics request.

That physics request may trigger a chunk request, that then enqueues
a task into the Mailbox ChunkTaskQueueSorter.

If anything requests that same chunk that is in the middle of conversion,
it's mailbox queue is going to be held up, so the subsequent chunk request
will be unable to proceed.

We delay post processing of Chunk.A() 1 "pass" by re stuffing it back into
the executor so that the mailbox ChunkQueue is now considered empty.

This successfully fixed a reoccurring and highly reproduceable crash
for heightmaps.
2020-04-18 04:44:35 -04:00
Shane Freeder
715a50fd4b Revert "Optimize entity list iteration requiring entities be in"
This reverts commit c1fdcae86a.
2020-04-17 18:24:06 +01:00
Aikar
1cc0c3ac35 Prevent Fire from loading chunks
Had my server die from this twice already as it can cause a chain reaction.
2020-04-17 00:29:53 -04:00
MiniDigger | Martin
d97ce19f49 Add Player Client Options API (#2883) 2020-04-17 06:10:38 +02:00
Spottedleaf
c1fdcae86a Optimize entity list iteration requiring entities be in
loaded chunks

We retain a list of loaded entities specifically for this usage
2020-04-16 16:53:45 -07:00
Spottedleaf
b415ceb617 Optimize ChunkProviderServer's chunk level checking helper methods
These can be hot functions (i.e entity ticking and block ticking),
so inline where possible, and avoid the abstraction of the
Either class.
2020-04-16 20:47:15 -07:00
Aikar
416b2f43e3 Forced Watchdog Crash support and Improve Async Shutdown
If the request to shut down the server is received while we are in
a watchdog hang, immediately treat it as a crash and begin the shutdown
process. Shutdown process is now improved to also shutdown cleanly when
not using restart scripts either.

If a server is deadlocked, a server owner can send SIGHUP (or any other signal
the JVM understands to shut down as it currently does) and the watchdog
will no longer need to wait until the full timeout, allowing you to trigger
a close process and try to shut the server down gracefully, saving player and
world data.

Previously there was no way to trigger this outside of waiting for a full watchdog
timeout, which may be set to a really long time...

Additionally, fix everything to do with shutting the server down asynchronously.

Previously, nearly everything about the process was fragile and unsafe. Main might
not have actually been frozen, and might still be manipulating state.

Or, some reuest might ask main to do something in the shutdown but main is dead.

Or worse, other things might start closing down items such as the Console or Thread Pool
before we are fully shutdown.

This change tries to resolve all of these issues by moving everything into the stop
method and guaranteeing only one thread is stopping the server.

We then issue Thread Death to the main thread of another thread initiates the stop process.
We have to ensure Thread Death propagates correctly though to stop main completely.

This is to ensure that if main isn't truely stuck, it's not manipulating state we are trying to save.
2020-04-16 04:53:50 -04:00
Zach Brown
a94ea15b5a Be less strict with vanilla teleport command limits
Just limit to the same valid location check as BaseBlockPosition, no
reason to restrict to the world border.

Ref: GH-3165
2020-04-16 21:16:35 -05:00
Zach Brown
5041e3e821 Restrict vanilla teleport command to within worldborder
Fixes GH-3165
2020-04-16 20:11:44 -05:00
Aikar
902942d9e4 Fix Optional null issue - Fixes #3155
Also check class loader cache before locking to speed up cached hits to avoid the lock

wasn't gonna make a unique build just for that but can lump it in here.
2020-04-16 03:57:02 -04:00
Spottedleaf
e7f799903e Fix incorect timing of mspt
Move oversleep and all task execution back into the timed area...
2020-04-15 18:48:07 -07:00
Spottedleaf
70131f31d3 Optimise entity hard collision checking
Very few entities actually hard collide, so store them in their own
entity slices and provide a special getEntites type call just for them.
This reduces entity collision checking impact (in my testing) by 25%
for crammed entities (shove 130 cows into an 8x6 area in one chunk).
Less crammed entities are likely to show significantly less benefit.
Effectively, this patch optimises crammed entity situations.
2020-04-16 00:40:47 -04:00
Spottedleaf
1b58ca2161 Don't run entity collision code if not needed
Will not run if max entity craming is disabled and
the max collisions per entity is less than or equal to 0
2020-04-16 00:40:34 -04:00
Spottedleaf
25d0fbed10 Optimise ArraySetSorted#removeIf
Remove iterator allocation and ensure the call is always O(n)
2020-04-15 18:24:39 -07:00
Aikar
9b09360ef5 Make the shutdown thread try to shutdown on main 2020-04-13 03:30:12 -04:00
Shane Freeder
207492e613 Prevent opening inventories when frozen 2020-04-13 07:33:46 +01:00
Spottedleaf
973a5271c3 Port 04-Util.patch from Tuinity (#3136) 2020-04-12 22:34:18 -07:00
Aikar
72fae0db1a Fix incorrectly loading chunks on a cancelled interact event
A players previous block break location is held onto permanently, and if
an interact event is cancelled, the client sends a stop breaking block packet
This then tries to update client about that old location.

This old location might then be in a now unloaded chunk, and it caused it to load.

We now also clear reference to it once abort destroy block is ran to stop trying
to send updates about the old block anyways.
2020-04-12 23:56:29 -04:00
Aikar
47c832382c Let invalid positioned entities clean up previous chunk by the chunkCheck
I had did a few of the operations myself, which would have broken chunkCheck
from doing it itself, which would leave some state left in the original chunk
and thats not good....
2020-04-12 22:55:43 -04:00
Spottedleaf
cd7e99804b Multiple memory and performance optimizations (removing streams) 2020-04-12 18:31:14 -04:00
Aikar
73e413e403 Improve the don't save entities patch per leafs feedback 2020-04-12 18:29:52 -04:00
Callahan
bd4c6a566c Port 20w15a Villager AI optimizations 2020-04-08 18:02:26 -05:00
Aikar
bc1203ad20 Restore preventing saving bad entities patch to full effect 2020-04-12 16:50:50 -04:00
Aikar
b03a0ea94b Restore Optimize Pathfinding patch 2020-04-12 16:50:34 -04:00
Aikar
0d4cde25b6 Allow shutting down server during a watchdog hang gracefully
If the request to shut down the server is received while we are in
a watchdog hang, immediately treat it as a crash and begin the shutdown
process. Shutdown process is now improved to also shutdown cleanly when
not using restart scripts either.

If a server is deadlocked, a server owner can send SIGUP (or any other signal
the JVM understands to shut down as it currently does) and the watchdog
will no longer need to wait until the full timeout, allowing you to trigger
a close process and try to shut the server down gracefully, saving player and
world data.

Previously there was no way to trigger this outside of waiting for a full watchdog
timeout, which may be set to a really long time...
2020-04-12 15:56:03 -04:00
Callahan
6dd459625e Async command map building
Reduces lag created on login and world change for sending the command map to client
2020-04-09 01:49:38 -05:00
DrHenchman
1e3eae5e0f Fix method profiler inbalance introduced in b667006 (#3132)
Fixes #3131
2020-04-12 16:41:57 +10:00
Aikar
cb8f1dcc40 Don't delay chunk unloads during entity ticking
Leaf informed me this could cause ordering issues.

So, the risk if this occurring is lowered now anyways, but if an
entity causes a sync chunk load, it could process an unload...

We will tackle the problem better in a future commit

Also fixed another async-chunks=false issue
2020-04-12 00:42:28 -04:00
Aikar
5fc72e41c0 Delay unsafe actions until after entity ticking is done - Fixes #3114
This will help prevent many cases of unregistering entities during entity ticking

Currently delays Chunk Unloads and Async Chunk load callbacks

Also dropped mid ticking chunk tasks during entity ticking to reduce this risk
2020-04-11 21:31:40 -04:00
Aikar
372e545d08 Use better variable for isStopping() API
Previous method only worked for a normal shutdown, and didn't include
when the server enters a closing state due to watchdog crashes

This is the correct variable to detect the server is in the middle of shutdown process
2020-04-11 21:26:16 -04:00
Aikar
a5eb244a55 Remove patch for MC-111480, fixed in 1.14
We were now starting entities at ID 2
2020-04-11 20:39:21 -04:00
Spottedleaf
f654174059 Remove streams from Mob AI System
The streams hurt performance and allocate tons of garbage, so
replace them with the standard iterator.

Also optimise the stream.anyMatch statement to move to a bitset
where we can replace the call with a single bitwise operation.
2020-04-11 20:15:22 -04:00
Aikar
6ad6ec1ac9 Fix Disabling Asynchronous Chunks
This fix is for the few people who are using such low end systems that
asynchronous chunk loading hurts them rather than helping.

The previous build made paper crash if you turned off async chunks, and
this fixes that issue.
2020-04-11 17:15:41 -04:00
Aikar
335e39dca1 Implement Chunk Priority / Urgency System for World Gen
Mark chunks that are blocking main thread for world generation as urgent

Implements a general priority system so that chunks that are sorted in
the generator queues can prioritize certain chunks over another.

Urgent chunks will jump to the front of the line, ensuring that a
sync chunk load on an ungenerated chunk does not lag the server for
a long period of time if the servers generator queues are filled with
lots of chunks already.

This massively reduces the lag spikes from sync chunk gens.

This is also a precursor to my next improvement to prioritize chunks
in front of the player (Frustum Priorization)
2020-04-11 04:44:21 -04:00
Aikar
98218db96f Use dedicated thread for main thread blocking chunk loads
In most cases, this change won't benefit much. However, there
exists the possibility that your Chunk Task threads are all busy
doing super slow work such as converting chunks.

If this occurs, the main thread blocking tasks, even at highest priority,
has to wait for some thread to become available.

This change gives us a waiting thread used only for main thread blocking
tasks, as well as an increased thread priority level, so that the OS
will give priority to this thread over the other threads.

This is more about guarantees, and won't be any real performanc boost
to anyone who has low or fast activity on their chunk tasks anyways.

But not all of us force upgrade our worlds, and this can be a life saver.

also reordered some patches because multiple PR's were merged.
2020-04-11 04:40:00 -04:00
BillyGalbreath
c8367bb129 Add tick times API and /mspt command (#3102) 2020-04-11 00:52:17 -05:00
Josh Roy
4042fd2067 Add API MinecraftServer#isStopping (#3129) 2020-04-11 01:51:35 -04:00
Aikar
81a57f3fde Fix issues with 167 causing crashes due to missing chunks - Fixes #3122
Forgot to flip the pending boolean back to false, causing it to copy
empty data on the next tick if nothing else triggered a load.

haven't managed to actually reproduce the crash others got, but did
verify that the bad copy was occurring erasing the data.

also fixed a bug with chunk load callback not executing before
another one was scheduled.
2020-04-10 14:11:46 -04:00
Aikar
13e8f5eb83 Fix issue with 0,0 chunks causing crash 2020-04-09 23:02:05 -04:00
Aikar
0db7fba2d3 Updated Upstream (CraftBukkit/Spigot)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

CraftBukkit Changes:
183139d4 SPIGOT-5665: Improve loading spawn egg NBT
dec5df26 SPIGOT-5667: Can't add recipe without (vanilla) datapack

Spigot Changes:
ae72bf43 SPIGOT-5666: Customizable End City Seed
2020-04-09 21:33:10 -04:00
Aikar
f481e800ee Drowned is a RangedEntity (fixes API for Drowned to support Ranged) 2020-04-09 21:30:38 -04:00
Aikar
5d01e6b184 Don't move existing players to world spawn
This can cause a nasty server lag the spawn chunks are not kept loaded
or they aren't finished loading yet, or if the world spawn radius is
larger than the keep loaded range.

By skipping this, we avoid potential for a large spike on server start.
2020-04-09 21:22:21 -04:00
Aikar
6814a6cfb6 Fix issue with loading chunk during ticking chunks issue
Also speed up performance of copying pending/updating to visible by
directly copying the internal array data instead of putAll
2020-04-09 20:45:46 -04:00
Aikar
c0e449f4de Speed up processing of chunk loads and generation
Credit to Spotted for the idea

A lot of the new chunk system requires constant back and forth the main thread
to handle priority scheduling and ensuring conflicting tasks do not run at the
same time.

The issue is, these queues are only checked at either:

A) Sync Chunk Loads
B) End of Tick while sleeping

This results in generating chunks sitting waiting for a full tick to
complete before it will even start the next unit of work to do.

Additionally, this also delays loading of chunks until this same timing.

We will now periodically poll the chunk task queues throughout the tick,
looking for work to do.
We do this in a fair method that considers all worlds, not just the one being
ticked, so that each world can get 1 task procesed each before the next pass.

We also cap the throughput of these task processes to 1 per world per 0.1ms or
200 max per tick, to ensure that high volume of tasks do not overload the current
tick time.

In a view distance of 15, chunk loading performance was visually faster on the client.

Flying at high speed in spectator mode was able to keep up with chunk loading (as long as they are already generated)
2020-04-09 02:25:18 -04:00
Aikar
7c041b2460 Increase Light Queue Size
Wiz mentioned that large WorldEdit operations cause light to run on
main thread. The queue was small, set to 5.. this bumps it to 20
but makes it configurable per-world.

The main risk of increasing this higher is during shutdown, some
queued light updates may be lost because mojang did not flush the
light engine on shutdown...

The queue size only puts a cap on max loss, doesn't solve that problem.

Don't touch this unless you know you have a problem and ok with the risk.
2020-04-08 21:36:10 -04:00
Aikar
2eb3d58a04 Don't load chunks when attempting to unload a chunk
Big Brain Logic
2020-04-08 21:13:48 -04:00
Aikar
587c1e02a5 Improve Optimize Memory use logic to make iterator safer and fix bad plugins like P2
Accessing world state async is bad mkay. But we know you didn't do it city <3
2020-04-08 21:07:59 -04:00
Aikar
00570dc4bb Fix incorrect keyword use on visibleChunksClone 2020-04-08 04:12:48 -04:00
Aikar
b7de98fcb0 Updated Upstream (Bukkit/CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
7361a62e SPIGOT-5641: Add Block.getDrops(ItemStack, Entity)
1dc91b15 Add specific notes about what is not API
2b05ef88 #484: Allow statistics to be accessed for offline players

CraftBukkit Changes:
f7d6ad53 SPIGOT-5603: Use LootContext#lootingModifier in CraftLootTable
5838285d SPIGOT-5657: BlockPlaceEvent not cancelling for tripwire hooks
f325b9be SPIGOT-5641: Add Block.getDrops(ItemStack, Entity)
e25a2272 Fix some formatting in CraftHumanEntity
498540e0 Add Merchant slot delegate
b2de47d5 SPIGOT-5621: Add missing container types for opening InventoryView
aa3a2f27 #645: Allow statistics to be accessed for offline players
2122c0b1 #649: CraftBell should implement Bell
2020-04-08 03:49:15 -04:00
Aikar
988431a872 Optimize PlayerChunkMap memory use for visibleChunks
No longer clones visible chunks which is causing massive memory
allocation issues, likely the source of Humongous Objects on large servers.

Instead we just synchronize, clear and rebuild, reusing the same object buffers
as before with only 2 small objects created (FastIterator/MapEntry)

This should result in siginificant memory use reduction and improved GC behavior.
2020-04-08 03:09:05 -04:00
Aikar
9bd9caa6ae Use entity.dead instead of entity.die()
Fixes #3096
2020-04-04 21:09:27 -04:00
Aikar
89718e929a Restore accidently reverted item in water fix 2020-04-02 23:12:21 -04:00
Aikar
6bf47f7c5a Dead Player's shouldn't be able to move
This fixes a lot of game state issues where packets were delayed for processing
due to 1.15's new queue but processed while dead.
2020-04-02 19:42:20 -04:00
Aikar
507ed674f0 Don't tick dead players
Causes sync chunk loads and who knows what all else.
This is safe because Spectators are skipped in unloaded chunks too in vanilla.
2020-04-02 17:19:44 -04:00
Aikar
a25f99d254 Updated Upstream (Bukkit/CraftBukkit)
Upstream has released updates that appears to apply and compile correctly.
This update has not been tested by PaperMC and as with ANY update, please do your own testing

Bukkit Changes:
122289ff Add FaceAttachable interface to handle Grindstone facing in common with Switches
a6db750e SPIGOT-5647: ZombieVillager entity should have getVillagerType()

CraftBukkit Changes:
bbe3d58e SPIGOT-5650: Lectern.setPage(int) causes a NullPointerException
3075579f Add FaceAttachable interface to handle Grindstone facing in common with Switches
95bd4238 SPIGOT-5647: ZombieVillager entity should have getVillagerType()
4d975ac3 SPIGOT-5617: setBlockData does not work when NotPlayEvent is called by redstone current
2020-04-02 17:07:06 -04:00
Aikar
6b48080833 And that's why hand editing patch files in nano is risky.... make it compile 2020-04-02 03:58:46 -04:00
Aikar
bbd6a5ac5f Improve Entities in water activation immunity and let items always move 2020-04-02 03:56:05 -04:00
Aikar
d4d5552df0 Optimize Collision Chunk lookup and avoid loading far chunks
Try to use a faster chunk lookup for collision detection, and only
fall back to the original for nearby chunks.

The collision code takes an AABB and generates a cuboid of checks rather
than a cylinder, so at high velocity this can generate a lot of chunk checks.
2020-04-02 02:42:24 -04:00
Aikar
845631969a Prevent Double PlayerChunkMap adds crashing server
Suspected case would be around the technique used in .stopRiding
Stack will identify any causer of this and warn instead of crashing.
2020-04-02 01:45:33 -04:00
Aikar
b667006970 Fix issues with Activation Range causing large chunk lookups.
Where I blocked movement did not consider velocity buildup, which I assume
then "unleashes" if something was really trying to push that entity, and moves
it a very large distance.

Additionally, this method was completely misnamed, as movementTick
is more "doLotsOfTickThings", and ended up breaking AI too, which the whole
point of temporary wake ups was to let AI run to trigger new immunity.

Also fixed numerous behavioral rules for Immunity to improve vanilla gameplay,
suchas bees that are angry or moving towards a flower or hive, any insentient
that is targetting any enemy (Accidently made it any player), and included flying
mobs such as phantoms by reducing the type check to insentient instead of Creature.

Also improved inWater immunity to consider if the mob is movable by water or not.
2020-04-02 01:25:36 -04:00
Aikar
305833ef12 Improve entity.getCurrentChunk() and use it for entity.isChunkLoaded() 2020-04-02 00:43:11 -04:00
chickeneer
673d46cae1 Do not allow bee's to load chunks for beehives 2020-03-31 03:54:42 -04:00
Aikar
4f615b0842 Remote Connections shouldn't hold up shutdown
Bugs in the connection logic appears to leave stale connections even, preventing shutdown
2020-03-31 03:52:57 -04:00
Aikar
328a6a5531 Fix bug in double register fix
could accidently delay entity reg by a tick if the bug this was fixing was encountered
2020-03-31 03:52:30 -04:00
Aikar
497043e491 Remove incorrect IO flush for save-all that doesn't have flush parameter
The entire reason the if statement exists is to only flush and print when done if flag is true

This avoids /save-all from hurting as much as it was before, such as from backup plugins.
2020-03-31 03:15:34 -04:00
Aikar
7a1b44c9de Fix unregistering entities from unloading chunks
CraftBukkit caused a regression here by making unloading chunks not
have a ticket added and returning unloaded future.

This caused entities who were killed in same tick their chunk is unloading
to not be able to be removed from the chunk.

This then results in dead entities lingering in the Chunk.

Combine that with a buggy detail of the previous implementation of
the Dupe UUID patch, then this was the likely source of the "Ghost entities"
2020-03-31 03:05:04 -04:00
Aikar
4bbe3334fb Ensure Entity is never double registered
If something calls register twice, and the world is ticking, it could be
enqueued to add twice.

Vs behavior of non ticking of just overwriting state.

We will now simply log a warning when this happens instead of crashing the server.
2020-03-31 03:04:47 -04:00
Aikar
01be191ab7 Fix many issues with dupe uuid resolve patch
This was not applied correctly, and would completely blow up chunk entity
registration if this feature was turned off....

Additionally, change how the entities are removed to be more consistent with other code.

Surface some of the logs indicating there is a problem as we are having so many issues with
entities that we don't need to be surpressing logs like that.
2020-03-31 02:52:12 -04:00