OpenSource/PaperMC
OpenSource/PaperMC
1
0
Fork 0
mirror of https://github.com/PaperMC/Paper.git synced 2024-12-30 16:19:03 +01:00
Code Issues Projects Releases Packages Wiki Activity
PaperMC/Spigot-Server-Patches/Optimize-NibbleArray-to-use-pooled-buffers.patch

376 lines
17 KiB
Diff
Raw Normal View History

Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Aikar <aikar@aikar.co>
Date: Wed, 6 May 2020 23:30:30 -0400
Subject: [PATCH] Optimize NibbleArray to use pooled buffers
Massively reduces memory allocation of 2048 byte buffers by using
an object pool for these.
Uses lots of advanced new capabilities of the Paper codebase :)
diff --git a/src/main/java/net/minecraft/server/ChunkRegionLoader.java b/src/main/java/net/minecraft/server/ChunkRegionLoader.java
index 0000000000000000000000000000000000000000..0000000000000000000000000000000000000000 100644
--- a/src/main/java/net/minecraft/server/ChunkRegionLoader.java
+++ b/src/main/java/net/minecraft/server/ChunkRegionLoader.java
@@ -0,0 +0,0 @@ public class ChunkRegionLoader {
}
if (nibblearray != null && !nibblearray.c()) {
- nbttagcompound2.setByteArray("BlockLight", nibblearray.asBytes());
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
+ nbttagcompound2.setByteArray("BlockLight", nibblearray.asBytesPoolSafe().clone()); // Paper
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
}
if (nibblearray1 != null && !nibblearray1.c()) {
- nbttagcompound2.setByteArray("SkyLight", nibblearray1.asBytes());
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
+ nbttagcompound2.setByteArray("SkyLight", nibblearray1.asBytesPoolSafe().clone()); // Paper
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
}
nbttaglist.add(nbttagcompound2);
diff --git a/src/main/java/net/minecraft/server/LightEngineStorage.java b/src/main/java/net/minecraft/server/LightEngineStorage.java
index 0000000000000000000000000000000000000000..0000000000000000000000000000000000000000 100644
--- a/src/main/java/net/minecraft/server/LightEngineStorage.java
+++ b/src/main/java/net/minecraft/server/LightEngineStorage.java
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
@@ -0,0 +0,0 @@ public abstract class LightEngineStorage<M extends LightEngineStorageArray<M>> e
protected NibbleArray j(long i) {
NibbleArray nibblearray = (NibbleArray) this.i.get(i);
- return nibblearray != null ? nibblearray : new NibbleArray();
+ return nibblearray != null ? nibblearray : new NibbleArray().markPoolSafe(); // Paper
}
protected void a(LightEngineLayer<?, ?> lightenginelayer, long i) {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
@@ -0,0 +0,0 @@ public abstract class LightEngineStorage<M extends LightEngineStorageArray<M>> e
Optimize Light Engine Massive update to light to improve performance and chunk loading/generation. 1) Massive bit packing/unpacking optimizations and inlining. A lot of performance has to do with constant packing and unpacking of bits. We now inline a most bit operations, and re-use base x/y/z bits in many places. This helps with cpu level processing to just do all the math at once instead of having to jump in and out of function calls. This much logic also is likely over the JVM Inline limit for JIT too. 2) Applied a few of JellySquid's Phosphor mod optimizations such as - ensuring we don't notify neighbor chunks when neighbor chunk doesn't need to be notified - reduce hasLight checks in initializing light, and prob some more, they are tagged JellySquid where phosphor influence was used. 3) Optimize hot path accesses to getting updating chunk to have less branching 4) Optimize getBlock accesses to have less branching, and less unpacking 5) Have a separate urgent bucket for chunk light tasks. These tasks will always cut in line over non blocking light tasks. 6) Retain chunk priority while light tasks are enqueued. So if a task comes in at high priority but the queue is full of tasks already at a lower priority, before the task was simply added to the end. Now it can cut in line to the front. this applies for both urgent and non urgent tasks. 7) Buffer non urgent tasks even if queueUpdate is called multiple times to improve efficiency. 8) Fix NPE risk that crashes server in getting nibble data Fixes #3489 Fixes #3363
2020-06-05 07:25:11 +02:00
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
protected void a(long i, @Nullable NibbleArray nibblearray, boolean flag) {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
if (nibblearray != null) {
Optimize Light Engine Massive update to light to improve performance and chunk loading/generation. 1) Massive bit packing/unpacking optimizations and inlining. A lot of performance has to do with constant packing and unpacking of bits. We now inline a most bit operations, and re-use base x/y/z bits in many places. This helps with cpu level processing to just do all the math at once instead of having to jump in and out of function calls. This much logic also is likely over the JVM Inline limit for JIT too. 2) Applied a few of JellySquid's Phosphor mod optimizations such as - ensuring we don't notify neighbor chunks when neighbor chunk doesn't need to be notified - reduce hasLight checks in initializing light, and prob some more, they are tagged JellySquid where phosphor influence was used. 3) Optimize hot path accesses to getting updating chunk to have less branching 4) Optimize getBlock accesses to have less branching, and less unpacking 5) Have a separate urgent bucket for chunk light tasks. These tasks will always cut in line over non blocking light tasks. 6) Retain chunk priority while light tasks are enqueued. So if a task comes in at high priority but the queue is full of tasks already at a lower priority, before the task was simply added to the end. Now it can cut in line to the front. this applies for both urgent and non urgent tasks. 7) Buffer non urgent tasks even if queueUpdate is called multiple times to improve efficiency. 8) Fix NPE risk that crashes server in getting nibble data Fixes #3489 Fixes #3363
2020-06-05 07:25:11 +02:00
- this.i.put(i, nibblearray);
+ NibbleArray remove = this.i.put(i, nibblearray); if (remove != null && remove.cleaner != null) remove.cleaner.run(); // Paper - clean up when removed
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
if (!flag) {
this.n.add(i);
}
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
} else {
- this.i.remove(i);
+ NibbleArray remove = this.i.remove(i); if (remove != null && remove.cleaner != null) remove.cleaner.run(); // Paper - clean up when removed
}
}
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
diff --git a/src/main/java/net/minecraft/server/LightEngineStorageArray.java b/src/main/java/net/minecraft/server/LightEngineStorageArray.java
index 0000000000000000000000000000000000000000..0000000000000000000000000000000000000000 100644
--- a/src/main/java/net/minecraft/server/LightEngineStorageArray.java
+++ b/src/main/java/net/minecraft/server/LightEngineStorageArray.java
@@ -0,0 +0,0 @@ public abstract class LightEngineStorageArray<M extends LightEngineStorageArray<
public void a(long i) {
if (this.isVisible) { throw new IllegalStateException("writing to visible data"); } // Paper - avoid copying light data
- this.data.queueUpdate(i, ((NibbleArray) this.data.getUpdating(i)).b()); // Paper - avoid copying light data
Fix concurrency issue in light engine (Vanilla bug) Mojang implemented a cache like chunks have, but this cache is accessed by multiple threads and is totally not safe. So just remove it Fixes #3466 Also missed a pooled nibble release, so slid that in there too.
2020-05-28 22:59:38 +02:00
+ NibbleArray updating = this.data.getUpdating(i); // Paper - pool nibbles
+ this.data.queueUpdate(i, new NibbleArray().markPoolSafe(updating.getCloneIfSet())); // Paper - avoid copying light data - pool safe clone
Improve Chunk Prioritization and Internal Scheduler In previous MC versions, we had a rather simple internal scheduler for delayed tasks that would just keep pushing task back until desired tick was reached. The method it called to schedule the task changed behavior in 1.14, and now this scheduler is not working nowhere near what it was supposed to be doing. This was causing long delayed task to eat up CPU (In Oversleep for example) Rewrite this to just use the CraftScheduler for scheduling delayed tasks. Once this was fixed, it became quite clear the code that delayed ticket additions for chunks based on distance was clearly not right, as it was tested on the previous broken logic. So the ticket delay process has been vastly revamped to be even smarter. Chunks behind the player can load slower than the chunks in front of the player. We also can delay ticket adding until one of its neighbors has loaded, as this lets us get a smoother spiral out for the chunks (minus frustum intent). Additionally on frustum previous commit inadvertently broke frustum trying to fix an issue when the real fix lied elsewhere, so restore chunk priority so it works again.
2020-06-09 09:17:25 +02:00
+ if (updating.cleaner != null) MCUtil.scheduleTask(2, updating.cleaner, "Light Engine Release"); // Paper - delay clean incase anything holding ref was still using it
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
this.c();
}
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
diff --git a/src/main/java/net/minecraft/server/LightEngineStorageSky.java b/src/main/java/net/minecraft/server/LightEngineStorageSky.java
index 0000000000000000000000000000000000000000..0000000000000000000000000000000000000000 100644
--- a/src/main/java/net/minecraft/server/LightEngineStorageSky.java
+++ b/src/main/java/net/minecraft/server/LightEngineStorageSky.java
@@ -0,0 +0,0 @@ public class LightEngineStorageSky extends LightEngineStorage<LightEngineStorage
j = SectionPosition.a(j, EnumDirection.UP);
}
- return new NibbleArray((new NibbleArrayFlat(nibblearray1, 0)).asBytes());
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
+ return new NibbleArray().markPoolSafe(new NibbleArrayFlat(nibblearray1, 0).asBytes()); // Paper - mark pool use as safe (no auto cleaner)
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
} else {
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
- return new NibbleArray();
+ return new NibbleArray().markPoolSafe(); // Paper - mark pool use as safe (no auto cleaner)
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
}
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
}
}
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
@@ -0,0 +0,0 @@ public class LightEngineStorageSky extends LightEngineStorage<LightEngineStorage
((LightEngineStorageSky.a) this.f).a(i);
}
- Arrays.fill(this.a(i, true).asBytes(), (byte) -1);
+ Arrays.fill(this.a(i, true).asBytesPoolSafe(), (byte) -1); // Paper
k = SectionPosition.c(SectionPosition.b(i));
l = SectionPosition.c(SectionPosition.c(i));
int i1 = SectionPosition.c(SectionPosition.d(i));
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
diff --git a/src/main/java/net/minecraft/server/NibbleArray.java b/src/main/java/net/minecraft/server/NibbleArray.java
index 0000000000000000000000000000000000000000..0000000000000000000000000000000000000000 100644
--- a/src/main/java/net/minecraft/server/NibbleArray.java
+++ b/src/main/java/net/minecraft/server/NibbleArray.java
@@ -0,0 +0,0 @@
package net.minecraft.server;
+import com.destroystokyo.paper.util.pooled.PooledObjects; // Paper
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
+
+import javax.annotation.Nonnull;
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
import javax.annotation.Nullable;
public class NibbleArray {
- @Nullable
- protected byte[] a;
+ // Paper start
+ public static byte[] EMPTY_NIBBLE = new byte[2048];
+ private static final int nibbleBucketSizeMultiplier = Integer.getInteger("Paper.nibbleBucketSize", 3072);
Allow server startup for those poor people running <1G Xmx Clamp logic for poolsize so we dont pass zero
2020-05-17 06:06:19 +02:00
+ private static final int maxPoolSize = Integer.getInteger("Paper.maxNibblePoolSize", (int) Math.min(6, Math.max(1, Runtime.getRuntime().maxMemory() / 1024 / 1024 / 1024)) * (nibbleBucketSizeMultiplier * 8));
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
+ public static final PooledObjects<byte[]> BYTE_2048 = new PooledObjects<>(() -> new byte[2048], maxPoolSize);
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ public static void releaseBytes(byte[] bytes) {
+ if (bytes != null && bytes != EMPTY_NIBBLE && bytes.length == 2048) {
+ System.arraycopy(EMPTY_NIBBLE, 0, bytes, 0, 2048);
+ BYTE_2048.release(bytes);
+ }
+ }
+
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
+ public NibbleArray markPoolSafe(byte[] bytes) {
+ if (bytes != EMPTY_NIBBLE) this.a = bytes;
+ return markPoolSafe();
+ }
+ public NibbleArray markPoolSafe() {
+ poolSafe = true;
+ return this;
+ }
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ public byte[] getIfSet() {
+ return this.a != null ? this.a : EMPTY_NIBBLE;
+ }
+ public byte[] getCloneIfSet() {
+ if (a == null) {
+ return EMPTY_NIBBLE;
+ }
+ byte[] ret = BYTE_2048.acquire();
+ System.arraycopy(getIfSet(), 0, ret, 0, 2048);
+ return ret;
+ }
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
+
+ public NibbleArray cloneAndSet(byte[] bytes) {
+ if (bytes != null && bytes != EMPTY_NIBBLE) {
+ this.a = BYTE_2048.acquire();
+ System.arraycopy(bytes, 0, this.a, 0, 2048);
+ }
+ return this;
+ }
+ boolean poolSafe = false;
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ public java.lang.Runnable cleaner;
Optimize performance of object pool synchronized arraydeque ends up still being way faster. Kinda shocked how much that strategy was using, it wasn't really that complicated... but oh well, this is even simpler and not seeing blocked threads show up at all in profiling because the lock is held for such a short amount of time. also because most uses are on either server thread pool or chunk load pool. Also optimize the pooling of nibbles to not register Cleaner's for Light Engine directed usages, as we know we are properly controlling clean up there, so we don't need to rely on GC. This will return them to the pool manually, saving a lot of Cleaners. Closes #3417
2020-05-21 03:45:43 +02:00
+ private void registerCleaner() {
+ if (!poolSafe) {
+ cleaner = MCUtil.registerCleaner(this, this.a, NibbleArray::releaseBytes);
+ } else {
+ cleaner = MCUtil.once(() -> NibbleArray.releaseBytes(this.a));
+ }
+ }
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ // Paper end
+ @Nullable protected byte[] a;
+
public NibbleArray() {}
public NibbleArray(byte[] abyte) {
+ // Paper start
+ this(abyte, false);
+ }
+ public NibbleArray(byte[] abyte, boolean isSafe) {
this.a = abyte;
+ if (!isSafe) this.a = getCloneIfSet(); // Paper - clone for safety
+ registerCleaner();
+ // Paper end
if (abyte.length != 2048) {
throw (IllegalArgumentException) SystemUtils.c(new IllegalArgumentException("ChunkNibbleArrays should be 2048 bytes not: " + abyte.length));
}
@@ -0,0 +0,0 @@ public class NibbleArray {
public void a(int i, int j) { // PAIL: private -> public
if (this.a == null) {
- this.a = new byte[2048];
+ this.a = BYTE_2048.acquire(); // Paper
+ registerCleaner();// Paper
}
int k = this.d(i);
@@ -0,0 +0,0 @@ public class NibbleArray {
public byte[] asBytes() {
if (this.a == null) {
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
this.a = new byte[2048];
+ } else { // Paper start
+ // Accessor may need this object past garbage collection so need to clone it and return pooled value
+ // If we know its safe for pre GC access, use asBytesPoolSafe(). If you just need read, use getIfSet()
+ Runnable cleaner = this.cleaner;
+ if (cleaner != null) {
+ this.a = this.a.clone();
+ cleaner.run(); // release the previously pooled value
+ this.cleaner = null;
+ }
+ }
+ // Paper end
+
+ return this.a;
+ }
+
+ @Nonnull
+ public byte[] asBytesPoolSafe() {
+ if (this.a == null) {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ this.a = BYTE_2048.acquire(); // Paper
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
+ registerCleaner(); // Paper
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
}
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
+ //noinspection ConstantConditions
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
return this.a;
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
}
+ // Paper end
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
public NibbleArray copy() { return this.b(); } // Paper - OBFHELPER
public NibbleArray b() {
- return this.a == null ? new NibbleArray() : new NibbleArray((byte[]) this.a.clone());
+ return this.a == null ? new NibbleArray() : new NibbleArray(this.a); // Paper - clone in ctor
}
public String toString() {
diff --git a/src/main/java/net/minecraft/server/NibbleArrayFlat.java b/src/main/java/net/minecraft/server/NibbleArrayFlat.java
index 0000000000000000000000000000000000000000..0000000000000000000000000000000000000000 100644
--- a/src/main/java/net/minecraft/server/NibbleArrayFlat.java
+++ b/src/main/java/net/minecraft/server/NibbleArrayFlat.java
@@ -0,0 +0,0 @@ public class NibbleArrayFlat extends NibbleArray {
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
public NibbleArrayFlat(NibbleArray nibblearray, int i) {
super(128);
- System.arraycopy(nibblearray.asBytes(), i * 128, this.a, 0, 128);
+ System.arraycopy(nibblearray.getIfSet(), i * 128, this.a, 0, 128); // Paper
}
@Override
@@ -0,0 +0,0 @@ public class NibbleArrayFlat extends NibbleArray {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
@Override
public byte[] asBytes() {
- byte[] abyte = new byte[2048];
+ byte[] abyte = BYTE_2048.acquire(); // Paper
for (int i = 0; i < 16; ++i) {
System.arraycopy(this.a, 0, abyte, i * 128, 128);
diff --git a/src/main/java/net/minecraft/server/PacketPlayOutLightUpdate.java b/src/main/java/net/minecraft/server/PacketPlayOutLightUpdate.java
index 0000000000000000000000000000000000000000..0000000000000000000000000000000000000000 100644
--- a/src/main/java/net/minecraft/server/PacketPlayOutLightUpdate.java
+++ b/src/main/java/net/minecraft/server/PacketPlayOutLightUpdate.java
Latest progress - restored patches, some issues resolved
2020-06-26 08:29:44 +02:00
@@ -0,0 +0,0 @@
package net.minecraft.server;
import com.google.common.collect.Lists;
+import io.netty.channel.ChannelFuture; // Paper
+
import java.io.IOException;
import java.util.Iterator;
import java.util.List;
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
@@ -0,0 +0,0 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
private List<byte[]> h;
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
private boolean i;
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ // Paper start
+ java.lang.Runnable cleaner1;
+ java.lang.Runnable cleaner2;
+ java.util.concurrent.atomic.AtomicInteger remainingSends = new java.util.concurrent.atomic.AtomicInteger(0);
+
+ @Override
+ public void onPacketDispatch(EntityPlayer player) {
+ remainingSends.incrementAndGet();
+ }
+
+ @Override
Latest progress - restored patches, some issues resolved
2020-06-26 08:29:44 +02:00
+ public void onPacketDispatchFinish(EntityPlayer player, ChannelFuture future) {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ if (remainingSends.decrementAndGet() <= 0) {
+ // incase of any race conditions, schedule this delayed
Optimize Light Engine Massive update to light to improve performance and chunk loading/generation. 1) Massive bit packing/unpacking optimizations and inlining. A lot of performance has to do with constant packing and unpacking of bits. We now inline a most bit operations, and re-use base x/y/z bits in many places. This helps with cpu level processing to just do all the math at once instead of having to jump in and out of function calls. This much logic also is likely over the JVM Inline limit for JIT too. 2) Applied a few of JellySquid's Phosphor mod optimizations such as - ensuring we don't notify neighbor chunks when neighbor chunk doesn't need to be notified - reduce hasLight checks in initializing light, and prob some more, they are tagged JellySquid where phosphor influence was used. 3) Optimize hot path accesses to getting updating chunk to have less branching 4) Optimize getBlock accesses to have less branching, and less unpacking 5) Have a separate urgent bucket for chunk light tasks. These tasks will always cut in line over non blocking light tasks. 6) Retain chunk priority while light tasks are enqueued. So if a task comes in at high priority but the queue is full of tasks already at a lower priority, before the task was simply added to the end. Now it can cut in line to the front. this applies for both urgent and non urgent tasks. 7) Buffer non urgent tasks even if queueUpdate is called multiple times to improve efficiency. 8) Fix NPE risk that crashes server in getting nibble data Fixes #3489 Fixes #3363
2020-06-05 07:25:11 +02:00
+ MCUtil.scheduleTask(5, () -> {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ if (remainingSends.get() == 0) {
+ cleaner1.run();
+ cleaner2.run();
+ }
Improve Chunk Prioritization and Internal Scheduler In previous MC versions, we had a rather simple internal scheduler for delayed tasks that would just keep pushing task back until desired tick was reached. The method it called to schedule the task changed behavior in 1.14, and now this scheduler is not working nowhere near what it was supposed to be doing. This was causing long delayed task to eat up CPU (In Oversleep for example) Rewrite this to just use the CraftScheduler for scheduling delayed tasks. Once this was fixed, it became quite clear the code that delayed ticket additions for chunks based on distance was clearly not right, as it was tested on the previous broken logic. So the ticket delay process has been vastly revamped to be even smarter. Chunks behind the player can load slower than the chunks in front of the player. We also can delay ticket adding until one of its neighbors has loaded, as this lets us get a smoother spiral out for the chunks (minus frustum intent). Additionally on frustum previous commit inadvertently broke frustum trying to fix an issue when the real fix lied elsewhere, so restore chunk priority so it works again.
2020-06-09 09:17:25 +02:00
+ }, "Light Packet Release");
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
+ }
+ }
+
+ @Override
+ public boolean hasFinishListener() {
+ return true;
+ }
+
+ // Paper end
public PacketPlayOutLightUpdate() {}
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
public PacketPlayOutLightUpdate(ChunkCoordIntPair chunkcoordintpair, LightEngine lightengine, boolean flag) {
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
this.a = chunkcoordintpair.x;
this.b = chunkcoordintpair.z;
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
this.i = flag;
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
- this.g = Lists.newArrayList();
- this.h = Lists.newArrayList();
+ this.g = Lists.newArrayList();cleaner1 = MCUtil.registerListCleaner(this, this.g, NibbleArray::releaseBytes); // Paper
+ this.h = Lists.newArrayList();cleaner2 = MCUtil.registerListCleaner(this, this.h, NibbleArray::releaseBytes); // Paper
for (int i = 0; i < 18; ++i) {
NibbleArray nibblearray = lightengine.a(EnumSkyBlock.SKY).a(SectionPosition.a(chunkcoordintpair, -1 + i));
@@ -0,0 +0,0 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
this.e |= 1 << i;
} else {
this.c |= 1 << i;
- this.g.add(nibblearray.asBytes().clone());
+ this.g.add(nibblearray.getCloneIfSet()); // Paper
}
}
@@ -0,0 +0,0 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
this.f |= 1 << i;
} else {
this.d |= 1 << i;
- this.h.add(nibblearray1.asBytes().clone());
+ this.h.add(nibblearray1.getCloneIfSet()); // Paper
}
}
}
@@ -0,0 +0,0 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
Current non compilable status of all patches - THIS IS NOT READY THERE IS STILL NO ETA. GOBLINS WILL EAT YOU.
2020-06-26 03:58:00 +02:00
this.i = flag;
Optimize NibbleArray to use pooled buffers Massively reduces memory allocation of 2048 byte buffers by using an object pool for these. Uses lots of advanced new capabilities of the Paper codebase :) Targets 3072 * 8 buffers per 1GB of heap memory up to a max consideration of 6GB of heap (any more over 6GB won't give more nibble pool) You can control the 3072 number by setting -DPaper.nibbleBucketSize=2048 Remember this number is * 8 then * heap memory in GB That is 98304 objects for 4GB of memory, at 2064 bytes roughly, meaning 194MB You may also control max number of pooled objects directly instead of any dynamic calculation using -DPaper.maxNibblePoolSize=1024000 While this will use more old generation by a tad bit, allocation rate will drop significantly, causing less young generation GC's. This commit has gone through extensive testing for over a day and confident it no longer has any issues with light corruption.
2020-05-17 03:41:56 +02:00
this.c = i;
this.d = j;
- this.g = Lists.newArrayList();
- this.h = Lists.newArrayList();
+ this.g = Lists.newArrayList();cleaner1 = MCUtil.registerListCleaner(this, this.g, NibbleArray::releaseBytes); // Paper
+ this.h = Lists.newArrayList();cleaner2 = MCUtil.registerListCleaner(this, this.h, NibbleArray::releaseBytes); // Paper
for (int k = 0; k < 18; ++k) {
NibbleArray nibblearray;
@@ -0,0 +0,0 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
if ((this.c & 1 << k) != 0) {
nibblearray = lightengine.a(EnumSkyBlock.SKY).a(SectionPosition.a(chunkcoordintpair, -1 + k));
if (nibblearray != null && !nibblearray.c()) {
- this.g.add(nibblearray.asBytes().clone());
+ this.g.add(nibblearray.getCloneIfSet()); // Paper
} else {
this.c &= ~(1 << k);
if (nibblearray != null) {
@@ -0,0 +0,0 @@ public class PacketPlayOutLightUpdate implements Packet<PacketListenerPlayOut> {
if ((this.d & 1 << k) != 0) {
nibblearray = lightengine.a(EnumSkyBlock.BLOCK).a(SectionPosition.a(chunkcoordintpair, -1 + k));
if (nibblearray != null && !nibblearray.c()) {
- this.h.add(nibblearray.asBytes().clone());
+ this.h.add(nibblearray.getCloneIfSet()); // Paper
} else {
this.d &= ~(1 << k);
if (nibblearray != null) {
Fix high memory use of non ticking chunks The nibble pooling for NBT Tags was 'semi' leaked from loaded chunks as we store the NBT Tag of Tile Entities in a Chunk, but don't process them and remove them until chunk reaches Entity Ticking status.... This caused some phantom references to persist causing high memory use of these chunks. So I just got rid of pooling from NBT deserialization and we'll have to take the hit on memory allocations there because too many cascading concerns with anyone using NBT Tag Byte Arrays. Fixes #3431
2020-05-23 00:58:26 +02:00
diff --git a/src/main/java/org/bukkit/craftbukkit/CraftChunk.java b/src/main/java/org/bukkit/craftbukkit/CraftChunk.java
index 0000000000000000000000000000000000000000..0000000000000000000000000000000000000000 100644
--- a/src/main/java/org/bukkit/craftbukkit/CraftChunk.java
+++ b/src/main/java/org/bukkit/craftbukkit/CraftChunk.java
@@ -0,0 +0,0 @@ public class CraftChunk implements Chunk {
sectionSkyLights[i] = emptyLight;
} else {
sectionSkyLights[i] = new byte[2048];
- System.arraycopy(skyLightArray.asBytes(), 0, sectionSkyLights[i], 0, 2048);
+ System.arraycopy(skyLightArray.getIfSet(), 0, sectionSkyLights[i], 0, 2048); // Paper
}
NibbleArray emitLightArray = lightengine.a(EnumSkyBlock.BLOCK).a(SectionPosition.a(x, i, z));
if (emitLightArray == null) {
sectionEmitLights[i] = emptyLight;
} else {
sectionEmitLights[i] = new byte[2048];
- System.arraycopy(emitLightArray.asBytes(), 0, sectionEmitLights[i], 0, 2048);
+ System.arraycopy(emitLightArray.getIfSet(), 0, sectionEmitLights[i], 0, 2048); // Paper
}
}
}
Reference in a new issue Copy permalink