diff options
| author | Howard Hinnant <hhinnant@apple.com> | 2013-10-30 15:10:54 +0000 |
|---|---|---|
| committer | Howard Hinnant <hhinnant@apple.com> | 2013-10-30 15:10:54 +0000 |
| commit | 811c96fa0e7626f4ba2c0844b670fa930d153d19 (patch) | |
| tree | 9b2bc80c3aabf5bf732920588f5b51cda9737cc5 | |
| parent | b364428fca3c7faf3f1e886e9b0501a6ecd2eea7 (diff) | |
| download | bcm5719-llvm-811c96fa0e7626f4ba2c0844b670fa930d153d19.tar.gz bcm5719-llvm-811c96fa0e7626f4ba2c0844b670fa930d153d19.zip | |
Rehash but don't grow when full of tombstones.
This problem was found and fixed by José Fonseca in March 2011 for
SmallPtrSet, committed r128566. But as far as I can tell, all other
llvm hash tables retain the same problem: the bucket count can grow
without bound while size() remains near constant by repeated
insert/erase cycles that tend to fill the container with tombstones.
Here is a demo that has been reduced to a trivial case:
int
main()
{
llvm::DenseSet<unsigned> d;
for (unsigned i = 0; i < 0xFFFFFFF; ++i)
{
d.insert(i);
d.erase(i);
}
}
While the container size() never grows above 1, the bucket count grows
like this:
nb = 64
nb = 128
nb = 256
nb = 512
nb = 1024
nb = 2048
nb = 4096
nb = 8192
nb = 16384
nb = 32768
nb = 65536
nb = 131072
nb = 262144
nb = 524288
nb = 1048576
nb = 2097152
nb = 4194304
nb = 8388608
nb = 16777216
nb = 33554432
nb = 67108864
nb = 134217728
nb = 268435456
The above program currently consumes a few GB ram. This patch brings
the memory consumption down by several orders of magnitude, and keeps
the bucket count at 64 for the above test.
llvm-svn: 193689
| -rw-r--r-- | llvm/include/llvm/ADT/DenseMap.h | 5 |
1 files changed, 2 insertions, 3 deletions
diff --git a/llvm/include/llvm/ADT/DenseMap.h b/llvm/include/llvm/ADT/DenseMap.h index fac0b958ea2..ce322cce4e0 100644 --- a/llvm/include/llvm/ADT/DenseMap.h +++ b/llvm/include/llvm/ADT/DenseMap.h @@ -438,9 +438,8 @@ private: this->grow(NumBuckets * 2); LookupBucketFor(Key, TheBucket); NumBuckets = getNumBuckets(); - } - if (NumBuckets-(NewNumEntries+getNumTombstones()) <= NumBuckets/8) { - this->grow(NumBuckets * 2); + } else if (NumBuckets-(NewNumEntries+getNumTombstones()) <= NumBuckets/8) { + this->grow(NumBuckets); LookupBucketFor(Key, TheBucket); } assert(TheBucket); |

