微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦！

泄漏检测：在引用被垃圾回收之前未释放

分类：编程问答

如何解决泄漏检测：在引用被垃圾回收之前未释放

我在Cassandra 3.11.5中遇到以下错误

LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@356cb07f) to class org.apache.cassandra.io.util.SafeMemory$MemoryTidy@1709177197:Memory@[7e7a3b64b910..7e7a3b96b910) was not released before the reference was garbage collected


ERROR [CompactionExecutor:3763] 2020-11-02 02:34:53,316  CassandraDaemon.java:228 - Exception in thread Thread[CompactionExecutor:3763,1,main]
org.apache.cassandra.io.FSReadError: java.io.IOException: Map Failed
        at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:157) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:310) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:246) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:170) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions.<init>(MmappedRegions.java:73) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions.<init>(MmappedRegions.java:61) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:362) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:290) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.sstable.sstableRewriter.maybeReopenEarly(sstableRewriter.java:180) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.sstable.sstableRewriter.append(sstableRewriter.java:135) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:65) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:142) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:201) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:85) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:268) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_60]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_60]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_60]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_60]
        at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84) [apache-cassandra-3.11.5.jar:3.11.5]
        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_60]
Caused by: java.io.IOException: Map Failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:939) ~[na:1.8.0_60]
        at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:153) ~[apache-cassandra-3.11.5.jar:3.11.5]
        ... 23 common frames omitted
Caused by: java.lang.OutOfMemoryError: Map Failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method) ~[na:1.8.0_60]
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:936) ~[na:1.8.0_60]
        ... 24 common frames omitted
WARN  [GossipTasks:1] 2020-11-02 02:34:53,302  FailureDetector.java:278 - Not marking nodes down due to local pause of 7468254535 > 5000000000
ERROR [Reference-Reaper] 2020-11-02 02:34:53,208  Ref.java:229 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@33fef449) to class org.apache.cassandra.io.util.FileHandle$Cleanup@1098210945:/data/cassandra/data/elvisevallogks/decision_detail-7351f6205f3811ea922ce1efeeba3e49/md-3695-big-Index.db was not released before the reference was garbage collected
ERROR [CompactionExecutor:3762] 2020-11-02 02:34:53,208  CassandraDaemon.java:228 - Exception in thread Thread[CompactionExecutor:3762,main]
org.apache.cassandra.io.FSReadError: java.io.IOException: Map Failed
        at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:157) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions$State.add(MmappedRegions.java:310) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions$State.access$400(MmappedRegions.java:246) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions.updateState(MmappedRegions.java:181) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions.<init>(MmappedRegions.java:73) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions.<init>(MmappedRegions.java:61) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.MmappedRegions.map(MmappedRegions.java:104) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.util.FileHandle$Builder.complete(FileHandle.java:362) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.sstable.format.big.BigTableWriter.openEarly(BigTableWriter.java:290) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.sstable.sstableRewriter.maybeReopenEarly(sstableRewriter.java:180) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.io.sstable.sstableRewriter.append(sstableRewriter.java:135) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter.realAppend(DefaultCompactionWriter.java:65) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.writers.CompactionAwareWriter.append(CompactionAwareWriter.java:142) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:201) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:85) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:268) ~[apache-cassandra-3.11.5.jar:3.11.5]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_60]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_60]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_60]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_60]
        at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:84) [apache-cassandra-3.11.5.jar:3.11.5]
        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_60]
Caused by: java.io.IOException: Map Failed
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:939) ~[na:1.8.0_60]
        at org.apache.cassandra.io.util.ChannelProxy.map(ChannelProxy.java:153) ~[apache-cassandra-3.11.5.jar:3.11.5]
        ... 23 common frames omitted
Caused by: java.lang.OutOfMemoryError: Map Failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method) ~[na:1.8.0_60]
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:936) ~[na:1.8.0_60]
        ... 24 common frames omitted
INFO  [Service Thread] 2020-11-02 02:34:53,877  StatusLogger.java:51 - MemtablePostFlush                 0         0           3596         0                 0

这些是设置

# Cassandra-specfic parameters
# add
cassandra        soft    memlock         unlimited
cassandra        hard    memlock         unlimited
cassandra        soft    nproc           65535
#change
*                soft    nofile          655350
*                hard    nofile          655350

用于卡桑德拉工艺

cat /proc/25586/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    0                    bytes
Max resident set          unlimited            unlimited            bytes
Max processes             65535                692334               processes
Max open files            655350               655350               files
Max locked memory         unlimited            unlimited            bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       692334               692334               signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

我在线搜索，建议的设置是设置vm.max_map_count 1048575，并验证该值设置正确。堆大小设置为32 GB，并且计算机具有171 GB RAM。

sysctl vm.max_map_count
vm.max_map_count = 1048575

该节点不会崩溃，但是尝试连接时会出现随机超时。

解决方法

可能您可能需要检查并启用disk_access_mode从默认的“ Standard”到“ mmap_index_only”，它通过使用仅映射索引文件来避免页面错误。页面错误有时会导致内存泄漏，建议根据群集利用率将堆硬编码为特定大小。

问题在于，当堆默认为32或更高时，您会松散压缩堆中的许多对象，并且执行效果不佳，因此，当您有机会使用这两个更改时，这两个更改会很好滚动重启。

谢谢， Ganesh

版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。

苹果市值2025年有望达4万亿美元
pythonJavaScriptjavaHTMLreactjsC#AndroidCSSNode.jssqlrpython-3.xMysqLjQueryc++pandasFlutterangularIOSdjangolinuxswifttypescript路由器JSON路由器设置无线路由器h3c华三华三路由器设置华三路由器电脑软件教程arraysdocker软件图文教程Cvue.jslaravelspring-bootreact-native