Unsafe.defineClass 挂起 - 反射调用缓慢

问题描述:

我在 Java 8/Spring 5/Tomcat 8.5 Web 应用程序中观察到超长时间运行的请求(> 30 秒).大多数请求正常响应.值得注意的是,在正常运行几天后,长跑者的频率增加了.

I'm oberserving extreme long running requests (>30s) in a Java 8 / Spring 5 / Tomcat 8.5 web application. The majority of requests responds normally. It's remarkable that the frequency of long runners increases after some days of uptime.

线程转储显示 Unsafe.defineClass 挂起:

Threaddumps show that Unsafe.defineClass is hanging:

at sun.misc.Unsafe.defineClass(Native Method) 
at sun.reflect.ClassDefiner.defineClass(ClassDefiner.java:63) 
at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:399) 
at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:394) 
at java.security.AccessController.doPrivileged(Native Method)  
at sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:393) 
at sun.reflect.MethodAccessorGenerator.generateMethod(MethodAccessorGenerator.java:75) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:53) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:498) 
at org.springframework.core.annotation.AnnotationUtils.getValue(AnnotationUtils.java:1328)  
at org.springframework.core.annotation.AnnotatedElementUtils$MergedAnnotationAttributesProcessor.getAdaptedValue(AnnotatedElementUtils.java:1627) 
at org.springframework.core.annotation.AnnotatedElementUtils$MergedAnnotationAttributesProcessor.overrideAttributes(AnnotatedElementUtils.java:1610) 
at org.springframework.core.annotation.AnnotatedElementUtils$MergedAnnotationAttributesProcessor.postProcess(AnnotatedElementUtils.java:1598) 
at org.springframework.core.annotation.AnnotatedElementUtils$MergedAnnotationAttributesProcessor.postProcess(AnnotatedElementUtils.java:1515) 
at org.springframework.core.annotation.AnnotatedElementUtils.searchWithFindSemantics(AnnotatedElementUtils.java:1145) 
at org.springframework.core.annotation.AnnotatedElementUtils.searchWithFindSemantics(AnnotatedElementUtils.java:1066) 
at org.springframework.core.annotation.AnnotatedElementUtils.searchWithFindSemantics(AnnotatedElementUtils.java:1038) 
at org.springframework.core.annotation.AnnotatedElementUtils.findMergedAnnotationAttributes(AnnotatedElementUtils.java:650) 
at org.springframework.core.annotation.AnnotatedElementUtils.findMergedAnnotation(AnnotatedElementUtils.java:726)
....

有人见过这个问题吗?

可能是因为您使用了保守的 -XX:MaxMetaspaceSize 或没有足够的 RAM 来耗尽 Metaspace服务器?由于 GC 在 Metaspace 上运行,您的缓慢可能是它努力清理它的结果.默认情况下,Metaspace 内存在达到 MaxMetaspaceSize 时被收集,因此这 30 秒的暂停可能是 GC.

Could it be that you are running out of Metaspace either by using a conservative -XX:MaxMetaspaceSize or not having enough RAM on the server? Since GC runs on Metaspace your slowness might be the result of it struggling to clean it. By default the Metaspace memory is collected if it reaches the MaxMetaspaceSize so this 30 sec pause might be the GC.

然而,还有一个更大的问题.在您的代码中分配这些类的内容以及为什么它经常发生以填充元空间.通常类是在启动期间创建的,之后创建的频率要低得多.也许您正在进行热部署并且 ClassLoader 正在泄漏?

There is however a bigger question. What in your code is allocating these classes and why does it happen often enough to fill Metaspace. Normally classes are created during startup and after that much less frequently. Perhaps you are doing hot deployments and the ClassLoader is leaking?

在 Java 8 之前,这可能是由 GC 和 PermGen 耗尽引起的,例如在这个问题中被问到它导致OutOfMemoryError.

In pre Java 8 this could be caused by the GC and PermGen running out e.g. as asked in this question where it caused OutOfMemoryError.