求助>应用3天左右CPU一直上升>
4回复

应用3天左右CPU一直上升



应用基本上第三天cpu就开始上升,并且一直不会降下来,应用没有什么流量,也不是什么CPU密集型应用,一直没找到原因。重启的时候,dump的log看有大量的http请求在等待,从日志没看出原因。

jvm重启之前dump的线程日志: https://thread.share.perfma.com/detail/1600198

jvm重启之前dump内存日志:
https://memory.share.perfma.com/detail/1600214

提前感谢

319 阅读
请先登录,再评论

回复列表

可以按照这篇文章来定位一下,具体耗 CPU 的线程,然后再来具体分析一下。 如何定位消耗CPU最多的线程

王虹凯1周前

这要分析 线程栈信息

还有这些日志提供参考

   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        - locked <0x0000000718b4ff90> (a sun.nio.ch.Util$2)
        - locked <0x0000000718b4ff80> (a java.util.Collections$UnmodifiableSet)
        - locked <0x0000000718b4fe58> (a sun.nio.ch.EPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:313)

"container-0" #17 prio=5 os_prio=0 tid=0x00007fd010dea000 nid=0x19 waiting on condition [0x00007fcfedbeb000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.catalina.core.StandardServer.await(StandardServer.java:570)
        at org.springframework.boot.web.embedded.tomcat.TomcatWebServer$1.run(TomcatWebServer.java:181)

"Catalina-utility-2" #16 prio=1 os_prio=0 tid=0x00007fd010de7000 nid=0x18 waiting on condition [0x00007fcfedcec000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x0000000718b7eae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:745)

"Catalina-utility-1" #15 prio=1 os_prio=0 tid=0x00007fd010ceb800 nid=0x17 waiting on condition [0x00007fcfedded000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x0000000718b7eae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:745)

"lettuce-eventExecutorLoop-1-1" #14 daemon prio=5 os_prio=0 tid=0x00007fd011a45800 nid=0x16 waiting on condition [0x00007fcff81fd000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x0000000718bceae0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
        at io.netty.util.concurrent.SingleThreadEventExecutor.takeTask(SingleThreadEventExecutor.java:256)
        at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:64)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:745)

"mysql-cj-abandoned-connection-cleanup" #12 daemon prio=5 os_prio=0 tid=0x00007fd0111a8000 nid=0x15 in Object.wait() [0x00007fd0009a8000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
        - locked <0x00000007194897c8> (a java.lang.ref.ReferenceQueue$Lock)
        at com.mysql.cj.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:85)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

"Service Thread" #7 daemon prio=9 os_prio=0 tid=0x00007fd010142000 nid=0x10 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C1 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007fd01013d000 nid=0xf waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007fd01013a800 nid=0xe waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007fd010139000 nid=0xd waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007fd010101000 nid=0xc in Object.wait() [0x00007fd0143a4000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
        - locked <0x00000007180f0768> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007fd0100ff000 nid=0xb in Object.wait() [0x00007fd0144a5000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
        - locked <0x00000007180f0148> (a java.lang.ref.Reference$Lock)

"VM Thread" os_prio=0 tid=0x00007fd0100fa000 nid=0xa runnable

"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007fd01001e000 nid=0x8 runnable

"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007fd010020000 nid=0x9 runnable

"VM Periodic Task Thread" os_prio=0 tid=0x00007fd010145000 nid=0x11 waiting on condition

JNI global references: 2069

Heap
 PSYoungGen      total 1045504K, used 776163K [0x0000000780000000, 0x00000007c0000000, 0x00000007c0000000)
  eden space 1042432K, 74% used [0x0000000780000000,0x00000007af34ce10,0x00000007bfa00000)
  from space 3072K, 89% used [0x00000007bfd00000,0x00000007bffac110,0x00000007c0000000)
  to   space 3072K, 0% used [0x00000007bfa00000,0x00000007bfa00000,0x00000007bfd00000)
 ParOldGen       total 1703936K, used 197948K [0x0000000718000000, 0x0000000780000000, 0x0000000780000000)
  object space 1703936K, 11% used [0x0000000718000000,0x000000072414f168,0x0000000780000000)
 Metaspace       used 71989K, capacity 76812K, committed 76928K, reserved 1116160K
  class space    used 8714K, capacity 9527K, committed 9600K, reserved 1048576K

孙邺鑫1周前
回复 手机用户162382:

可以先通过top 找出哪一个线程占用了CPU资源,然后再进一步分析

回复