Igor Kromin |   Consultant. Coder. Blogger. Tinkerer. Gamer.

After fixing high CPU usage on WebLogic due to a corrupted file store I was left with a puzzle. On the hour, every hour the WebLogic Managed Server would see a small but detectable spike in CPU usage. These spikes varied between 10% and 20% and did not coincide with CPU spikes due to garbage collection. The server was otherwise idle, no other users accessing it, nothing scheduled to run on it, just sitting and doing nothing on the most part.

As I mentioned in the previous article where I described how I fixed high CPU usage in WebLogic, these spikes occurred on the hour. Specifically they were about 2.5 minutes past the hour. It didn't matter when the server was started/restarted, like clockwork at 2.5 minutes past every hour I would see a CPU spike...
wlscpuspike.png


From earlier debugging I knew that the high CPU usage was to do with a corrupted file store. This gave me a bit of a clue of where to look next. I looked over at this WebLogic documentation: Configure diagnostic archives. Here it had this interesting setting -
In the Store Size Check Period field, enter the interval, in minutes, between the times when the store will be checked to see if it has exceeded the preferred store size.


Ok the documentation appeared to be incorrectly stating this was in minutes, it was in fact in hours (later versions of the documentation corrected this). However, that was a periodic check but it did not indicate that it was a check that was scheduled on the hour. The documentation implied that the check simply happened after the specified period of time elapsed.

The default setting for Store Size Check Period was 1 hour. I decided to bump it up to 2 hours to see what change it would make. This was done on the Managed Server diagnostic archives configuration.



Managed Server restarted, on the next hour, CPU spike again. Obviously that setting had no effect. I then decided to see what the AdminServer had configured. It was the default 1 hour value, so I changed that to two and restarted both the Admin and Managed Servers.
wlscpuspike2.png


Success! On the first hour...no CPU spike! On the second hour, the CPU spike appeared in line with the new setting. It was interesting that only the AdminServer setting would be responsible for the change in behaviour even though the CPU spike was occurring on the JVM for the Managed Server. However I am not ruling out the possibility that because I didn't restart the AdminServer on my first setting change that these changes didn't actually come into effect.

That was a mystery solved, and it showed an interesting timing behaviour with WebLogic internal scheduling.

-i

Please leave your comments or feedback below!
comments powered by Disqus
Other posts you may like...
Hi! You can search my blog here ⤵
Or browse the recent top tags...

Recent Blog Posts

How to stop macOS adding shadows to window screenshots

How to run Atari Lynx games on the SNES Classic Mini

Maven dependency scopes with relation to WAR file packaging and the WEB-INF/lib directory

Hacking the Sonoff B1 WiFi LED bulb to run custom firmware

What does an idle WebLogic server run on the hour to cause a CPU spike

How to open the Sonoff B1 wifi LED bulb to access its internal circuitry

Australian release SNES mini classic unboxing and a quick play through

Troubleshooting high CPU usage for JVM threads

How to fix WebLogic high CPU usage due to a corrupted file store

Mini review of the Sonoff B1 WiFi light bulb

Recent Galleries

Atari Lynx repair - Part 5 - McWill LED screen mod installation

Atari Lynx repair - Part 4 - screen cover replacement

Atari Lynx repair - Part 2 - re-capping the motherboard

Atari Lynx repair - Part 3 - broken speaker replacement

Atari Lynx repair - Part 1 - introduction and case disassembly

Building a custom Atari Lynx game box storage shelf unit in a day

Protecting old Atari Lynx game boxes with snug fit plastic sleeves

Monument Valley 2 is released and does not disappoint

Space Food - Chocolate Ice Cream with Chocolate Chips

Legeod Star Wars AT-DP kit

Blogs and Friends

Matt Moores Blog
Georgi's FlatPress Guide
Perplexing Permutations
The Security Sleuth
Ilia Rogatchevski
Travelling Fairy

Blog Activity

Blog Activity