WLDF: DomainRuntime MBean Server Missing

While configuring Watch and Notification or collecting Metrics for checking (Server state -For Example) whether it's running or not,  DomainRuntimeServer  is required as ServerRuntime dies along with the managed Server. Thus we won't be able to send an email alert as soon as a Managed Server dies using WLDF. 

Sometimes you will not be able see DomainRuntime in MBean server List when setting up WLDF.

Here is how you enable the DomainRuntimeMBean Server:

1. Create the Watch with ServerRuntime and configure the Watch with all the settings you need.

2. Once the settings are applied, edit the query inside the watch and change "ServerRuntime" to DomainRuntime.

3. Restart the whole Domain. You should see DomainRuntime in the MBean Server List. 

Example:

(${ServerRuntime//[weblogic.management.runtime.ServerLifeCycleRuntimeMBean]//State} != 'RUNNING')

Should be changed to

(${DomainRuntime//[weblogic.management.runtime.ServerLifeCycleRuntimeMBean]//State} != 'RUNNING') 

Why don't we see DomainRuntime Initially ?? 

WebLogic Server does not initialize the Domain Runtime MBean Server until a client requests a connection to it

Huge Pages & Memory Utilization

At times you may find that Unix machines are showing 100% memory utilized, however you may not be running anything on the machine .  This could happen for the following reasons:

1.       Zombie/defunct processes consuming memory

Command to find them if there any: ps aux | awk '{ print $8 " " $2 }' | grep -w Z

2.       Java processes consuming all the memory for several reasons (Eg : memory leaks, heap spaces consuming more memory than the physical RAM etc .. )

3.       Huge pages configured on the machine.

The 1^st & 2^nd points are common issues that you may want to look into as soon as an issue occurs. The 3^rd point is what we will be explaining here:

Look at a screen shot below of a linux machine that has nothing (No programs) running on it.

A description about Huge Pages: A large page is essentially a block of contiguous physical-memory addresses that are reserved for a process. These blocks of memory is blocked only for programs (user processes) running on the server.

 Here is how you can view information about huge pages:

If your applications are memory intensive then using huge pages is recommended. In a 64 bit Linux OS heaps as large as 80% of the available physical memory can be configured using huge pages.

References:

For Linux: http://www.mjmwired.net/kernel/Documentation/vm/hugetlbpage.txt

For Solaris: http://www.solarisinternals.com/wiki/index.php/Multiple_Page_Size_Support