Watchdog#
The SHARK Watchdog is a program that monitors the system operation and resources, and in case an unwanted condition is met, is able to restart services and send warnings by email and thereby, at an early state, can avoid system stops.
The Watchdog is used by defining rules that trigger actions. The following rules are available:
- SHARK link is reading files from an import folder.
- Check that a DirWatch task is running.
- Check that a file is updated regulary.
- Check that the size of a certain file is below a max limit.
- Ping an IP address.
- Check that free disk space is OK.
- Call a customized Stored Procedure for doing database checking.
- Actions that can be performed:
- None. Emails will be sent, but no other action taken.
- Restart a service.
- Run a system command (for example a script file).
The watchdog can also send an email report when a rule is triggered.
The Watchdog will read the Shark Registry at startup to get its configuration. The preferred way is then to avoid database access that might end in a block situation that will make the Watchdog stop working.
Configuration#
The watchdog is configured in the SHARK registry from the Administration menu → System Configuration.
| Parameter | Value | Note |
|---|---|---|
| Watchdog.ActionInvervalMin | Integer | Minimum value in seconds between actions are taken. Set to for example 1800 (30 minuts) to avoid repeated restarts and similar side effects. |
| Watchdog.CheckInterval | Integer | The time in seconds between the rules are executed. Typicals values 300 (5 minuts). |
| Debug | false | true |
| InstallationName | String | Used in email reports to identify the site. |
| ReportMethod | None | |
| WindowsSystemFolder | Path | Path to the Windows System32 folder (used to find for example the net.exe command). |
Windows Installation#
The Watchdog is running as a Windows Service using the gator_service.exe program. Typically it is installed by the standard SHARK installation program.
It can be installed manually by this command executed from the shark_2 bin folder:
sharkjservice.exe – i SharkWatchdog "Shark Watchdog" "dk/logiware/watchdog/Watchdog"
"C:\Program Files\Shark_2\bin\shark_watchdog.jar" "C:\Program Files\Shark_2\bin"
"C:\Program Files\Shark_2\jre\1.6.0\bin\client\jvm.dll"
It might be necessary to adjust the paths to the shark_watchdog.jar file, and to the Java folder depending on the actually installation.
After the service is installed the following entries are created in the Windows Registry:
Email Configuration#
The email configuration is used for sending messages when a rule is triggered. The email system required a valid SMTP server or it can use a web server at Logiware. The later solution has the advanced that it only requires port 80 access to the internet.
HTTP Configuration#
If HTTP is set as report method and if status reporting is enabled, the HTTP group of parameters must be defined.
EnableStatusReporting: If true the Watchdog will report the currenct status to Logiware (both successful and unsuccessful).
ProxyServer: If the Watchdog is running behind a proxy server, set here the name of the proxy.
ProxyServerPort: Port number used by the proxy server.
URL: URL where reporting is done, default is Logiwares Web Server.
Rule Types#
There are a number of different Rules that can be used. Each rule runs at fixed interval and do some checking, if the rule is triggered, it can use an action to try to fix the possible problem.
### Rule: Check SHARK link is importing files
This rule actually perform several checks to verify that Shark Link is running correctly.
It will check that the scheduler are running by checking that a registry time is constantly updated in the Shark registry.
It will check that the import folder (if any) does not contain old files that are not read by Shark Link and that it is able to create and delete a file in the import folder.
Rule: Ping - Try to ping an IP address#
The ping rule is used for checking the network connection to an IP address within specific intervals.
Rule: WatchDogTimer - Check a registry value is updated#
Checks a specific time entry in the Shark Registry is updated at regular intervals.
[!NOTE] Checking the registry forces the Watchdog to make a database access. If the problem is connection to the database, this might lead to situation where the Watchdog get blocked by a connection problem.
WatchDog → Rules → \<name>
Parameter |
Value |
Notes |
Rule |
WatchDogTimer |
|
RuleParameters |
<registry path>,<entry>,<delay> |
path: The registry path Entry: Entry is the parameter name. Delay: The delay in seconds before the rule is triggered. |
Action |
None|RestartService |
What to do when the rule is triggered |
ActionParameters |
RestartService: Service Name |
|
Name |
String |
Name of this rule (used for logging etc.) |
Enabled |
true|false |
|
The \<registry path> + \<entry> most point to a value that is timer updated at regular intervals. If the time read is older than the delay, then the rule will be triggered.
Example:
To monitor the Scheduler is running:
Set the Rule Parameters: “Scheduler.WatchDog,HeartBeat,120”
It points to the registry Scheduler.WatchDog.HearBeat that must contain a date, for example: “2013-07-19 12:05.00”.
It points to:
Rule: FreeDiskSpace - Check for free space on disk#
This rule checks that the amount of free disk space is larger than the specified value.
The rule requires an absolute path to a file on the disk to check, this file must exist to make to rule check working correctly.
Rule: SharkControl - Check Shark Control is running#
This rule checks that Shark Control is running and no devices are failing.
Rule: StoredProcedure - Call a Stored Procedure#
Call a stored procedure in the database and check the response from the procedure to check for errors.
The procedure must return two columns (int, varchar(..)). The first column is the status, if is different from 0 it is considered to be an error. The next column is the error message if any.
Rule: FileDate - Check file is updated#
Checks that the specified file is updated at regular intervals. The interval is specified in hours. Typical used to check that a backup file is generated/updated or that some other code, like a Groovy script is running.
Example of Groovy script that generates a file, that can be used by the Watchdog:
import java.text.SimpleDateFormat
...
// If the watchdog file is not updated, the watchdog will restart the service
if (updateWatchDog) {
def formattedDate = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date())
new File("C:/Program Files (x86)/Shark/watchdog/watchdog.txt").write("${formattedDate}: Importing Multicase orders")
}
## Rule: File is updated
Checks that a specified file is updated within a certain time. An alternative way to check another process is running. It as the advantage, that is will work without cheking anything in the database.
WatchDog → Rules →
| Parameter | Value | Notes |
|---|---|---|
| Rule | FileUpdated | |
| RuleParameters | file path | |
| Action | None/RestartService | What to do when the rule is triggered |
| ActionParameters | Leave empty | |
| Name | String | Name of this rule (used for logging etc.) |
| Enabled | true | false |
Rule: Cleanup Folder#
Cleanup files in the specified folder, that are older than the specified number of days.
This rule does not support actions.
| Parameter | Value | Notes |
|---|---|---|
| Rule | CleanupFolder | |
| RuleParameters | Not used | |
| Action | None | |
| ActionParameters | Leave empty | |
| Name | String | Name of this rule (used for logging etc.) |
| Enabled | true | false |
Rule: FileNotRead#
This rule will check if there are any files in the folder, older than a certain time.
It is typical used to check if the SHARK file importer is working correctly and may force a restart of the service, if required.
| Parameter | Value | Notes |
|---|---|---|
| Rule | FileNotRead | |
| RuleParameters | Not used | |
| Action | None | RestartService |
| ActionParameters | Typical the name of a service to restart. | |
| Name | String | Name of this rule (used for logging etc.) |
| Enabled | true | false |
| MaxTimeSeconds | 0-.. | The age of the file in seconds before the rule will be triggered. |
| Extension | \<file extension> | For example: XML |
| WatchFolder | \<path> | Path to the folder that will be monitored. |
Rule: RegistryFlag#
This rule is intended to restart a service with a flag in the Registry.
The rule will trigger, when the specified flag is true. Typical use is for the 2 standard supported flags:
- WebServer.Restart
- Scheduler.Restart
| Parameter | Value | Notes |
|---|---|---|
| Rule | RegistryFlag | |
| RuleParameters | Not used | |
| Action | RestartService | Other actions can also be used |
| ActionParameters | Typical the name of a service to restart. | |
| Name | String | Name of this rule (used for logging etc.) |
| Enabled | true | false |
| Parameters.RegistryPath | \<path> | Path to the registry entry that will be monitored. It must be of type boolean. When it is true the rule will be triggered. |
This configuration below will setup automatic restart of the Scheduler.
Actions#
When a rule is triggered, it can specify an action to be executed.
Action: None#
Do nothing
Action: RestartService#
Restarts the specified service
Action: Reboot#
Reboot the server.
Action: Execute#
Execute a Windows command.