StreamBase Error and Exception Handling

Here are some observations about error and exception handling in StreamBase applications picked up over the years:

1. Be very familiar with the material on the Using Error Ports and Error Streams documentation page.
2. Make sure your unhandled StreamBase errors and Java exceptions are making into the log at ERROR level. That way admins and operators can find them easily.
3. The logging subsystem prior to StreamBase 10 didn't capture the stderr stream and that had to be captured separately, but I think that is fixed in StreamBase 10.
4. The StreamBase 10 runtime has multiple log files where errors and exceptions end up; monitor them all.
5. If you ever see a Java OutOfMemoryError, that's a catastrophic failure and the integrity of data even before the error appeared is questionable. Shut down immediately, and expect to do a manual reconciliation.
6. StreamBase errors are analogous to Java exceptions, and any Java exceptions thrown by StreamBase components become StreamBase errors (usually eval-errors).
7. Java Errors are not StreamBase errors. Don't confuse them.
8. Error ports are for catching StreamBase errors that the application will handle programatically, potentially rethrowing the error if it can't be handled programatically. They are not generally used as a mechanism for generated errors to report to the application operators directly.
9. Adapter/Plugin Operator status ports may indeed emit tuples that eventually should be thrown as errors to be logged. Unfortunately, historically there has been no standard for the way status ports emit tuples and what their contents are, so it's case by case. There's convention being applied slowly across the adapter set for a Status port to have a Status field whose contents may be Error, and I would think in most cases those should be logged at ERROR level using the Log Adapter if they aren't being completely handled by application logic.