To create and distribute, since long, we have had the power of Oracle Reports at our fingertips. These same report definitions can still be used successfully nowadays.
With all the technical changes over the years, we’ve come to a point where Oracle Reports Server, is now part of Oracle Fusion Middleware proposition. It is a (small) container, application or component of this software stack, where it is loaded in an application server and thus deployed.
Although this takes Oracle Reports Server into the modern age, it adds a lot of complexity for smaller implementations, a lot of tweaking and tuning to get it all running smoothly. One of these examples I would like to address here…
We were faced with a situation where we were running a report, an e-mail distribution report, sending out specific information to a multitude of e-mail addresses. And there was this situation where this report was run on a system, which was somewhat overloaded. This is not good, but not always completely predictable or avoidable.
For this report to run very long, actually was not a big issue as these individual e-mails were just informational and not time-bound on a minute-scale.
The problems started when we were getting word that some recipients of these e-mails were getting multitudes of copies of these e-mails! Which was sloppy at best…
Wait… multitudes of e-mail from a job which is running slow… That’s odd, they should be getting either a late e-mail or perhaps no e-mail at all! Not a lot of e-mails, that is contradictory!
This called for an investigation.
Finally we stumbled across a technology designed to do no harm, I would like to call your attention to idempotence!
Funny thing is, this setting is not part of Oracle Reports Server, but of another component used in this complexity, Oracle HTTP Server (OHS).
- “When set to ON and if the servers do not respond within WLIOTimeoutSecs (new name for HungServerRecoverSecs), the plug-ins fail over.”
- As stated on WikiPedia, an idempotent operation is “that can be applied multiple times without changing the result beyond the initial application”
Obviously, this was not the case with our situation! This idempotent operation was applied multiple times and it did change the result far beyond the initial application. Up to 40 e-mails per recipients on one specific occasion.
In effect, what happened, was that the Oracle Report, which was called through a URL, did not complete within WLIOTimeoutSecs, was just restarted. The URL, as it was called to start the process, was called again from withing the Oracle Fusion Middleware Stack, starting a new run of e-mail distribution… over and over again.
We resolved the situation by two completely different actions.
- we made sure this operation was not run on a system which was too busy to handle the load
- we made sure Oracle Fusion Middleware wouldn’t get the crazy idea to re-run this operation again.
Number two was fixed by the adding of the following instruction to the installation guide (and actually doing this too):
Toggle parameter IdemPotent to ‘off’ in reports_ohs.conf in: <drive>\oracle\middleware\asinst_1\config\OHS\ohs1\moduleconf\reports_ohs.conf This can be accomplished through the Enterprise Manager Webtier -> Ohs1 -> Oracle http Server -> Administration -> Advanced Configuration -> Choose a file -> reports_ohs.conf Add the following line to the body: ## Added <name> <date> IdemPotent off
Our problems are gone now!