Oracle Database - Parallel execution in an RAC environment
You can use the additional resources to:
- support additional users (and hence reduce the load on the other servers)
- and/or use the additional resources to directly improve the performance of the operations running on the database.
By default the Oracle database enables inter-node parallel execution (parallel execution of a single statement involving more than one node).
If you use a relatively weak interconnect, relative to the I/O bandwidth from the server to the storage configuration, then you may be better of restricting parallel execution to a single node or to a limited number of nodes.
Inter-node parallel execution will not scale with an undersized interconnect.
As a general rule of thumb, your interconnect must provide the total I/O throughput of all nodes in a cluster (since all nodes can distribute data at the same point in time with the speed data is read from disk).
So, if you have a four node cluster, each node being able to read 1GB/sec from the I/O subsystem, the interconnect must be able to support 4 x 1GB/sec = 4GB/sec to scale linearly for operations involving inter-node parallel execution.
It is not recommended to use inter-node parallel execution unless your interconnect satisfies this requirement (or comes very close).
Use instance_groups and parallel_instance_groups or database services (starting with Oracle Database 11g) to limit inter-node parallel execution. It is recommended to use services beginning with Oracle database 11g. The parameter instance_groups is going to be deprecated and only retained for backwards compatibility reasons.