Review these Metalink notes:

BUG:9309554 – ERRORS WHEN MOUNTING A DBFS STORE USING FSTAB DUE TO MISSING LIBRARIES
NOTE:1052901.1 – Accessing DBFS mount point reports Input/Output error
NOTE:1053008.1 – Errors when mounting a DBFS store using fstab due to missing libraries

Considerations before start

YUM installation is recommended to avoid issues on dependencies.

dbfs_client pre-requisites

Apply “NOTE:1150157.1 – List of Critical Patches Required For Oracle 11.2 DBFS and DBFS Client”

Installation

To successful set up DBFS on CentOS 5 64Bits follow this article on ORACLE-BASE.

Does it works on RAC?

Yes!, DBFS Linux client offers high availability (HA) using RAC technology. The failure of a database instance is detected based on FAN notification. RAC configuration requires an extra service for failover. Once done, dbfs_client transparently redirects file acces to surviving RAC instances on node failures. Any outstanding transaction is replayed to surviving RAC instance, to achieve this follow this steps:

Modify the service

exec DBMS_SERVICE.MODIFY_SERVICE(service_name => ‘service_name’,
aq_ha_notifications => true,
failover_method => ‘BASIC’,
failover_type => ‘SELECT’,
failover_retries => 180,
failover_delay => 1);

Adding the service to the cluster  (clustereware)

Admin Managed

srvctl add service -d OCM11P -s DBFS -r OCM11P1,OCM11P2 -y AUTOMATIC
-e SELECT -m BASIC -w 1 -z 180 -q TRUE

Police Managed

srvctl add service -d OCM11P -s DBFS -g Default -y AUTOMATIC
-e SELECT -m BASIC -w 1 -z 180 -q TRUE

Limitations

• No support for aync IOs.
• Can not be used when database is NOT running.
• No NFS support to exports.
• No file locking support
• No memory-mapped files
• No files flags to support O_DIRECT

Typo and errors in referenced article

The article has errors in /sbin/mount.dbfs script, use the following script:

#!/bin/bash
export ORACLE_HOME=/oracle/app/oracle/product/11.2.0/dbhome_1
echo $@
nohup $ORACLE_HOME/bin/dbfs_client $@ < /sbin/BGH.f &

(The password is stored in a file called /sbin/BGH.f)

The number of nines is not often used by the database or system when measuring high availability for data center environments because it is difficult to extrapolate or calculate. For practical and real life purposes, availability is calculated more as a probability or average downtime in annual basis. You can use Grid Control to monitor SLA for each of the systems involved. For a deep understanding, I can recommend  a very good book wrote by Michael New called: “Oracle Enterprise Manager 10g Grid Control Implementation Guide” (SLA Objectives for Services - Page 439)

Learning Path

Exam Topics are:

Who should read this white paper?

This white paper is intended to be accessible to those who are not relatively new to the Real Applications Cluster performance. Familiarity with advanced Oracle concepts and SQL language is assumed.

About Technical Reviewers

Acknowledgements

This simple word paper is dedicated to all those in pursuit of extraordinary performance. I really appreciate the opinions, ideas and concepts proposed by my friends and colleagues, in fact I used many of them, so thank you very much.

Jorge Teodoro – Sr. Engineer – Technologist
Reinaldo González – Oracle Database Specialist
Fernando Sciaccaluga – Oracle Technology Specialist
Rosa Zahora – Oracle Database Specialist
Marcelo Ochoa – Oracle Developer Specialist – Oracle ACE
Erik Peterson – Oracle Director – Reviewer

Download pdf.

このページについて

基本的にこの書類はノードがどのように情報交換するかを分かりやすく説明してあります。もしパフォーマンスで働いたら、これは義務だと思います。同様にSingle-instanceのインスタンスはディスクI/Oを避けるのには非常に大切です。しかし可能な場合は、既にアクセスされたデータを保存します。RAC構成は似ていますがデータは一つのインスタンスのメモリにあるかもしれません。つまり、RACは別なインスタンスから必須なデータをリクエストしてディスコから読むより良いです。手順なんてかなり分かりやすくて効果的ですけれども、ステップバイステップを利用しなかったら終了に失敗するつもりです。注意深く読んで下さい。リクエストは夫々インターコネクトに介してGlobal Cache Requestへ呼ばれます。別にこの書類はデータベース管理者に必要なすべてのツールを支援しあげますからReal applicationsがGlobal Cacheで待機させます。

誰がこの白い文書を読むべきでしょうか。

この白い文書の本音はReal Applications Clusterのパフォーマンスで初心者じゃない方にアクセスするように意図させました。親しみで高度なORACLEの概念とSQL言語を仮定させます。

Download

Japanese translation by: Bruno Valerio

11g Release 2 has introduced a new option to administer the cluster called “Police-based Management”, traditional physical method is still available.

This is an incredible new feature that needs to be researched, clearly marks the new direction that Oracle wants to give us, BTW, it’s a  smart direction/possibility  where the efforts will be concentrated in the business rules/needs and not in the basic/traditional cluster administration.

Policy-Based Management

• Enables dynamic capacity assignment when needed to provide server capacity in accordance with the priorities you set with policies
• Enables allocation of resources by importance, so that applications obtain the required minimum resources, whenever possible, and so that lower priority applications do not take resources from more important applications
• Ensures isolation where necessary, so that you can provide dedicated servers in a cluster for applications and databases

Server Pools

Server Pools enables you to dynamically assign capacity to meet your policies and priorities, as well as to allocate physical resources by importance. Your critical applications will always have a minimum of resources.

All servers that join the grid are automatically assigned to the “Free Pool” initially. This Free Pool is a server pool that is automatically created during the installation. From here, servers can be assigned to other server pools. Another pool, the Generic Pool is also automatically generated, and all servers will be part of this pool too, in order to ensure compatibility with prior releases. Any pre-11gR2 database will need to run in this generic server pool.

To finish, in my personal opinion  it’s key to understand Server Pools,  database administrator needs to know how the cluster will meet the client’s requirements using  the clusterware available options correctly.

If you want to try clustwerare commands check this link:  Server Pools Experiments in RAC 11g R2

Source

Oracle® Clusterware Administration and Deployment Guide
11g Release 2 (11.2)
Part Number E10717-11

Thanks for your interest.  I wrote a White Paper that actually is  under revision.  It will be distributed the first month exclusively  from http://oracleracsig.org

Bookmark this page and come back after Jun 21, 2010.

Regards,

J

Note: 1z0-050 is an upgrade exam. Certification can only be achieved with this exam if the candidate already holds the 10g DBA OCP.

Basic Knowledge

from Oracle documentation:

“The DBA can assess the I/O capability of the database’s storage system by using the PL/SQL function DBMS_RESOURCE_MANAGER.CALIBRATE_IO(). This routine issues a very I/O intensive read-only workload to the database’s files to assess the maximum IOPS (I/O requests per second) and MBPS (megabytes of I/O per second) that can be sustained. This data can be reexamined at any time using the DBA table, DBA_RSRC_IO_CALIBRATE, that stores I/O calibration results. The calibration should be performed when the database is idle and during off-peak hours to minimize the heavy I/O workload from interfering with the regular workload and vice versa. I/O calibration can be used to understand the performance of the storage subsystem and figure out whether I/O performance problems stem from the database or the storage subsystem. Unlike various external I/O calibration tools, this tool uses the Oracle code stack and issues I/O randomly rather than sequentially. The results, therefore, much more closely match the actual database performance”

How it works ?

Basically calling  to the CALIBRATE_IO procedure in the DBMS_RESOURCE_MANAGER package. It drives some I/O to the database files to test the throughput.

Why use this and not other vendor tool?

Because this tool uses Oracle database software libraries to drive the I/O; not some generic I/O generator. This capability is very important, remember, you are testing the I/O for an Oracle database..

Calibrating (on 11g R2)

First of all, you need to have asynch I/O capability otherwise this will fail.

Step 1

SQL> show parameters options
filesystemio_options string ASYNCH
filesystemio_options = asynch
SQL> show parameters disk
disk_asynch_io = true
filesystemio_options = asynch

Step 2

Before proceed  you will need to eval if your tablespaces support ASYNC_IO.

SELECT name, asynch_io
FROM v$datafile f,v$iostat_file i
WHERE f.file# = i.file_no
AND filetype_name = ‘Data File’
/

SQL> /
/oracle/app/oracle/oradata/R2/system01.dbf     ASYNC_ON
/oracle/app/oracle/oradata/R2/sysaux01.dbf     ASYNC_ON
/oracle/app/oracle/oradata/R2/undotbs01.dbf  ASYNC_ON
/oracle/app/oracle/oradata/R2/users01.dbf       ASYNC_ON
/oracle/app/oracle/oradata/R2/timesten_01.dbf         ASYNC_ON
/oracle/app/oracle/oradata/R2/goldengate.dbf        ASYNC_ON
/oracle/app/oracle/oradata/R2/FLOW_1044720297807647.dbf  ASYNC_ON
/oracle/app/oracle/oradata/R2/FLOW_1259909767198597.dbf   ASYNC_ON

Even if the above query returns ASYNC_ON and you may not be able to run the calibrate process probably you will need to check kernel settings to verify how many async slots have the system configured and how many are in use, to do this I did:

$ cat /proc/sys/fs/aio-max-nr    ->  To find out the maximum available async I/O slots
65536

$ cat /proc/sys/fs/aio-nr  -> To find out how many are being used
32768

Step 3

Now you are ready to  start the calibration process.  CALIBRATE_IO accepts two parameters:

• NUM_PHYSICAL_DISKS – is the number of physical disks you have. Remember: this is the number for disks, not LUNs.
• MAX_LATENCY – is the maximum latency for the disk access (in milliseconds)

vi calibrate_io.sql, add the following:

– dbms_resource_manager.calibrate_io(
– num_physical_disks IN PLS_INTEGER DEFAULT 1,
– max_latency IN PLS_INTEGER DEFAULT 20,
– max_iops OUT PLS_INTEGER,
– max_mbps OUT PLS_INTEGER,
– actual_latency OUT PLS_INTEGER);

SET SERVEROUTPUT ON
DECLARE
lat INTEGER;
iops INTEGER;
mbps INTEGER;
BEGIN
– DBMS_RESOURCE_MANAGER.CALIBRATE_IO (, , iops, mbps, lat);
DBMS_RESOURCE_MANAGER.CALIBRATE_IO (1, 10, iops, mbps, lat);  – (1 Disk, 20 ms max latency)
DBMS_OUTPUT.PUT_LINE (‘max_iops = ‘ || iops);
DBMS_OUTPUT.PUT_LINE (‘latency = ‘ || lat);
dbms_output.put_line(‘max_mbps = ‘ || mbps);
end;
/

Step 4

Depending of your hardware the process could take 3 minutes or more to complete, while running you can query a fixed view:

select * from v$io_calibration_status

When calibration is complete  it produces three OUT parameters, they are:

• MAX_IOPS – the maximum number of I/O Operations per second
• MAX_MBPS – the maximum Megabytes per second
• ACTUAL_LATENCY – the actual latency observed during the test

SQL> @calibrate_io

max_iops = 125
latency = 7
max_mbps = 44

PL/SQL procedure successfully completed.

Results were obtained running the calibration package on a personal laptop with the following characteristics:

• Hardware: LAPTOP HP 550 4GB 160 GB internal Disk, Core 2 Duo
• Software: Linux CentOS 64 Bits, 4 GB RAM

Of course your corporate hardware should be more fast.