ASM Cluster File System (ACFS)

 Prior to Oracle 11.2, ASM was used to store database related files, but was not suitable to act as a general purpose file system to store Oracle binaries, core dump files, parameter or trade files. With the introduction of grid infrastructure in Oracle 11.2, ACFS also got introduced as part of grid. ACFS is a POSIX compliant general purpose cluster file system built on top of the ASM Dynamic Volume Manager (ADVM). ADVM provides so called volumes on which the ACFS can be created. An ADVM volume is created as part of an ASM disk group. Volumes are not partitioned which is equivalent to using LVM in a linux system. 

In addition to using ACFS for shared Oracle Homes, it is used to store application data that previously could not be stored in ASM. For instance, external tables refer to directory files to load data from the file system to the database. If this directory file exists on the file system on the first node, a user trying to access the external table from the third node will not be able to do it. If we move this file to a directory on the ACFS mount, this problem is solved as the ACFS mounts are shared across all nodes.

Creating an ACFS using ACMCA

First create a diskgroup as described in the blog below.

Adding a new Shared disk group in RAC in VM Workstation 10

The diskgroup can then be used to store an ADVM volume. 

In the Volumes tab, click on the CREATE button.

In the new screen that comes, we specify a name for the ADVM (DGVOL1) and place the diskgroup we created on this volume. Unlike a classical LVM on a linux file system, we do not have to partition the volume It can be dynamically resized. You can optionally specify a stripe width and also the number of volumes in the stripe set. By default these default to 128K and 4. On clicking OK, ASCMA creates the volume and displays it on the overview screen. This volume needs an ACFS file system.

Click on the ASM cluster file system tab. Here the user has two options. Either specify the ACFS home as a shared home for the Oracle RDBMS installation for which 4.6 GB is required or create a general purpose file system. When creating a general purpose file system, you have to specify a mount point. The ACFS will be mounted on the cluster node. Here you have the option of registering the mount point. If you register the mount point, the file system is mounted automatically when the grid infrastructure starts. If you do not register the mount point it can always be done later.

Creating and mounting an ACFS using the command line

sqlplus / as sysasm
SQL> CREATE DISKGROUP DG2 EXTERNAL REDUNDANCY DISK 'ORCL:DG2';

Diskgroup created. 

Using the ALTER DISKGROUP cmd you can set the attributes like compatible.asm and compatible.advm based on the version of oracle.

Mount the diskgroup on all the nodes with the following command

$ srvctl start diskgroup –g DG2

Once the diskgroup is created, you can create an ADVM volume and add it to the diskgroup created with the following command.

SQL> Alter diskgroup DG2 add volume VOL2 size 1G mirror strip _width 128k stripe_columns 4;

You can see the details of the volume created from the v$asm_volume view. This view will give the details of the volume device created. Lets say it is /dev/asm/VOL2-777 and the volumne_name is ACFSDGVOL2

 SQL} SELECT volume_name, state, volume_device FROM V$ASM_VOLUME 
     WHERE volume_name ='ACFSDGVOL2';

volume_name	state	volume_device
ACFSDGVOL2	ENABLED	/dev/asm/VOL2-777

You can also get the volume device information using asmcmd

ASMCMD> volinfo -a
Diskgroup Name: DATA1
         Volume Name: ACFSDGVOL2
         Volume Device: /dev/asm/VOL2-777
         State: ENABLED
         Size (MB): 5120
         Resize Unit (MB): 32
         Redundancy: UNPROT
         Stripe Columns: 4
         Stripe Width (K): 128
         Usage:
         Mountpath:

The next step is to create a filesystem on the volume just created.

$/sbin/mkfs -t acfs /dev/asm/VOL2-777

We then create a mount point to mount the file system.

$mkdir $ORACLE_BASE/acfsmounts
$mkdir $ORACLE_BASE/acfsmounts/dg2_vol2

Login as root user to mount the filesystem created on the mount point.

$mount –t acfs  /dev/asm/VOL2-777  /u01/app/oracle/acfsmounts/dg2_vol2

You can register the acfs with the mount registry with the following command.

$ /sbin/acfsutil  registry  –a  /dev/asm/VOL2-777   /u01/app/oracle/acfsmounts/dg2_vol2

You can view the details of the filesystem created with the following command.

$ /sbin/acfsutil info fs

This filesystem can then be used like any normal linux filesystem. df -k will give the details of the filesystem and the mountpoint. Any directories or files created in the filesystem can be accessed on all other nodes of this filesystem.

Once the filesystem is created and mounted, ensure that the permissions are set to allow access to the filesystem for the appropriate users.

# chown -R oracle:dba /u01/app/oracle/acfsmounts/dg2_vol2

You can create a test file and check if it is accessible on the other node. For instance, on node 1, create a file as follows

# echo “Testing ACFS file system” > /u01/app/oracle/acfsmounts/dg2_vol2/acfsfile.txt

You can access this file on Node 2 as follows

# cat /u01/app/oracle/acfsmounts/dg2_vol2/acfsfile.txt.

The output will be

# Testing ACFS file system

References

Pro Oracle 11g RAC on Linux : Steve Shaw and Martin Bach

God's Word for the day

The wisdom of the humble lifts their head high

  and seats them among the great

Do not praise individuals for their good looks

  or loathe anyone for appearances alone

The bee is small among flying creatures

  but what it produces is the best of sweet things

Do not boast about wearing fine clothes

  and do not exalt yourself when you are honored.

For the works of the Lord are wonderful

  and his works are concealed from humankind.

Sirach 11:1-4