Home > Flash Storage, Oracle 11g > Moving 8TB database to Flash Storage

Moving 8TB database to Flash Storage

In recent years we all get accustomed to all kinds of virtualization – VM Ware, Solaris Zones, Solaris Domains, etc… While mechanics may be different, one common trait remains – try and pile up more applications and databases into common set of resources.

One flavor of virtualization is Storage Arrays. Multiple databases and applications get their slices of big Storage Array in the form of LUN. As it happens when big companies in pursuit of efficiency turn every process into a conveyer, the storage provisioning planning is frequently reduced to LUN sizes. In the end you end up with mysterious storage slowdowns when one or more of “other” apps quietly rump up IO utilization to a point where Storage Array cache becomes saturated. Because you have no visibility into what else is running on the same storage – you are left just wondering. That’s the downside of “virtualize everything” trend.

I will have another post about practical ways to detect present and past storage slowdowns.

Here I wanted to share a remarkable IO improvement we got after moving to a Flash storage array.

The story started with several databases running on different RAC grids  experiencing periodic slowdowns due to increased IO latency. Normally with an EMC Array on normal rotational disks the IO latency is in single digits 5-10 ms. During slowdowns the latency went up to 50-100 ms and resulted in database active sessions going from normal 5-7 active sessions to 50-70.

Long story short, management decided to move one critical database off shared EMC array to HP 3PAR flash storage array. Thanks to ASM the migration process turned out to be simple and straightforward. In fact it was so simple to migrate 8TB database that it was almost disappointing.

After our SA had discovered and labeled provisioned SSD LUNs with oracleasm add disk command, to actually migrate 8TB all I needed to do was to run this command:

alter diskgroup DG_IVASCPDCLU1_ORA1 
drop disk 'ASM05','ASM06','ASM07','ASM08','ASM09','ASM10',
          'ASM11','ASM12','ASM13','ASM14','ASM15','ASM16',
    'ASM17','ASM18','ASM19','ASM20','ASM21','ASM22',
    'ASM23','ASM24','ASM25','ASM26','ASM27','ASM28',
                    'ASM31','ASM32','ASM33','ASM34',
    'ASM35','ASM36','ASM37','ASM38','ASM39','ASM40',
    'ASM41','ASM42','ASM43','ASM44','ASM45','ASM46',
    'ASM47','ASM48'
 add disk 'ORCL:SSDASM09','ORCL:SSDASM10',
          'ORCL:SSDASM11','ORCL:SSDASM12','ORCL:SSDASM13','ORCL:SSDASM14','ORCL:SSDASM15','ORCL:SSDASM16',
    'ORCL:SSDASM17','ORCL:SSDASM18','ORCL:SSDASM19','ORCL:SSDASM20','ORCL:SSDASM21','ORCL:SSDASM22',
    'ORCL:SSDASM23','ORCL:SSDASM24','ORCL:SSDASM25','ORCL:SSDASM26','ORCL:SSDASM27','ORCL:SSDASM28',
    'ORCL:SSDASM29','ORCL:SSDASM30','ORCL:SSDASM31','ORCL:SSDASM32','ORCL:SSDASM33','ORCL:SSDASM34',
    'ORCL:SSDASM35','ORCL:SSDASM36','ORCL:SSDASM37','ORCL:SSDASM38','ORCL:SSDASM39','ORCL:SSDASM40',
    'ORCL:SSDASM41','ORCL:SSDASM42','ORCL:SSDASM43','ORCL:SSDASM44','ORCL:SSDASM45','ORCL:SSDASM46',
    'ORCL:SSDASM47','ORCL:SSDASM48','ORCL:SSDASM49','ORCL:SSDASM50';

At this point wonderful ASM automatic rebalance feature kicked in and all I had to do was just sit back relax and watch rebalance moving data from EMC luns to Flash luns with

— monitor rebalance
alter diskgroup DG_IVASCPDCLU1_ORA1 rebalance power 4;
select * from V$ASM_OPERATION;
or
asmcmd> lsop

Its online operation and works for any kind of asm groups – data, FRA, redo,  voting group, acfs etc…

The db performance difference was amazing.

Here is how latency behaved while ASM group rebalance was going on (moving from EMC to Flash) :

Latency EMC to 3PAR a

Once on SSD the IO latency dropped to a fraction of a millisecond.

Here is Database active sessions chart on a Typical Monday on EMC:

Top Activity - EMC - Monday 20151116

And here is Database active sessions chart first Monday on Flash:

Top Activity - 3PAR - Monday 20151123

Pretty impressive.

 

Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: