EMC's VPLEX & Planned Takeover of DataDirect Networks

Talk to the average Storage Engineer who manages the growth of your datacenter’s modular system about Petaflops, Exabytes, Petabytes of Archives or 1TB of sustained bandwidth and you’ll probably find them scratching their heads in disbelief. This is the reality that does exist in the world of super computing and what is sometimes referred to as Extreme Storage. While some Storage Managers would feel they are suffering with their exponential data growth and decreasing budgets, their problems can’t be classified as ‘Extreme’ unless they’re dealing with ExaBytes (1018 bytes) of storage with trillions of data transactions per second, trillions of files and a data transfer rate from storage to application that exceeds a TB per second. Couple that with the conundrum that it’s for relatively few users and requires the data to be secure, both for short-term and long-term retention and then you have a real case for Extreme Storage……. well at least for now.

Such figures though are not the concern of the average datacenter manager or storage vendors with architectures that are catered and designed for IOPS centric database driven applications so much so that even SNIA has yet to give Extreme Storage the relevance of a definition in their Storage Dictionary. Not EMC though, where if my sources are correct, they deem Extreme Storage not only a key to their own future but also that of the Storage industry’s such that they are already concocting an audacious takeover plan of the company DataDirect Networks.

Before I embark upon my controversial claim, let’s rewind back a few weeks to EMC World, Boston where most of the buzz centred on the launch of the new VPLEX. A nifty idea that would take cloud computing enthusiasts to an ever approaching reality by creating heterogeneous pools of storage that can be accessed and shared over a distance. Couple that with VMware integration and you have the ability to VMotion your applications across datacenters that are miles apart - a great idea and one that strikes a double whammy at both Storage vendors HDS/IBM/HP/NetApp and the virtualisation ‘catch-up guys’ MS Hyper-V and Red Hat. As IT Directors yearn for a virtual infrastructure for their applications that goes beyond the physical limitations of the datacenter, EMC’s trump card of setting up a centrally managed pool of virtual resources spread across numerous datacenters via VPLEX is nothing short of a ‘virtual’ revolution. With Site Recovery Manager, VMware already had the edge over their competitors by in essence providing an extended version of their ‘high availability’ concept that could span across data centers. With VPLEX the VMotion concept of moving a virtual server across physical platforms ‘on the fly’ can now also be extended across datacenters. Moreover while EMC have currently failed to corner the market of virtualisation of heterogeneous storage dominated by HDS and IBM with their product Invista, the launch of the VPLEX now takes that battle head on with the added value of cross-site virtualisation. So how then does this link to my bold prediction that Extreme Storage is next on EMC’s radar with more significantly a proposed takeover of the company DDN?

The VPLEX model is poised to have four versions, two of which are already available namely VPLEX Local and VPLEX Metro with VPLEX Geo and VPLEX Global to follow suit. VPLEX Local is the straightforward virtualisation of heterogeneous storage systems behind one management pane within your datacenter, a solution that has successfully been offered by HDS for several years. VPLEX Metro though allows the concept to stretch up to 100km, hence enabling the virtualisaion of storage spanning datacenters across cities. Based on a combination of hardware and software which is placed between the traditional FC attached storage systems and the servers, the VPLEX rack virtualizes the heterogeneous mix of disk arrays into what EMC term ‘a federated pool of virtual storage’. As for the hardware itself, it contains a management server, FC switches, Ethernet switches, the standard redundant power supplies and the VPLEX engines. Within each engine rests a pair of quad core Intel CPUs and directors which each contain 32 FC ports with 8Gbps bandwidth. With an active-active cluster spread over one to four VPLEX engines the requirements to seamlessly VMotion applications across a 100km distance is more than easily met, hence being coined VPLEX Metro. The question that now stands is for the proposed VPLEX Geo and VPLEX Global i.e. would such hardware and performance stats add up for say data that needs to be VMotioned across continents as the name suggests? Indeed such distances and endeavours would not be the requirement of EMC’s regular customer base of industries that demand financial transaction processing but rather those that are facing a content nightmare and need the expertise and performance figures that are associated with Extreme Storage.

When you’re talking Extreme Storage you’re talking DDN i.e. DataDirect Networks. While relatively unknown, DDN still possess an impressive resume of HPC clients from NASA, Lawrence Livermore Laboratories to movie special effects users like Pacific Title & Art Studio. Thus as far as being a company that can act as a platform from which EMC can build out its ‘global’ and ‘geo’ cloud storage offerings, DDN already have credible references to do so quite easily.

Furthermore a potential acquisition of DDN will allow EMC to penetrate a HPC customer base that they’re currently unfamiliar to. Fields ranging from High Energy Physics companies such as Fermilab, Nuclear research organisations such as CERN, particle physics research companies such as DESY to National Security and Intelligence are all potential clients that EMC could take on with a new Extreme Storage Platform that incorporates VPLEX and deals with large data that is locally or globally distributed with long-term retention. It would clearly give EMC a major distinction from its current major competitors.

Ironically though it’s one of EMC’s current competitors, HP that have already made moves into Extreme Storage with their Ibrox based HP 9100 Extreme. The HP 9100 was marketed as an Extreme Storage system and was shamelessly targeting Web companies and their like who required multipetabytes of data storage. HP’s aim was to profit from an emerging market of heavy users such as ever growing and popular social networks with their online subscriber information and video content as well as users of video surveillance systems and research organizations. While this was a brave attempt even HP had to concede to DDN’s supremacy and expertise in the field when they only this week agreed an OEM relationship for DataDirect Networks (DDN)’s S2A9900 disk array to be bundled with the Lustre File System resold by the SCI group within HP. Indeed HP are now like every large HPC OEM vendor out there – reselling DDN. With partnerships already with IBM, Dell and SGI, the one big name missing from the list is EMC. Now with the VPLEX Global and Geo offering soon to be unveiled, a relationship with DDN whether it be an acquisition or OEM seems inevitable.

In fact DDN and EMC are certainly no strangers to each other, when last year the former launched a direct onslaught on EMC's Atmos cloud storage product with their Web-Optimised Scaler (WOS). Designed for geographically dispersed global storage clouds, the WOS is a geo-cluster of quasi-filers that store objects with an API-access to a global namespace. Boasting scalability that is currently growing beyond 200 billion files of storage and 1 million file reads per second for objects, such stats are effortlessly achieved through the simultaneous access of numerous WOS boxes. Hence not only flooring EMC's Atmos in terms of transaction rates and file retrieval rates, being a file-based product the WOS also hits the EMC NAS jugular namely the Celerra. As for how the WOS works on a global scale; the storing of objects, which are files or groups of files are each given a unique object number which identifies the datacenter containing the WOS system that stores the object and the object itself. Datacenters are linked via WOS nodes which form the WOS cloud while the WOS API is used to access servers to read or write objects to the WOS cloud. A straightforward concept but the question now is how much of this explanation will replace the word DDN with EMC and WOS with VPLEX Global come the launch of the latest EMC masterplan? Put this in the wider context of the upcoming VPLEX Geo and Global, and I have little doubt that EMC Execs (renowned for preferring to spend outrageously than OEM a potential competitor) are furiously sharpening their pencils, carefully concocting a takeover of the still relatively small yet growing company that is DDN.

After the rapidly swift takeover of Data Domain last year, nothing surprises me anymore with regards to the financial clout of EMC. So while a takeover of DDN will not only bring about the removal of the competitive edge that DDN currently poses and enable EMC’s vision of ‘VPLEXing’ across the globe to become an instant reality, the benefits of such a deal would bring even more so to EMC’s constantly growing portfolio. In the words of Bob Dylan,“Times are a changing” and the digital content explosion brought about by the rapid growth of online, nearline and backup data pools has left the traditional storage systems designed by EMC and their like defunct and inadequate to compete in such a vast growing market. Like a crumbling empire the domination of transactional data that factored so heavily in the design of storage systems has ended with an unscrupulous coup de etat of unstructured data requiring extreme performance, scalability and density becoming the mainstream. EMC have clocked on to this and are pushing their future be involved in this direction. Should a DDN deal go through, EMC will not only have advanced themselves into a new customer base but would also bring in vast technical expertise ranging from high-speed FPGA parity calculation accelerators instead of RAID systems, high speed Infiniband interconnects etc. that can only enhance their current Enterprise and Modular range. As for EMC’s direct competitors such as HP, IBM, HDS etc. who will they have to turn to for an OEM deal or expertise should they also decide to enter the fast growing market trend towards Extreme Storage……perhaps EMC themselves if these predicted developments are to bear fruit.

The Unified Storage Battlefield Could Decide the Future of Storage

In the past week HDS finally revealed their response to the VMware-Cisco-EMC alliance with the launch of a unified computing platform including integrated storage, server, and networking technology. With the aid of Microsoft, HDS have stated that their centralized storage, server and networking platform will be launched early next year. In the tradition of my enemy’s enemy is my friend, HDS have also signed an OEM deal with Microsoft under which Microsoft System Center Operations Manager, System Center Virtual Machine Manager and Windows Server 2008 R2 will be tightly integrated with Hyper-V. Added to this is HDS Dynamic Provisioning and the HDS Storage Cluster for Microsoft Hyper-V. Moreover despite the secrecy, the networking brains behind the platform are most probably Brocade, the grandfathers of SAN who also now have a sound grip on IP networking since their acquisition of Foundry back in 2008.

Well, it’s no surprise that with the current turmoil brought upon by the disbandment of the SUN OEM deal, HDS are desperate to announce a new product despite it being more than six months away. But the trend towards Unified Storage is one that is being followed by many in an attempt to adhere to the economic climate and the rapid drive towards consolidation. While at one point it was NetApp’s domain of which no one seemed to be interested in, the Unified Storage demand has grown considerably with customers seeing the mass of potential and savings that come with running and managing files and applications from a single device. By consolidating file-based and block-based access in a single platform and hence supporting FC, iSCSI, and NAS, customers immediately reap the benefits of reduced hardware requirements, lower capital expenditures and simplified single pane management. So now the war of vendors has entered a new battlefield in which nearly all are making a bid to usurp the lion’s share of the spoils. But like in every battle there will ultimately be casualties…

Cisco, the IP kings have bravely entered the arena and are pushing forward plans with their combination of networking, blade servers and storage in a single architecture i.e the Unified Computing System (UCS) platform. Whether they can convince customers that an IP switch company can build servers remains to be seen, but Cisco already proved doubters wrong when they successfully entered the SAN market by drawing on the IP niche that they had established in just about every data center in the world.

HP's acquisition of 3Com on the other hand was instigated to provide the networking brains for their ‘converged computing’ model that binds server, storage, and networking resources. How the powerhouse of HP will fair is not at as difficult to predict given the success of their blade systems and credence amongst customers as a server platform provider. But are they entering the arena too late and how will this fair with their OEM relationship with HDS?

Within this battlefield of generals, there are also some charlatans who have cheekily tried to gain some market share just by coining the term ‘unified storage’. IBM and NEC for example, have brought out backup and recovery systems within a single architecture that lack any NAS support, yet still coin the term ‘unified storage’. Such pretenders may suffer an early death especially when smaller companies such as BlueArc go the whole nine yards with their Titan systems that not only support SAN and NAS but can also utilize WAN via Riverbed's Steelhead networking solution.

Then there’s the SUN 7000 series from Oracle’s Sun Microsystems. A great bargain for the amount of functionality that it provides from unlimited snapshots, integral data compression, iSCSI thin provisioning, virus scanning, remote replication as well as the expected support for CIFS, NFS, HTTP, FTP and iSCSI. Additionally the 7000 series supports RAID 5, RAID 6 arrays and ZFS Hybrid storage pools which can capitalize on the high performance of Flash memory devices and DRAM memory. Yet despite how great the 7000 is, it’s coming from a camp that has been mortally wounded with the messy Oracle takeover and the bureaucracy that surrounds it, to which customers are now suffering the effects of. Will customers purchase a great product that will immerse it into an eon of political wrangling when they need and rely on quick and timely support?

It’s evident that HDS or anyone else for that matter, which coins the term ‘Unified Storage’, is going to have a tough time dealing with EMC. The marketing machine which currently knows no bounds, made an unashamed onslaught on the small business market cornered by NetApp when they launched the Celerra. While in essence it was just a Clariion with a NAS gateway, it fully supported SAN and NAS as well as NFS 2, 3 and 4, and CIFS file sharing. Furthermore EMC’s entry into the market seems to be with a strategic plan that seems to span the company as a whole, which is minimizing its different hardware platforms.

When EMC released the V-Max platform, one of the most notable things was its usage of hardware components that were already available on other EMC hardware platforms. From the Clariion-esque disk drives, flash drives, DAE’s, LCC’s, Intel x64 CPU’s, fans to power supplies, the Celerra, like the V-Max is also made in the same mould. With the Clariion, CDL, EDL and Celerra platforms all sharing similar hardware components, it’s only a matter of time before the anomalous architecture of the archive platform, Centera is either changed to fit the mould or replaced completely in favour of a unified platform that seamlessly integrates with the Celerra or Clariion.

As Cisco had done before them when they added SAN to their IP portfolio and what NetApp have done to some extent with ONTAP, EMC’s common hardware direction could eventually lead to underlying software being the only thing which distinguishes different EMC platforms.

So while currently unified storage limits the level of control in file-based versus block-based I/O and hence does give lesser performance than its dedicated block-based counterpart, a strategic approach that takes a long term look at the term ‘unified’ could change the face of high end storage systems in the future. As storage systems move further towards consolidation, it is indeed the winner in the battlefield of unified storage that that will eventually draw others to a new beginning and approach and ultimately the end of the current trend of 7 feet tall high end enterprise systems that have housed data centers for so many years. A self tiering SATA / SSD Unified Platform without FC disks?….Let’s watch this space.

Data Domain's CPU Centric Deduplication Genius is no Dupe

Last year EMC’s somewhat controversial acquisition of Data Domain right under the noses of NetApp raised several eyebrows to say the least. Considering the reported amount of $2.1 billion and their already deduplication packed portfolio which consisted of the source based Avamar, the file-level deduplication/compression of its Celerra filer and their Quantum dedupe integrated VTLs, some heads were left scratching as to what actually was the big deal with the target based deduplication solution of Data Domain. Almost a year on and with Data Domain’s DD880 being adopted by an ever growing customer base, the heads have stopped scratching and are paying close attention as to what is probably the most significant advancement in backup technology of the last decade.

With deduplication currently being all the rage, with possibly only ‘Cloud Computing’ overshadowing it, the benefits of deduplication are becoming an exigency for backup and storage architects. With most backup software producing copious amounts of duplicate data stored in multiple locations, deduplication offers the ability to eliminate those redundancies and hence use less storage, less bandwidth for backups and hence shrink backup windows. With source based and file level based deduplication offerings, it is Data Domain’s target based solution i.e. the big black box that is clearly taking the lead and producing the big percentages in terms of data reduction. So what exactly is so amazing about the Data Domain solution, when upon initial glance at for example the DD880 model, all one can see is just a big black box? Even installing one of the Data Domain boxes hardly requires much brainpower apart from the assignment of an IP address and a bit of cabling. And as for the GUI, one could easily forget about it as the point of the ‘big black box’ is that you just leave it there to do its thing and sure enough it does its thing.

And while the big black box sits there in your data center the figures start to jump out at you where an average backup environment can see a reduction of up to 20 times. For example a typical environment with a first full backup of 1TB with only 250GB of physical data will immediately see a quadrupled reduction. If such an environment was to take weekly backups with a logical growth rate of 1.4TB per week but with only a physical growth of 58GB per week, the approximate reduction could go up to more than 20 times within four months:

Reduction =
First Full + (Cumulative Logical Growth x Number of weeks) / Physical Full + (Cumulative Physical Growth x Number of weeks)

e.g. After 25 weeks
Reduction = 1TB + (1.4TB x 25) / 0.250TB + (0.058TB x 25)
= 35TB / 1.7TB
= 21 times less data is backed up

So how does Data Domain come up with such impressive results? Upon closer inspection, despite being considered the ‘latest technology’, Data Domain’s target based deduplication solution has actually been around since 2003, so in other words these guys have been doing this for years. Now in 2010 with the DD880, to term their latest ‘cutting edge’ would be somewhat misleading when a more suitable term would be ‘consistently advancing’. Those consistent advancements have come from the magic of the big black box being based on its CPU-centric architecture and hence not reliant upon adding more disk drives. So whenever Intel unveils a new processor, Data Domain does likewise with its incorporation into their big black box. Consequently the new DD880’s stunning results are the result of its incorporation of a quad-socket quad-core processor system. With such CPU power the DD880 can easily handle aggregate throughput to up to 5.4 TB per hour and single-stream throughput of up to 1.2 TB per hour while supporting up to 71 TB of usable capacity, leaving its competitors in its wake. Having adopted such an architecture, Data Domain have pretty much guaranteed a future of advancing their inline deduplication architecture by taking advantage of every inevitable advance on Intel's CPUs.

Unlike the source based offerings, Data Domain’s Target-based solution is controlled by a storage system rather than a host and thus takes the files or volumes from the disk and simply dumps them onto to the disk-based backup target. The result is a more robust and sounder solution to a high change-rate environment or one with large databases where RPOs can be met a lot easier than with a source-based dedupe solution.

Another conundrum that Data Domain’s solution brings up is the future of tape based backups. The cheap RAID 6 protected 1 TB / 500 GB 7.2k rpm SATA HDD disks used by the DD880 alongside the amount of data reduced via its deduplication also brings into question the whole cost advantage of backing up to tape. If there’s less data to back up and hence fewer disks than tape required, what argument remains for avoiding the more efficient disk to disk back up procedure? An elimination of redundant data with a factor of 20:1 brings the economics of disk backup closer than ever to those of tape backups. Couple that with the extra costs of tape backups often failing, the tricky recovery procedures of tape based backups as well as backup windows which are increasingly scrutinized; this could well be the beginning of the end of the Tape Run guys having to do their regular rounds to the safe.

Furthermore with compatibility already with CIFS, NFS, NDMP and the Symantec OpenStorage, word is already out that development work is being done to integrate closer with EMC’s other juggernauts VMware and Networker. So while deduplication and its many forms saturate the market and bring in major cost savings to backup architectures across the globe, it is Data Domain’s CPU based, target based inline solution which has the most promising foundation and future and currently unsurpassable results. $2.1 billion? Sounds like a bargain.

Consider this Before Jumping on the Cloud

Thinking about jumping on the cloud? True, I’ve had no qualms in showing my skepticism towards the marketing of ‘cloud computing’ and it being the mere repackaging of solutions which have existed for years, but the fact is it still addresses a concept and reality that exists and one which holds numerous benefits. Indeed abandoningan existing on site IT infrastructure for a cloud provider that most often or not can’t offer the same level of security, control or performance is not an easy decision but one which CIOs and IT executives are seriously considering when weighing up the economic benefits. As with any change though, a move towards the cloud necessitates a sound and comprehensive assessment to avoid the trap of a short term benefit turning into a long term nightmare.

Integration Considerations:

Firstly your company should assess its business strategy and whether a cloud computing infrastructure could integrate with it. With all the convenience that cloud offers, the risk of data residing beyond your company’s vicinity still presents a significant security risk. Hence depending on your company, whether that be a startup, small, medium or enterprise an assessment of how much risk you are prepared to take in order to cut costs will initially be the first factor in deciding which data / applications to outsource to the cloud. This is what often leads enterprises to be content with using the cloud for archival while smaller business find it easier to go the whole nine yards. Once that’s decided your design for integration should first evaluate each process and system and determine the number of simultaneous requests that need to be handled. As always availability of information also needs to be paramount, regardless of whether the information resides internally or on the cloud. Thus the key for enterprises is to not think of the cloud as a substitute for their processes, policies and security but rather an extension of their existent architecture.

Choosing the Right Cloud:

If your company needs access to e-mail, e-commerce, gaming applications, etc. with cost reduction being the primary concern over security and availability then the public cloud provides the service to access such applications via a thin device with minimal financial overhead. Here the start up business can thrive but companies with an already substantial customer base and Tier 1 applications such as OLTP should think otherwise. In such an instance a hybrid cloud may be most suitable wherein the Tier1 application data remains in house with the cloud being utilized for backup, archiving and testing bringing in the consequent cost savings. Enterprises on the other hand with strict Sarbanes Oxley and security controls on customer data and service levels may outweigh the risk of information access control over cost and leave the cloud altogether, that is until security in the cloud becomes further enhanced.

Testing the Cloud with your Existent Virtualised Infrastructure:

The balance of scales will always alternate between the reliable and secure nature of data centers with the far cheaper ‘pay as you use’ option of cloud computing. Therefore a way to get a taste of the cloud and assess its suitability to your environment is to utilize the virtualization that is already taking place within your data center. If you are one of the few archaic infrastructures that has not already moved towards the virtualization of applications and infrastructure then you are already missing out on immediate cost benefits which could be existent within your own data center or what the marketing teams tag ‘the internal’ or ‘private’ cloud. With the independence of applications from their physical infrastructure, your in house virtualisation already provides the gateway and flexibility to cloud offerings which are based on external virtual infrastructures. By choosing a non-core application to experiment with such as your e-mail archiving you can quickly set up a test environment and get a feel of the bottlenecks, security, performance and billing procedures of the cloud. Upon sufficient testing and having gained an idea of the metrics you required, you will have a better idea and peace of mind on how best to deploy more mature applications to the cloud.

Choosing the Right Applications:

Choosing the right applications is also a serious precursor as not all applications are best suited for the cloud. If PaaS and IaaS are a consideration then a migration may be a necessity if the cloud may not support your application especially if there are incompatibility issues between the platform software and your application. Another issue is related to applications that require real time response that can’t be put at the risk of potential network latency, congestion or bandwidth clogging. Tight backup windows, graphics intensive applications or applications handling large I/O would need serious consideration and investigation of the SLAs provided by the cloud prior to any migration. On the flip side applications linked to your non core business operations may be ideal for deployment such as a marketing campaign website or a standalone application that doesn't require much interaction with other applications. Again applications which are used for testing purposes can easily be deployed to the cloud with minimal costs and risks to the business especially with many cloud providers offering free trial periods.

To conclude while network latency, data availability, security and application support are all valid concerns for customers thinking of utilizing the cloud, a sound approach and pre-analysis could easily alleviate them, bringing in major cost savings to your business. In other words jumping on the cloud bandwagon without the right assessment might see you falling straight through from the sky; but making sure the cloud you do jump on is substantial enough to support and sustain your company could also be the soundest business investment you’ll ever make.

Some think it's as easy as building bricks:

Are SUN to end Hitachi Data Systems OEM Relationship?

On January 26th just prior to the official announcement of Oracle’s takeover of SUN Microsystems, I confidently predicted in my article “SUN’s Oracle Merger” with regards to SUN’s storage portfolio that “One certainty is that the OEM partnership with HDS’ enterprise arrays will continue.” Perhaps it’s time to eat some humble pie. If current indications are anything to go by, it’s more than likely that the SUN agreement of reselling HDS Enterprise Storage may be coming to an abrupt end.

Oracle clearly have a different business approach to their customers than SUN Microsystems did, and that includes dealing with Hitachi Data Systems. Admittedly I‘ve never been a great fan of SUN’s storage systems, often finding them to be the epitome of a server company that builds storage i.e. a box with lots of disks and sparse in-built intelligence. But with the recent launch of the 7000 series (which still may come under scrutiny by NetApp considering it’s more than coincidental similarities) and the intelligent storage systems built by Larry Ellison’s other plaything Pillar Data Systems, their modular market is now pretty well covered. How Oracle/SUN could plan to cover the gap that will be left from a potential removal of the USPV / USPVM (ST9990V /ST9985V) could lie with the approach shown with the recent Exadata V2. Oracle databases that are directly attached to boxes chocked full of flash drives may well be the answer Oracle/ SUN will be offering to get themselves free from the entanglement of Enterprise storage vendors.

While there seems to be a game plan of some sort in the Oracle/SUN camp, if this supposition were to come true it will have major implications on Hitachi Data Systems and their next steps forward. Personally I’m happy if this will happen as it may at last be the kick up the ‘Back End Director’ that HDS need to finally start marketing and addressing a customer base, certainly within the EMEA region that is still oblivious to its existence. I’ve often shown my frustration at HDS and their inability and lack of drive to push forward their brand and products to consumers who have settled for inferior products from other vendors that were merely marketed better. Resting on the laurels that HP and SUN were rebadging and reselling their Enterprise Systems and doing all the hard work for them, the downside was that HDS’ cross bar architecture storage systems and virtualization technology were firmly placed in thousands of datacenters, unbeknownst to the IT Directors that bought them. Another issue was that unlike the SUN relationship in which only the colour of the doors and the SUN badge changed, HP buy HDS Enterprise Storage Systems and actually change the microcode making them more ‘HP’ than ‘HDS’. So a true untainted HDS USPV could now potentially only be purchased from Hitachi Data Systems themselves. This could be the beginning of a HDS revolution or a slow withering death of sales.

But I’m confident if the leadership at HDS takes the right steps and investment, this could finally be the key to a future market share they have been lacking in. There is no doubting the quality of the HDS Enterprise range from the still reliant cross bar architecture and vIrtualisation through the array USPV systems. Hence maintaining those sales and support deals with existent SUN customers may not be such a great overhaul especially with an updated USPV on the horizon. Where the real challenge lies is drawing customers to the equally good modular AMS and WMS range which are rarely found in Datacenters yet alone virtualized behind their own Enterprise Storage Systems. Also the HNAS range made by BlueArc are also a range to be reckoned with but are hardly making NetApp sales guys break a sweat as potential customers are often unaware of their existence. Plus all the latest initiatives which HDS have taken such as High Availability Manager, IT Operations Analyzer, or the Hitachi Content Archive platform HCP, as excellent as they are, are still not making the waves and marketing noise their credentials deserve.

So in a twist of fate, should the continuation of the SUN OEM relationship fall through, Hitachi Data Systems may be forced into being the marketing machine it up to now has shied away from in order to maintain and advance its presence in the industry. The positive thing is that the products are and always have been good enough – now it’s time for the marketing guys to promote it.

The Best Back Up Solution for VSphere4?

When VMware first introduced VCB as part of the ESX package, it never did seem more than a temporary / complimentary solution for customers who had a small environment of 100 VMs or less. With the launch of VSphere4 and the subsequent introduction of APIs which allowed external applications and scripts to communicate directly to the ESX, it was apparent that VMware was beginning the gradual move to offload the backup solution to the Backup experts. Now having run with VSphere4 for more than six months, it seems a good time to assess and evaluate who has taken advantage and the lead with incorporating all the latest features of ESX4.

To recap; with VMs being encapsulated in a single disk file the principal was that image-level backups instead of traditional file-level allowed backups to be much faster. With VSphere4, VMware introduced improved support for thin provisioning which not only had the potential to reduce the amount of actual storage allocated but also shorten backup windows. The idea was straightforward; by using thin provisioning the system admin is given the ability to over-commit disk space, especially handy as the majority of VMs don’t use all of their allocated disk space. Thus this eradicates the problem with the majority of disk-to-disk backup applications using image-level backups i.e. no more backing up of a complete virtual disk file of a VM when most of it wasn’t actually used.

Another new addition to VSphere4 and it’s vStorage APIs was the feature of CBT (Changed Block Tracking). With CBT the VMkernel could now track the changed blocks of a VM’s virtual disk. Just by querying the information using an API call to the VMkernel, CBT alleviates the burden from the backup applications having to scan or keep track of changed blocks. This results in much quicker incremental backups as the overhead of scanning the whole VM image for changes since the last backup could now be eradicated.

So in looking around for solutions which best incorporated these new features I eventually came across Veeam. Utilising the thin provisioning feature to remove the overhead of no longer seeking out empty disk blocks and the unnecessary backing up of those empty blocks, Veeam also incorporates compression algorithms on the target backup device. Hence Veeam have a solution that not only reduces the amount of space used on the source host datastores but also the target backup storage device.

Furthermore Veeam are currently the only third party API that offer support for CBT, although ESXpress are promising something similar with their upcoming version 4. Veeam have come up with several different modes to utilise the CBT feature namely, SAN mode, Virtual Appliance mode and Network mode. Each mode brings less I/O to each device depending on your setup and thus less resource consumption when performing backups consequently leading to reduced backup windows.

So while Veeam are currently leading the way, the time is certainly ripe for more third party APIs to be developed and incorporated making VM backup nightmares a thing of the past.

Infiniband – Boldly Going Where No Architecture Has Gone Before

Back in 2005 we all knew that Fibre Channel and Ethernet would eventually support transmission rates of 10 Gbit/s and above and now in 2010 that day has pretty much dawned on us. In the excitement of those days what was always a concern was that the host’s I/O bus would need to transmit data at the same rate. But with all the advancements of PCI-E, the nature of all parallel buses is that their transmission rate can only be increased to a limited degree so how was this potential barrier ever going to be solved? The solution being penned around at the time was InfiniBand. Not only did it carry a name that seemed straight out of a Star-Trek episode but it also promised a ‘futuristic’ I/O technology which replaced the PCI bus with a serial network. That was five years ago and bar a few financial services companies that run trading systems I hadn’t really seen any significant implementations or developments of the technology that was marketed with the phrase ‘to Infiniband and beyond’. But two weeks ago that suddenly changed.

Before I delve into the latest development of the architecture that’s bold enough to imply ‘infinity’ within its name one should ascertain as to what exactly justifies the ‘infinite’ nature of Infiniband. As with most architectures the devices in Infiniband communicate by means of messages. That communication is transmitted in full duplex via an InfiniBand switch which forwards the data packets to the receiver. Also like Fibre Channel, InfiniBand uses 8b/10b encoding enabling it to package together four or twelve links to produce a high transmission rate in both directions. Using Host Channel Adapters (HCAs) and Target Channel Adapters (TCAs) as the end points, the HCAs act as the bridge between the InfiniBand network and the system bus while the TCAs make the connection between InfiniBand networks and the peripheral devices that are connected via SCSI, Fibre Channel or Ethernet. In other words for SAN and NAS folk that basically means HCAs are the equivalent to PCI bridge chips while the TCAs are in the same vein as HBAs or NICs.













Additionally HCAs carry the ability to be used for not just interprocessor networks, attaching I/O subsystems, but also for multi-protocol switches such as Gbit Ethernet switches. Herein lies the promise of a sound future with Infiniband due to its independence from any particular technology. Indeed the standard is not just limited to the interprocessor network segment, with error handling, routing, prioritizing and the ability to break up messages into packets and reassemble them. Even messages can be a read or write operation, a channel send or receive message, a multicast transmission or even a reversable transaction-based operation. With RMDA existent between the HCA and TCA, rapid transfer rates are also easily produced as the HCA and TCA each allow permission to read or write to the memory of the other. Once that permission is granted write or read location is instantly provided thus enabling the superior performance boost. With such processes, control of information and it’s route occurring at the buslevel, it’s not surprising that the InfiniBand Trade Association view the bus itself as a switch. Add to the equation that InfiniBand uses Internet Protocol Version 6, you’re faced with an almost ‘infinite’ amount of device expansion as well as potential throughput.

So fast forwarding to the end of January 2010 and I finally read headlines such as ‘Voltaire’s Grid Director 4036E delivering 2.72 terabits per second’. At last the promise of Infiniband is beginning to be fulfilled as a product featuring 34 40 Gb/s InfiniBand ports i.e. a collective 2.72 terabits per second proved this was no longer Star Trek talk. With an integrated Ethernet gateway which bridges traffic between Ethernet-based networks via an InfiniBand switch, the Voltaire 4036E is one of many new developments we will soon witness utilsing Infinband to provide unsurpassed performance. With high performance requirements for ERP applications, virtualization and ever growing data warehouses always increasing, converging Fibre Channel and Ethernet with InfiniBand networks into a unified fabric now seems the obvious step forward in terms of scalability. Couple that with the cost savings on switches, Network interface cards, power /cooling, cables and cabinet space and you have a converged network which incorporates an already existent Ethernet infrastructure.











InfiniBand suppliers such as Mellanox and Voltaire may have their work cut for them with regards to marketing their technology in the midst of an emerging 10gigE evolution but by embracing it they may just ensure that Infinband does indeed last the distance of ‘infinity and beyond’.

Sun’s Oracle Merger – A marriage made in heaven or a deal with the devil?

With only the ‘you may now kiss the bride’ custom to follow, the ORACLE/SUN marriage (or dare I say SUN/ORACLE) is now finally complete. After months of legal wrangling which has caused nothing but embarrassment and dwindled SUN’s stature within the market sphere, reports also came out that half of Sun's 27,000 staff will be made redundant. Thus initial indications are clear that Oracle, known for its past agnosticism to open source has an eye for the merger being based on maximizing profit. In the meantime Sun’s competitors are probably smiling wryly as the delay of the merger played into their immediate interests but what threats and challenges does this partnership now pose to the once great open source vendor which did so much for developing the tech and e-commerce industry.

One thing which Oracle will most probably do is address and remediate the main cause for Sun’s tragic decline prior to the days when talk of ‘takeovers’ and ‘falling stock shares’ became the norm. In my humble opinion that was linked to Sun failing to consolidate on its strengths by audaciously venturing into unknown avenues only to find that it couldn’t compete with the existent competition. By spreading itself too thinly the ambitious nature of the company soon led it into labyrinths it couldn’t escape from. One such adventure was its acquisition of StorageTek.

StorageTek, known for their solid modular storage arrays and robust tape libraries had a decent reputation of their own prior to Sun’s takeover. Data Center managers, IT Directors and their like knew they had solid products when they purchased the brand StorageTek but in a miscalculated maneuver, Sun decided to rename all their Storage products with the Sun Microsystems brand. Suddenly Sun’s Sales team had to sell what for the average IT Director was seemingly a new and unproven product based on an unneeded name change. Additionally when these storage products took on the same name as Sun’s other storage company, StorEdge further confusion came into the mix. Couple that with an emerging market for disk based backups, purchasing a company that’s forte was tape libraries didn’t particularly make the best business sense.

So what future does Oracle have in plan for Sun’s current Storage portfolio? One certainty is that the OEM partnership with HDS’ enterprise arrays will continue, but as for their own range of modular arrays the future doesn’t look so promising. In a market with products such as EMC’s Clariion, HDS’ AMS range and ironically Larry Ellison’s Pillar Data systems, the truth of the matter is that Sun’s current modular range simply can’t compete. As cost effective as they are, their performance and scalability were always limited in relation to their direct competitors, something that was already acknowledged by Sun prior to the takeover when they disbanded the SE6920 due to its direct competition with the HDS equivalent USPVM.

Furthermore if Oracle’s push with the Exadata V2 is a sign of things to come, one can hardly see them developing an integrated backup model based on an increasingly frowned upon tape infrastructure made by StorageTek. Don’t get me wrong, I’ve worked with the SL8500 tape library and often wonder in amazement as the robotic arms gesticulate as if they were in the climactic scene from a Terminator movie. But that’s the problem …. it’s so 1990s. Add to the equation the NAS based SUN 7000 Unified Storage System which has received rave reviews and the question resonates as to whether Oracle will forsake its modular storage and tape libraries to further focus on just this trend.

Another venture in which Sun entered yet in hindsight did little to further their reputation was server virtualization. While VMware was taking off at the time with ESX 3 and the magic of Vmotion, DRS, HA, VCB etc. Sun had the dilemma that the server virtualization revolution taking place was compatible on x86 architecture and not Sun’s mainstay SPAARC. Not satisfied with reselling VMware for its x86 platforms, Sun decided to introduce their own version of virtualization which was compatible with their SPAARCs, namely Global zones. With huge monster servers such as the M series, the concept was to have numerous servers (zones) utilizing the resources of the one physical box i.e. the global zone. But in an industry that was moving further towards blade servers and consolidation via virtualization, the concept of having huge physical servers housing several virtual servers that couldn’t be Vmotioned and could only offer high availability by having a cluster of even more huge servers, seemed bizarre to say the least. No one disputes the great performance and power of Sun’s SPAARC servers but to offer them as a virtualization platform is completely unnecessary. Moreover the x86 platforms which haven’t radically changed over the years apart from their purple casing now being a slicker silver one, have also proved to be less than reliable when ESX is installed upon them. Indeed my only experience of the legendary PSOD was on the one occasion I had witnessed ESX installed on Sun x86 hardware. As RedHat and others make moves into the virtualization sphere with solutions superior to the Sun model, the questions begs as to what role virtualization will hold for Oracle. Larry Ellison has already made it evident that he wants to give full support for the SPAARC, but I’m not so sure, especially when Oracle decided to house Intel Xeons and not Sun SPARCs as the core of their Exadata V2.

As for the excellent host based virtualization of VirtualBox, the opensource nature of the product simply doesn’t fit in with Oracle’s approach of utilizing its dominant position to leverage big bucks from its customer base. With Oracle also already having Xen-based virtualization technology, I doubt virtualization will remain in the development radar of the newly occupied Sun offices. Come to think of it, will any of the opensource products remain?

Another aspect which worries me even further is the future of Solaris and ZFS. Despite Larry Ellison’s quotes of focusing on Java and Solaris, Solaris administrators still feel a touch uneasy, something which RedHat have taken advantage of by offering discount Solaris to RedHat conversion courses. As for ZFS, I’ve made no qualms as to my admiration of what is the most system admin friendly file system and logical volume manager on the market. But the recent legal wrangling over copyright with NetApp which is sure to escalate and Apple’s subsequent rejection for their OS leaves the revolutionary filesystem in a rather precarious position. Is Oracle going to put up a fight or will it be a case of no profit means no gain?

Despite the great wedding celebrations and fanfare which will inevitably occur during the honeymoon period, I will sadly shed a tear as a fair maiden that believed and stood for the virtues of platform independent technologies is to be whisked off into the sunset by another burly corporate man. One can only hope that the aforementioned kiss is one of love and understanding which will rejuvenate Sun and not a fatal kiss of death.

Green Storage: MAID To Do More Than Just Spin Down

The fundamental Green problem of all data centers is that they cost a fortune to power up, which in turn produces heat which then costs a fortune to cool down. Within this vicious circle a bastion was set up to counter this, namely ‘Green Storage’, which has mostly taken shape in the form of virtualized storage, data deduplication, compression, thin provisioning, DC power and SSDs. Add to the circle power conservation techniques such as server virtualization and archiving onto nearline storage, and you have most companies claiming they are successfully moving towards being ‘Green’. Hence bring forth the proposition of MAID storage and many users would not see the need for it. Further reluctance towards the technology would also come from the fact that MAID has now somewhat tragically become synonymous with being merely a disk spin down technique, despite having the potential to be far more and concurrently bringing greater cost savings.

First coined around 2002, MAID (a massive array of idle disks) held a promise to only utilize 25% or less of its hard drives at any one time. As Green technology became fashionable, major storage vendors such as EMC, HDS, Fujitsu and NEC began incorporating MAID technology promising that drives could power down when they weren't in use, thus extending the lifecycle of cheap SATA drives and in turn reducing the costs of running the data center. But caught in the midst of being one of many features that the larger vendors were trumpeting, the development and progress of MAID failed to advance from its disk slowdown tag, leaving Data Center and Storage managers oblivious of its full potential. Furthermore MAID became associated with being a solution only suited for backups and archive applications as users were cautious of the longer access time that resulted as disks spun up after being idle.

Fast forward to 2010, with government regulations demanding more power savings, and the back of a year which saw data grow while budgets shrank, suddenly users are looking for further ways to maximize the efficiency of their data storage systems. Hence in a world where persistent data increases a hero in the vein of MAID may just be ideal.

To be frank, the detractors did have a point with the original MAID 1.0 concept which in essence was simply to stop spinning non-accessed disk drives. Also with a MAID LUN having to use an entire RAID group, the prospect of a user with less than a large amount of data meant an awful lot of wasted storage. Add in the scenario of data put on MAID that suddenly requires more user access and hence constant disk spin and the overall cost savings became miniscule. Therefore those that did go for MAID ended up utilizing the technology for situations where access requirements and data retrieval were not paramount, i.e. backup and archiving.

In retrospect what often gets overlooked is that even with tier 2 and tier 3 storage data only a fraction is frequently accessed therefore leaving MAID as a suitable counterpart to the less-active data sets. In conclusion the real crux of the matter is the potential access time overhead that occurs as disks have to be started up, which is a given when only one spin down level is available.

Now with updated ‘MAID 2.0’ technologies such as AutoMAID from Nexsan, varying levels of disk-drive ‘spin down’ are available which utilize LUN access history to adjust the MAID levels accordingly. With Level 0 you have hard drive full-spin mode, with full power consumption and the shortest data access time while Level 1 allows the unloading of disk read/write heads giving 15%-20% less than Level 0 in power usage and only a fraction of a second less in access time. Additionally you have Level 2, which not only unloads the disk heads but also slows the platters a further 30-50% from full speed, giving a 15 second range for access time on the initial I/O before being jolting up to full speed. Similar to MAID 1.0, Level 3 allows the disk platters to stop spinning; bringing power consumption down by 60%-70% with an access time of 30-45 seconds on the initial I/O. In a nutshell these various levels of MAID now open up the doors for the technology to be a viable option for both tier 2, 3 and 4 storage data without the apprehension of delayed access times.

Some companies have gone even further with the technology by adding the ability to dedupe and replicate data in its libraries. Thus users have the option to isolate drives from the MAID pool and dedicate others for cache, leaving the cache drives to continuously spin while simultaneously increasing the payback of deduplication. The possibilities for remote replication as well as policy-based tiering and migrations are obvious. An organization with a sound knowledge of their applications could make significant savings moving data off expensive tier 1 disks to a MAID technology that incorporates both deduplication and replication capabilities with minimum if any performance loss.

Moreover using MAID technology in a context where data becomes inactive during the night (user directories, CRM databases etc.), disks can easily be spun down when users leave their office. Saving on unnecessary spin cost and energy for numerous hours each evening, by also using an automated process for long periods of inactivity such as holiday periods, users would quickly increase energy savings as well as decrease man management costs.

No doubt that in the current mainstream MAID is still best suited for persistent data that's not static and depends largely upon accurate data classification practices. But once MAID 2 and its features of variable drive spin-down, deduplication and replication begin to get the attention they deserve, we may well see a ‘Green’ solution which really does bring significant cost savings and energy savings. With the real ‘Green’ concern of most IT Directors being that of the paper kind with Benjamin Franklin’s face on, that attention may just occur sooner than we think.