Welcome to the Flash Party EMC

Tuesday, November 12, 2013 posted by Dave Wright

EMC is launching their first new array architecture in over a decade this week with the XtremIO all-flash array. Prior to launch, following a long industry tradition, EMC is equipping their sales teams with “kill sheets” against other flash vendors. Typically these documents live in the murky world between “creative” positioning, FUD and outright untruth, but seldom are they publicly available, as is EMC’s SolidFire kill sheet. (ed: Document Removed) However, since it’s out there, it provides an interesting opportunity to compare and contrast EMC’s approach to to flash storage, and the corresponding product strengths and weaknesses.

On the surface, there are some clear parallels between the XtremIO and SolidFire architecture. Both utilize scale-out approaches that allow performance and capacity to scale. Each system utilizes in-line deduplication with a content-based placement algorithm to distribute data across the cluster. However, beyond the surface similarities, there are significant architectural differences that will dictate both where and why customers would use XtremIO.

EMC’s approach to scale-out is a series of traditional dual-controller / shared-disk 6U “bricks” linked together over an IB backplane. SolidFire utilizes shared-nothing scale-out 1U nodes linked together over standard 10GBE. EMC has optimized around high performance at low capacity points, while SolidFire has focused on high density for both capacity and performance while also being cost-competitive with disk. Each approach has its tradeoffs. Let’s look at some of EMC’s positioning points:

  • Ask SolidFire about the failure rates of consumer grade MLC flash memory and why they are not using eMLC for enterprise SSD resilience. 
    The irony of this argument is palpable. SolidFire, and every other ground-up flash architecture on the market, uses standard MLC flash. Like others, SolidFire has an architecture that minimizes any wear concerns, and we warranty the drives for five years. Real-world failure rates are dramatically less than spinning disk. The real question is: Why is EMC forced to use significantly more expensive eMLC on a supposedly flash optimized architecture?
  • XtremIO’s content aware architecture delivers up to 250K IOPS per node with <1 ms latency performance during real world workloads. SolidFire claims best case 50K IOPS per node with <2 ms latency performance.
    Besides being factually incorrect, as SolidFire offers up to 75K IOPS per node and <1 ms latency, it’s really not reasonable to compare a 6U / dual controller / 25 drive X-brick to a single SolidFire node. In the same 6U of rack space, SolidFire provides a unified pool of 450K IOPS. Furthermore, XtremIO’s 250K IOPS is based on a 100% read workload, rather than the more real-world mixed workload that SolidFire (and most other vendors) uses.
  • XtremIO has a no single point of failure (NSPOF) design with all drives accessible to active/active storage processors. SolidFire nodes have a single controller so a failure disables the entire node and up to 20% of capacity.
    SolidFire has a NSPOF design. Any component in the system - including entire nodes -  can fail, and not only will SolidFire continue to run with guaranteed Quality of Service, it self-heals and rebalances data among the remaining nodes. By comparison, if XtremIO loses a controller, it may continue to run but loses significant performance (due to active/active controllers). More importantly, the system is now non-redundant until the failed controller is replaced. If a shared disk shelf fails, the complete XtremIO system goes down as there is no redundancy beyond RAID within the disk shelf. The same goes for any failure that disables an entire X-brick. In reality, SolidFire’s HA design goes far beyond traditional dual-controller models by removing problem points such as shared-disk and eliminating the storage-failure fire drill with automatic self-healing and guaranteed QoS.
  • Ask SolidFire about their < 50% raw capacity per node… XtremIO raw capacity is more than 70% due to an efficient data protection algorithm with no mirroring.
    SolidFire utilizes a unique data protection scheme called Double Helix based on a patent-pending distributed replication architecture. Double Helix protects against both drive as well as node-level failure. Because we don’t need to do parity reads during drive failure, rebuilds complete faster and with minimal performance impact. XtremIO utilizes a modified RAID6 to protect data within a single disk shelf only. There is no protection between nodes, and the system requires expensive dual-controllers for EACH disk shelf to provide redundancy. The small trade-off we make in capacity is actually more than paid off by not requiring a second controller and separate disk shelf hardware.
  • Ask SolidFire to compare their 1.5kW energy efficiency for 5 nodes to XtremIO. A single X-Brick is half the power footprint at 750 watts.
    Glad they asked! SolidFire’s 5U / 5-node SF9010 footprint provides up to 48TB (RAW) and 173TB (effective) capacity. A single 6U X-brick 10TB (RAW) and 7TB (usable) capacity. The power/capacity efficiency isn’t even close! SolidFire requires less than half the power of XtremIO for similar capacity, not to mention a tiny fraction of the rack space. Even SolidFire’s smallest SF3010 node has better power and capacity density than XtremIO.
  • Ask SolidFire to show scalability to large node counts as all their customer wins to date are relatively small.
    Good to know EMC has such a clear view of our business! In reality, we have customers in production today with 20 node clusters, and some who regularly run 40 node clusters internally. By comparison, at launch XtremIO will support a max of four X-bricks but have previously mentioned scaling to eight in the future. Whether this is an architectural limit, or simply an attempt to protect the positioning of VNX/VMAX today isn’t clear, but what is clear is that SolidFire has the only production-proven scale-out flash with a full feature set, including data reduction, in the industry today. Even our smallest clusters are larger than the largest XtremIO system available at GA.
  • Ask SolidFire about their flexibility to start with a configuration smaller than 5 nodes. XtremIO starts with a single HA X-Brick for an attractive entry point.
    SolidFire’s starting footprint uses less rack space than a single X-brick, while offering significantly more effective capacity. A single X-brick certainly has less capacity, and will likely have a lower starting price, but fundamentally this illustrates the different use case that EMC is focused on. For EMC, XtremIO is a point solution for individual applications and use cases that simply can’t get enough performance from disk-based or hybrid systems. SolidFire was designed to fully compete as a complete disk replacement for multi-application and multi-tenant storage environments from 60TB to more than 3PB.

We could go on all day like this, but I think we’ve shown that while SolidFire has certainly made different design decisions in some areas (like shared-nothing vs. dual-controllers), there are strong advantages to the SolidFire approach in the use cases and environments we focus on. The SolidFire system leads the industry in flash density and cost-effectiveness, with a range of functionality such as Guaranteed QoS that can’t be found in any storage system - disk or flash based.

Finally, since EMC was nice enough to lay out some of their “questions” publicly,  here are a few to ask them:

  • Can you expand (or shrink) an existing cluster?
    Initial reports are that you can’t actually scale-out (or scale-down) an XtremIO cluster, or at least not without data migration. SolidFire provides a completely dynamic scale-out (and scale-down) capability with automatic data and IO balancing.
  • Why does each X-brick require a UPS?
    It appears that XtremIO commits writes in DRAM, rather than using an NVRAM device or power-protected RAID controller. The system requires a full UPS to protect against data loss on power failure. Few storage systems have ever taken this approach, as it carries inherent risks of data loss or corruption for situations like software faults or UPS failure.
  • What’s the performance impact of a controller failure?
    XtremIO utilizes a traditional active/active controller design, which means that loss of a controller translates to loss of half the IO ports and controller CPU. Usually this means a significant performance impact as well. By comparison, SolidFire’s shared-nothing model means that loss of a node only loses the capacity and performance proportional to what failed (for example, 5% in a 20-node system).
  • How much will it cost?
    Of course that’s the question everyone wants to know! EMC has a tricky job positioning XtremIO against its existing disk, hybrid, and even all-flash VNX/VMAX portfolio. Lacking compression, XtremIO is at a 2X or more effective capacity penalty compared to systems like SolidFire that provide in-line de-duplication, compression, and thin provisioning. The use of expensive “extras” such as eMLC, dual-controllers, separate disk shelves, and IB switches will increase the cost as well. Early reports have XtremIO at $8/GB or more, while SolidFire is now solidly under $3/GB for effective capacity. Cost certainly isn’t everything, but it further separates the use cases for something like XtremIO and SolidFire.

Hopefully we’ll see their response!

-Dave Wright, Founder & CEO