Another important dynamic we’ve observed is that VMware administrators are finding themselves becoming quasi-storage administrators, which is somewhat akin to asking a brain surgeon to operate on the heart. This is not due simply to the inherent complexity of most networked storage systems; it is an issue of specialization as well.

Ironically NFS, the protocol of geeks, is a means to simplify VMware and networked storage. It turns out there are a number of customers using NFS with VMware and as a result they have eliminated much of the day-to-day complexity of managing network storage with VMware. The following are the advantages of using NFS with VMware versus FC and iSCSI SAN approaches:

• It’s very simple to add NFS datastores. With NFS there is no LUN management. This simplicity addresses configuration issues that might come back to bite you in a SAN environment. The storage provisioning process for SANs often requires a dozen or more time-consuming steps. Missing any step may result in disaster and you might not know it until it happens.

• You don't have to deal with all of the complexity of Fibre Channel, WWNs, zones, ISLs, etc.

• You can easily increase or more importantly decrease the size of NFS datastores online for capacity reclamation.

• Since you are using NFS you don't have to deal with VMFS or RDMs. This means you can have volumes bigger than 2 TB. NetApp supports a 16 TB file system; Isilon and BlueArc can support 100s of TBs in a datastore (and theoretically more).

• There is no single disk I/O queue with NFS, which means that performance is dependent on network bandwidth and the storage system itself. It also means that NFS performance can keep up with even FC in VMware environments.

• You can backup whole VMs, or files within VMs.

• Restoration of VMs is flexible including individual VMs, multiple VMs or files within VMs.

• The cost of FC is higher including the equipment and support. This is an argument for both NFS and iSCSI over FC.

• One customer pointed out that the tools to troubleshoot IP networks are much better than FC, another advantage for both NFS and iSCSI.

• SAN expertise is more specialized than IP – it is harder to find the experts and to retain them (and they usually paid more).

The problem is there aren’t many vendors that offer storage systems that support NFS. There are literally dozens of SAN-based storage systems but only a handful of NFS or NAS storage systems in the market. Customers want choices and the fewer there are the less likely they will go down a particular path. And there are good reasons since competition fosters innovation, cost effectiveness and better service.

NFS for VMware has the potential of changing the storage landscape, but sadly, it probably won’t. The reasons are: it is a major challenge to educate the market; there is an enormous amount of incumbent SAN storage systems; and there is no one taking up the mantle to fight this good fight. Even NetApp has lost its fervor for NFS and has instead taken a “we provide whatever protocol you want” attitude, responding to the market versus driving it. That isn’t to say that NetApp isn’t promoting the use of NFS with VMware but it is not a core strategy and is only one of many things they talk about. BlueArc and Isilon are now focusing on NFS for VMware but they don’t have the same level of awareness or resources as the big storage vendors. However, if NFS for VMware is going to take off, these two vendors are in a position to sow some seeds and reap the rewards.

NFS is a more highly virtualized protocol than FC and even iSCSI. As a result it works much more easily and efficiently with VMware. If NFS is to become the dominant VMware protocol it is probably not the storage specialists that will make this happen but rather the VMware administrators who want highly virtualized networked storage without having to be a storage expert. The implications of this are significant as VMware continues to proliferate and those that manage these environments influence and decide what they need to be successful.

File Sprawl
The first big problem with file content is quite candidly, users. People create, copy, convert, forward, edit, scan and download files every day. It is the Wild West with few controls or restrictions. I remember one customer discovering they had 125 copes of a scanned Chinese menu on their Tier One storage system.

When you consider hundreds, thousands and tens of thousands of individuals creating all of this content you can do the math and see how easily file sprawl becomes a real and pervasive problem. There are a growing number of customers that have 100s of TBs and PBs of file storage. In many cases IT professionals have no idea how much file content they have, the value of that content, how much it is costing them, where it is being stored or how it is being protected.

Not only are we creating a ton of files but also many of these include images, video and audio content. Therefore we are creating lots of big files and this results in the consumption of expensive and hard to manage IT infrastructure.

NASty
Which brings me to the next big problem with file content and that is how we store it. Much of file content is stored on NAS storage systems and although there is great value in these solutions, they also create problems for IT professionals. For one, there are only a few vendors that provide NAS solutions and customers have a limited number options to choose from. Clearly, having more viable solutions in the market would foster more competition, cost effectiveness and innovation.

I’ve been talking to big NAS shops and one of their biggest challenges is NAS migration. Customers with hundreds of terabytes and petabytes of NAS file content feel they are essentially tethered to specific NAS devices because the complexity of moving that data is often perceived as insurmountable or far more trouble than its worth. One customer said he felt that he was being perpetually held for ransom by his NAS storage.

Unstructured = Useless
We often refer to files as unstructured data. Since by its very nature there is a lack of structure to unstructured content, it is hard for IT professionals to make any use of file data. However, we don’t dare risk deleting the majority of it because of the risk of needing it - and for most companies and organizations the cost of risk is less than the cost of capital.

Interestingly, industry studies have found that 60-80% of unstructured content is never used ninety days after its initial creation. Which really makes unstructured content synonymous with “useless” content. It cost so much to store and protect file content but then we never actually use it. Is it because the content really has no sustainable value or is it because we just don’t have the tools to easily and effectively make use of it?

The Backup and Recovery Conundrum
I believe the biggest challenge in a petabyte world is backup. Consider the new landscape with hundreds of terabytes and petabytes of file content being stored on multiple storage systems. Now ask, how do you protect all of this file content? And then think about how much that protection is going to cost you in money, time and resource. Legacy methods and the status quo are insufficient to meet the needs of today’s requirements. This means either a new method of file protection is required or you take the risk of not being able to recover data. The latter choice is a hard one to make especially when you consider the consequences could permanently damage your business and result in executives being made personally accountable. I consider this to be one of the biggest issues in the data center for the decade.

For the longest time we’ve been able to continue doing business as usual and solve the problem by throwing more IT infrastructure and people at it. We are at an inflection point where we can no longer be complacent with the status quo. Managing massive file stores is one of the “big” problems in the data center for the decade and IT professionals need to sound the alarm and make this a real priority.

The IT professionals that I’ve spoken to about private IT Clouds consider it to be the realization of IT as a utility within their own companies. It utilizes physical IT infrastructure for greater economies of scale, which is the essence of what a utility is all about. We've been talking about turning IT into a service utility for decades, so what has changed in that time? The biggest thing has been server virtualization, which creates an N-to-1 virtual server to physical server ratio. VLANs have done the same thing for networking. Additionally, some storage systems allow you to create virtual storage systems in a similar fashion. We have advanced IT technology to enable multi-tenancy and create logical systems within physical ones. The virtualization of IT infrastructure is relatively new and essential to enabling private IT clouds.

The critical pieces still missing for private IT clouds is the management, policy-based controls, reporting, analysis and "billing" systems for the entire IT ecosystem holistically. This is what will elevate private IT clouds from being a disparate set of virtual and physical infrastructure solutions to the new way we manage our data centers. However, this is the harder part to build because it requires core competencies that the infrastructure vendors don’t have. Additionally, you need to work with a wide range of solutions and vendors that may or may not cooperate.
Private IT clouds are real and will change the IT landscape but nothing is binary. We don’t go from doing things one way and switching it to another over night. It takes years with progress occurring step-by-step. There are points of acceleration and depending on the ease of making the shift, the clarity of the value proposition and the support of the ecosystem, this can happen sooner or later. Additionally, as markets emerge no one can really predict how it will all play out.

So what is an IT professional to do? Do what you always do when faced with something new and emerging. The early adopters will lead the way and some will make great choices and others will make mistakes. Keep reading, researching and analyzing. You might want to even dip your toe in if the cost and risk is minimal. Set cynicism aside and understand how private IT clouds can impact your environment for the short term and continue to evaluate the long-term implications.

- Nasuni provides a "NAS-like" software appliance. NAS-like is their term - so much for naming creativity - but essentially it is software that you download and run on a physical server as a virtual appliance.

- It looks like NAS storage to your applications. Right now it only supports Windows environments. Since they are going after the SMB market focusing on Windows is pragmatic. They will support NFS based on market demand.

- It takes "15 minutes" to get up and running and storing files in the cloud. They work with leading storage cloud providers including Amazon S3, Iron Mountain ASP and will soon support Nirvanix.

- One of the core technologies of Nasuni is its file caching capability. They keep the most frequently accessed files locally so you get fast performance and rarely have to perform reads from the cloud provider. It remains to seen what their cache hit rates are but it is in their best interest to make it as efficient as possible (see below for more on this). But their engineering team are cache algorithm gurus so I am pretty confident they did a good job.

- All billing is through Nasuni. I love this part for two reasons: 1. It makes the cost of the service very easy to understand. 2. The customer just has to go through Nasuni to set everything up. Making it easy for people to use public cloud services is extremely valuable and essential to its proliferation.

- Nasuni charges customers $300 to use their appliance on a month-to-month basis; $250 per month if you prepay for one year; and $200 if you prepay for two years. They pass through the cost of the cloud capacity with no mark up. Additionally, Nasuni PAYS for the cloud bandwidth service charges - the upload/download fees. That last part is very important since it is the variable that could make public cloud storage costs unpredictable and cost prohibitive. The reason Nasuni can do this is because of their file caching - since most of the reads will be local and rarely pulled from public cloud storage they will not incur heavy costs for bandwidth utilization. If you want to store 1 TB of data onto Amazon S3 for example and have a one year pre-paid deal with Nasuni the total costs for you will be $4,800. Its pretty simple.

- What do you get for your $4,800? You get the capacity in the cloud. You get NAS capabilities including file level access and logical snapshots. You get fast performance utilizing Nasuni file caching. Nasuni encrypts the data so it is secure. Additionally, they claim you don't have to backup any of the data because they keep protection copies and provide you with an easy way to recover data at the file or directory level. Nasuni stores both primary and snapshot copies efficiently using file level data deduplication and logical snaps. This last part is very important since the cloud storage providers don't provide these capabilities and this is where your capacity costs could otherwise be a runaway train.

- And you don't have to pay for the power, cooling and floor space that local storage requires.

- Also keep in mind that this is a Just-In-Time capacity model. You don't get charged for unused capacity but only for how much storage that the actual data consumes. In a world of 20%-50% capacity utilization rates that translates into significant cost savings. Additionally, you don't get any RAID capacity penalties. All of these things must be factored into your cost savings calculations.

- Another interesting thing about Nasuni is that their "NAS-like" appliance is a fixed charge. They don't make any more money if you are using 1 TB or 10 TB or 100 TB. The cost of using Amazon S3 capacity will go up but thats it.

- Now we must consider that the $3,000 or so cost of Nasuni software is an annuity. Customers have been saying for years that they are tired of paying a price per GB for storage systems. Well, Nasuni throws the old pricing model out the window. If you go with their two year option the cost of their software is $2,400 a year. Over four years it amounts to $9,600. The cost of other storage vendors' annual maintenance is higher than that. Additionally, with Nasuni there is no additional maintenance cost - the annual charge includes support and upgrades.

- Nasuni supports heterogeneous cloud services. You can use individual cloud providers or a mixture if you want. You can switch providers if you are unhappy with any of them (they don't have a bulk move tool yet but this is coming). Right now the list of service providers is limited but I suspect this will grow over time.

- If there are any issues with your service then you call Nasuni and not the cloud providers. Again, this simplifies the experience of using public cloud storage.

- I like their animated videos. I am rather critical of how vendors educate their customers but I give Nasuni points for telling their story intelligently and quickly and without me wanting to stick a sharp object in my eye. I also like their FAQ section - it lays everything out very clearly.

- Nasuni seems to make a lot of sense for file sharing, tiered storage, secondary copies, and backup. Both SMB and remote office / branch office customers should be interested in this solution. I also think that there is a use case for collaboration in the cloud using Nasuni between employees and/or partners.

Nasuni has thought through their solution holistically keeping the customer experience top of mind. Their biggest challenges will be over coming cloud cynicism. But I contend that the Nasuni product was designed to address the limitations and objections to using public IT cloud storage. Additionally, with their price point they will need to reach a lot of customers. Nasuni has to build an efficient machine to drive up awareness and customer acquisition. For IT professionals considering the cloud, Nasuni makes it easy and the cost and risk of using their solution is minimal.

A brief description of the Solix ExApps appliance. First, this solution is designed for data base applications. It is an appliance that provides the software and storage for access and retention of the data from the retired application. In other words - you will move the data from the retired app off of your primary storage system to the ExAPPS appliance. And since the application is retired you will also access the data via the ExAPPS appliance as well as a gateway.

The things that I think are valuable and compelling about ExAPPS:

1. It is application-aware and supports ODBC and SQL92. You can retire your applications but still read all of the data, run reports and queries.

2. It re-organizes the database format and reduces the storage capacity requirements - according to Solix - by 90%. This significantly reduces your capacity costs, floor space, power consumption and cooling requirements. Remember that the application is being retired and the data is fundamentally being stored as an active archive. "Compressing" it - or more accurately - reformatting the data so that it is capacity optimized is the smart and practical thing to do. Will it perform as well? Probably not - but not because of the reformatting but because it isn't being run on big servers or big storage any more. But who cares? It's retired!

3. Since ExAPPS is actually reformatting the structure of the database it doesn't require any "rehydration" of the data. This means that even if you have to fire up the data and access it - the entire process requires no additional capacity or processing cycles as a result of a rehydration process.

It is also important to note that retiring applications is typically not taken care of by an appliance alone but requires planning and services. These projects can take months to accomplish effectively. So Solix and probably over time their channel partners will work with you on this process as well as provide the appliance.

ExAPPS is an interesting solution. Solix has taken what they've learned with database archiving and developed a solution specific to database application retirement. They were able to use a good portion of their current technology and add some additional functionality - in effect creating a blue ocean market for themselves leveraging their unique application retirement appliance.

I haven't spoken to any of their customers yet. Solix claims to have a handful of early installations with some big companies. And I intend to talk to more customers in general about the challenges with application retirement and how big of a priority this is for their respective companies. Application retirement appliances could be a new segment and other archiving companies may follow Solix's lead. It could be very interesting to see how this evolves. The IT world is getting "old" and we really do need to start thinking about application retirement as a priority.

The cost of these systems and the amount of time required to build them were daunting and his personality was so volatile that he alienated the ecosystem required to build them.  He finally gave up and said that "another age must be the judge".  Babbage was a victim of his arrogance and as a result he played a big part in the destruction of his own dream. He also over-promised and under-delivered spending years and years on the project and endless amounts of money. At some point his funding source - which was the British government - terminated the relationship after spending what it would cost to build 22 brand new steam locomotives. Not only did the funding end but so did any good will he had with the British government since they refused to consider further funding of his Analytical Engine - which could have changed the world in deep and profound ways.

This is from the Computer Museum website - "The first complete Babbage Engine was completed in London in 2002, 153 years after it was designed. Difference Engine No. 2, built faithfully to the original drawings, consists of 8,000 parts, weighs five tons, and measures 11 feet long."  And it works.

It was very cool to see the thing in action - all of the little widgets rotating and gyrating making clinking and clanking noises.  I highly recommend visiting if you are in the area.  (Btw - the photo was taken by a very nice girl named Sara. I left my camera back at the hotel and asked Sara if she could take one and email it to me. She did a great job too! She got Babbage in the background, the Difference Engine and me all in the frame. Well done Sara and thank you!)

What would have the world have been like if the computer was invented over 120 years earlier? Would there be an IBM? Intel? Microsoft? Apple? VMware? It was so close to happening, albeit in a different manifestation but the value would be very similar. Certainly the world as we know it would be measurably different - a compelling pun to consider when contemplating the Difference Engine.

My first reaction when Cisco announced getting into the server business was - well let's just say I was a bit skeptical. First , Cisco UCS upsets the IT ecosystem, which is not necessarily a bad thing but it does have ramifications that need to be considered in terms of go-to-market, the impact to Cisco's channels and alliances, and of course it creates natural enemies that are big and powerful enough that even Cisco needs to be mindful of them - including Dell, HP and IBM. Certainly all three are already competitors on various levels but they also have alliances and must live together in the data center. But now Cisco is going after their bread and butter server business, which is a whole new ball game.

The second major reason to be skeptical is that the core competencies needed to sell and support servers is different than supporting networking devices. Cisco doesn't have the operations, sales, marketing, support, on and on and on. Third, server margins are not nearly as healthy as typical Cisco products. Do you want to focus your time and resources making 60 or 70 cents per dollar or 20 or 30 cents per dollar? The math is pretty simple. Related to that the server market is boring - it is declining and not growing.

Now this is a big deal to Cisco. Clearly a lot of thought went into this. And there are some really smart people that work there. So what is the business case that makes UCS compelling and strategic? Now maybe everyone reading this is already way ahead of me and get why this is important but I haven't read or seen anything concerning UCS that really focuses on my hypothesis and why this matters:

Cisco isn't really going after the server market per se but rather it wants to own the core of the data center. It wants to own the infrastructure that runs applications that run the business. At the heart - at the very core of the data center are compute resources - CPU and memory. "Owning" the core of the data center - can make you strategic, valuable and essential.

But it will take a lot more than just providing commodity CPUs and memory to own the core of the data center. Server virtualization by its very nature enables you to move from one CPU to another transparently. That is why if you want to own the core of the data center you need to really consider the value add provided in addition to the compute resources. And that value add has to be very, very sticky. Additionally, it should be something you are already really, really good at. For Cisco the first step in the value add strategy is a natural and obvious - go to where you are strongest - and with Cisco that is networking.

Therefore the other part of the UCS strategy is focused on converged networks, everything over Ethernet including the last big hold out, Fibre Channel. Fibre Channel over Ethernet (FCoE) becomes very important to them especially since networked storage is a critical part of the data center. A converged network using Cisco infrastructure including LAN, backup, management, VOIP and all storage protocols including iSCS, CIFS, NFS and FCoE results in lower cost, less support, and the reduction of cable spaghetti. The Cisco vision is to have all roads lead to Cisco UCS via a converged Cisco network.

By binding the core of the data center to the network - the arena that Cisco owns - gives them a major advantage and allows them to leverage their core competencies. And they can add more value - easier management, consolidated infrastructure, more efficient controls between the core of the data center and all the roads that lead to it. Additionally - you need the network in order to have a more efficient virtual server environment for mobility, availability, reliability and disaster recovery.

Cisco doesn't own the server virtualization software, the network storage, the applications - but they provide a platform to bring all of these things together. And they have the leverage - based on economics - to have everyone want to be their partner, even some of their major competitors - co-opetetion - because again of the economics involved. That is another essential part of Cisco's value add strategy, get all the partners to integrate and optimize around the UCS platform as evidenced by the VMware, EMC, NetApp, Cisco alliances. This intelligent and important move brought them closer together with the biggest players that provide the other essential aspects of the virtual data center - server virtualization and storage.

I am not convinced that Cisco is going to rule the universe with this strategy but it certainly poises them for owning more strategic footprint within their customer base (which is pretty much everyone). And I am just talking about strategy and vision - it is the go-to-market execution that will matter here and there are still a number of challenges facing them.. Dell, HP and IBM will not sit idly by and watch Cisco run rampant over them. Keep in mind that while Dell, HP and IBM servers are in a lot of places, they do not enjoy the same ubiquity as Cisco networking solutions. On the other hand these leading server companies are ready and willing to move money around to make deals. Cisco will have to perform a tough balancing act. They cannot win this game by matching hard dollar savings versus soft dollar esoterica and cost saving rhetoric. At the same time they can't allow themselves to lose sight of the bigger vision and driving customers viscerally towards it.

Integrating compute resources with networking solutions makes a ton of sense if you are Cisco. UCS enables Cisco to be the core of the data center with all roads leading to that core provided by Cisco's converged networking solutions. It is a strategy of Core + Access. For years Cisco has struggled to be more than IT "plumbing" - to be more strategic to IT, let alone the actual business units. By running business applications on Cisco equipment they are far better positioned to achieve this objective. By elevating themselves through providing solutions where business applications "live" is strategic and could be landscape changing.

- Intelligent tiered storage will be provided by a number of storage vendors. The key will be that the tiering will be at a sub-LUN level and will be based on policies. I cannot emphasis the value of intelligent tiering enough - I write about it in more detail in these two blogs here and here.

- File services will become more important as file data continues to consume more and more of our storage capacity. Included in this will be NAS and CAS solutions as well as software that manages this data. It is important that we efficiently store file data and we must make sure that all of this unstructured content becomes useful to our businesses. Right now unstructured data is pretty much synonymous with useless data. This dynamic must change - the enormous amount of unstructured content is the elephant in the data center and it is just sitting there taking up lots of space and needing to be constantly fed (I will end my analogy there so I don't offend anyone).

- Private IT clouds will be top of mind for many IT professionals. They want to leverage virtualization, management, policies and reporting to implement utility models for their IT environments. Sure, we've been talking about this for 20 years and it still hasn't really happened en masse but we now have more tools at our disposal than ever before to enable it.

- Public IT clouds will continue to be used for new web-based businesses, and lower tiers of applications and data. However, the big impact to the brick and mortar IT world is the use of applications in the cloud. The more customers use these services, the more IT infrastructure that normally would have been used in-house will not be required.

- Green IT will be more of a priority (or is this just wishful thinking on my part?). Data Centers produce 6% of green house gas emissions worldwide - more than the airline industry. It is essential that IT finally embrace Green as an initiative. And by doing so they will reduce cost and feel like they are actually doing something positive for the world. I quote the comedian Lily Tomlin once again - "I always wondered why somebody doesn't do something about that. Then I realized - I was somebody."

- More disk-to-disk backup with dedupe and less tape in the data center is inevitable. The majority of the world still uses tape as their primary backup target but the shift to disk with dedupe is well on its way and will become more pervasive in 2010.

- Automation is essential in a world that consists of PBs of storage capacity. How can you hope to efficiently manage all that infrastructure without letting smart machines make decisions for you? Yes, that is a scary thought but it is the only way to manage. That is why visibility is required for automation to work. IT professionals need the reporting tools to assure them that things are running efficiently.

- Data dedupe or rather data compression in primary storage will become a priority - if not a reality - in 2010. The problem with deduping data in primary storage is that if it is done in real-time it interjects latency. And if you want to access the data after deduping than you have to re-hydrate that data. However, with intelligent tiering - as described above - you can actually create a policy that moves unused data to a lower tier and then compress it. Intelligent tiering at the sub-LUN level enables other efficiencies - which is why it is so important to storage optimization. This solves the latency problem because it is a post process and rehydration isn't an issue because it is data that no one is accessing. And let's face it - 60-80% of your content is never accessed 90 days after its creation. That means that with 1 PB of data, 600 to 800 TB can be deduped or compressed. Even at a ratio of 2-to-1 you can lower capacity in that scenario by 300 to 400 TB. Sounds pretty good to me.

- Along those same lines, intelligent sub-LUN tiering will make SSD / EFD much more valuable to customers allowing them to have small amounts of this technology being used to maximum effect. As a result less FC drives and more SATA will be found in Enterprise-class storage.

- Cisco, Dell, HP and IBM will offer the IT world "unified" platforms. Some will be more unified than others. In the case of Cisco's Unified Computing System (UCS) they don't own much of the intellectual property but instead rely on VMware and EMC for the virtualization and storage systems. Dell, HP and IBM have the storage piece in in the case of HP and IBM can also use Xen or perhaps even KVM for the virtualization. But in the end, these solutions are integration of existing technologies - some of it good; some of it mediocre and some of it crap. And the TCO will be at best pretty good and at worst unquantifiable. Coming to a town near you in 2010.

- Desktop virtualization will rise in visibility, offerings and implementation in 2010. You don't virtualize desktops for consolidation but for management and security. IT professionals can upgrade and replace these systems much more easily. If the data is stored on network storage then the company controls it and not the user. It also reduces infrastructure costs by removing capacity from each desktop.

- PC power management is becoming a big item and will be a priority in 2010. There is software out there that manages the PC power down process and saves between 30% and 50% of the electricity bill of these systems. It is kind of a no-brainer to implement especially since many power companies actually give you an incentive to implement this software.

- Server virtualization continues to thrive. Most of the customers I've spoken with this year have implemented server virtualization and there is a growing number that have a "Virtual First" policy. In other words, applications are going on virtualized servers by default and if the business unit doesn't want this they need to justify why they need a dedicated physical server.