Monday, December 6, 2010

Understanding Cloud Storage

As the name indicates, Cloud storage is simply storage over cloud. It is the delivery of virtualized storage on demand over internet. It can also be defined as a model of networked online storage where data is stored on multiple virtual servers, generally hosted by third parties, rather than being hosted on dedicated servers.

The data center operators, in the background, virtualize the resources according to the requirements of the customer and expose them as storage pools, which the customers can themselves use to store files or data objects. Physically, the resource may span across multiple servers. Popular Vendors offering cloud storage are – Amazon S3, Windows Azure.





Cloud storage has the following characteristics:
  • Global Virtualization
  • Unlimited Scalability
  • Continuous Availability
  • Usage Based Service
  • Maintenance Services - Backup, Data Replication
Some of the real world use cases for cloud storage include:
  • Cloud Computing
  • Web facing applications
  • Backup and Archiving
IN PROGRESS....

FCoE: Converging Ethernet and SAN

Fibre Channel over Ethernet (FCoE) is an encapsulation of Fibre Channel frames over Ethernet networks. This allows Fibre Channel to use 10 Gigabit Ethernet networks (or higher speeds) while preserving the Fibre Channel protocol. In simple terms, FCoE allows running Fibre Channel protocol over 10 Gigabit ethernet. We will divide this discussion under the following heads.

1) Challanges with traditional data centers
2) Infrastructure required to implement FCoE.
3) Advantages of implementing FCoE.
4) Common challanges that enterprises face while migrating to FCoE.

Challanges with traditional data centers

A traditional data center will have separate infrastructure for Ethernet and SAN traffic. This basically would mean that every server will have a HBA and NIC. The data center will have both have an Ethernet switch and Fibre-Channel switch. There will be separate cabling for both Ethernet and SAN networks. It will look somewhat like in the following figure.



This setup has the following disadvantages.
  • Since the Ethernet and SAN networks will be running parallely it will lead to overall complexity.
  • High number of total number of overall adapters, cables, ports and switches.
  • Underutilized bandwidth.
  • High energy consumption


Infrastructure required to implement FCoE

As you can see that there is inherent complexity in the above architecture. FCoE provides a solution to this problem by converging Ethernet and SAN traffic. Let's see how. FCoE converges
  • HBA and NIC into one CNA (Converged Network Adapter)
  • Ethernet and FC switch into one FCoE switch
  • Ethernet and Fibre-Channel cables into one FCoE cable.

This converged network is then made to run on a 10GB ethernet which provides enough bandwidth so that both Ethernet and Fibre-Channel traffic can run concurrently.

Advantages of implementing FCoE
  • Reduces Complexity by 50%
  • Reduces total number of adapters, cables, ports and switches
  • Allows easy deploymeny in phases
  • Reduction in energy consumption
Common challanges that enterprises face while migrating to FCoE

It is obvious that when you plan to adopt a new technology you are bound to face challanges. Moving to FCoE is no different. Let's look at some of the challanges that most IT admins face.
  • Upper management in most of the organizations is hesistant in investing in FCoE infrastructure. Convincing them that despite of the initial investment FCoE will bring significant savings in future is a dawnting task.
  • In most organizations normally there are separate teams (sometimes totally different hierarchy reporting to a different director) managing ethernet and SAN network. To implement and run FCoE successfully both of these teams need to interact and work wit each other at the deepest level possible which is not an easy task. Both the teams will need training to work with FCoE (Learning curve will be higher for LAN/WAN team)
  • It is highly likely that your current vendor (For both LAN/WAN and SAN) does not has the expertise or support in FCoE infrastructure. It is also possible that switching to a different vendor will result in significant savings. If this is the situation you will have no option but to switch to a different vendor for implementing FCoE and this is not at all an east task. Most LAN/SAN adminsitrators and their teams have built-up an extensive experience in working with their current vendor. Hence a change to a different vendor is a bitter pill to swallow.
FCoE has come a long way. More and more vendors are coming with an end-to-end solution for FCoE. Recently NetApp and Cisco unveiled their End-to-End FCoE solution with VMWare. For more details look at: http://www.netapp.com/us/company/news/news-rel-20100728.html

Monday, November 29, 2010

Object Storage Devices

Object Storage Devices(OSDs) is the latest emerging technology in the storage arena. To understand the concept of OSDs let's first try to understand the limitations of existing storage technologies - NAS and SAN.

Let's talk about NAS first. In a typical NAS setup you will have a NAS server which exposes a single namespace and control point for business objects. Sounds perfect but there are inherent drawbacks with NAS technology where there is high data-transfer involved. Since NAS runs on high-latency TCP/IP protocol, LAN performance suffers when there is heavy data flow. NAS, though, has an advantage that it allows data stored on NAS file shares or filesystems to be shared across clients.

The above scenario reverses in the case of SAN. In SAN client and storage are connected via short-I/O paths and as a result is a technology of choice for data-transfer intensive applications. However, SAN does not allows data to be shared among clients. If client wants to shared data stored on SAN they will have to coordinate among themselves on their own.

As you can see both NAS and SAN suffers from drawbacks this is where OSDs come to the rescue. OSDs try to provide the best of both the worlds by overcoming the limitations of both NAS and SAN.
The fundamental innovation which enables this architecture is the reassignment of the responsibility for the mapping of I/O streams to sectors (block storage) from the client/server to the data store itself (OSD).
Since the OSD can perform the stream to sector mapping and the clients no longer need to worry about the algorithm to employ for this mapping, it makes OSD so suitable for data sharing in a cross-platform environment.

This cross-platform data sharing addresses the major benefit of NAS over SAN, and is inherent in Object Storage Arhcitecture (OSA).

The scalable performance is achieved by the fact that the data flow is direct between the client(s) and the OSD(s), without any need to pass through a server. This allows aggregate system performance to rise to the inherent capacity of the underlying fabric. This direct pipe from clients to storage addresses the major benefit of SAN with respect to NAS, and is inherent in OSA.

You just saw how OSDs combine the benenfits of both NAS and SAN into one storage device. In OSA apart from Client and OSDs there is one more actor involved - The Metadata Server(MDS). The MDS has the responsibility for maintaining all of the file system other than the mapping of streams to sectors. The MDS is where any hierarchical directory structure would be maintained, along with permissions, file-scope locking, etc.

Lets try to understand tis with an analogy of a 5-star hotel. Assume that the hotel is the OSD, various objects stored in there are the rooms of the hotel and the person sitting at the reception acts as MDS. Whenever a guest comes to the hotel and requests for access to a particular room he goes to the reception and asks for an electronic key to that room. The person at the reception also tells him the way to the floor. Once he has the key the guest can access the room any number of times. He will use the same key each and every time he wants to access the room. Also normally the key is valid for a certain period.

We just identified three primary objects in the architecture - client, OSD, and MDS. In any system, there are any number of clients, any number of ODSs, and one MDS. So one might think that because of the presence of a single logical MDS in any system, OSA also suffers from the same scalability problem as NAS does. It is correct that all communications must hit the MDS at some point but the situation is different than that of NAS. In NAS every request has to go through the NAS management server whereas in OSD the data itself never passes through the MDS. All data flow is directly between clients and OSDs. As explained above, once granted access to an object, the client may use the MDS-supplied credential (permission) across multiple accesses.

Here I have tried to throw some light on the concept of Object Storage Device. More information is available freely on the internet.