Ok, as promised, I'm turning iSCSI Notes into an article. This will reference a couple of extra articles within this FAQ as I did not want to have the one big document but instead chose some smaller ones. Following are links to some of those articles so you don't have to read through this.
- Build an iSCSI target - /index.php?action=artikel&cat=7&id=164&artlang=en
What is iSCSI? Well, if you got here, you probably already know the answer, but I'll give my own definition. iSCSI is a way of creating network attached storage. Basically, your "hard drives" on one or more computers appear to be local, yet they are all on another server someplace. This is not basic file sharing or anything, your computer actually thinks it has those drives physically inside it. You treat those disk(s) just like ones inside your computer; partition, format, etc... The advantage is you have excellent control over the capabilities of those "drives." The "server" (called a target) generally does nothing but take stuff off a disk and send it to your client machine (called an initiator). If you wish to increase the size of a disk, you can do it. If you need a disk to be available to more than one machine, you can do it.
- Consolidation - All your storage can be concentrated in one or more special purpose machines. I have had the situation several times where one server was almost full, yet the one right next to it was almost empty. However, I could not remove the drives from the empty one and install them on the full one, so I had to go purchase new drives to help the full one. With iSCSI (or any kind of Network Attached Storage), you allocate as much space as each device needs, secure that any extra space can be allocated where needed very rapidly.
- Disaster Recovery - We use a lot of Xen servers, and our DOMU's (virtualized machines) use the iSCSI as their storage. If one of our DOM0's (the actual physical server) develops a problem, we simply bring up its virtual machines on another physical server while we do repairs. Additionally, using something like DRBD, we can have multiple iSCSI servers mirroring the same data so if one of the iSCSI servers goes down, we can bring everything back up from one of the mirrors. Actually, using Hearbeat, this can be automatic.
- Maintenance - Even using Debian Linux, you still have to do maintenance and upgrades, and occassionally those upgrades will cause problems. The same rules apply from the Disaster Recovery above; you can move your Xen DOMU's to another server, then perform maintenance on the DOM0. Or, if you are using DRBD and Hearbeat, you can "fail" one of the servers (the other takes over), then do your maintenance, then bring it back up.
The bottom line is, using iSCSI, Xen, DRBD and Heartbeat you can have a system where losing a physical computer, or performing maintenance on a physical computer can be done with no downtime.
Disclaimer here. I use Debian Linux, and this article assumes that. If you are using any of the other, excellent distributions, you will need to adapt. I use Debian because it is probably the most bullet proof system available (and, no, I won't argue with the BSD people out there). Debian is generally behind the curve on new hardware, new software, things like that. And they absolutely are a total pain when installing on a system requiring proprietary drivers. But the result is a server that I forgot about which ran for 3 years with no maintenance or anything and did not get cracked, and did not go down (I do NOT advise that). That is why so many other distributions use Debian as their base (ie, they are Debian derivatives).
Ok, enough prostelyzing. On to setting up and using iSCSI.
iSCSI uses its own terminology, so first I'll do some definitions, then a brief explanation.
|ttarget||the iSCSI target is the server, the thing that holds all of the data. It is the "target" of all of the other machines|
|initiator||These are the consumers of the space offered by the target. The clients. They "initiate" a connection to the target, mount the devices, then add/remove data from them.|
|LUN||A "Logical Unit Number" (vs a physical unit), this is a unique identifier for an export from the target. Bottom line is, when your target exports devices, each one must have a unique LUN. See https://en.wikipedia.org/wiki/Logical_unit_number for a good definition and a history of what this was used for.|
|iqn||An "ISCSI Qualtified Name", this is a unique identifier for the target machine. So, to get to an export, you combine the iqn and (in theory) the LUN, though there are better ways to do it.|
What Packages to use
I'm using iet for the target, and open-iscsi for the initiator, all running on Debian Wheezy. Eventually, I intend to do an High Availability iSCSI cluster. I have read that iet will be going away, but am still using it for now.
The target name is defined to be the letters iqn (iSCSI Qualified Name) followed by the year/month (yyyy-mm), followed by the servers reversed domain name. Each field is separated by a period (dot).
Thus, for a machine built in Jan 2012, whose machine name is joe.example.com, the iqn would be:
Note, the machine name (joe) is not included in this.
After the iqn, a colon and a unique identifier for the export is used. So, to get the target "media" (which is a "share" exported by the target machine iqn.2012-01.com.example, you put a colon and the name media.
This must be a globally unique name; in theory it should only exist one place in the planet. Of course, you just need to make sure it is "global" in what your network can see.
Once you have set up your target, you can set up your intiator(s) to use the services. They will use the iqn (with share) to make this connection
How can I dynamically configure?
Use ietadm, which is a tool for managing IET software. You can perform the following operations to a running target.
Adding a target:
ietadm --op new --tid=[id] --params Name=iqn.foo.bar:baz
[id] must be unused. You can get a list of the currently used Target IDs by: cat /proc/net/iet/sessions
Adding a LUN:
Note: This adds a LUN to a pre-existing target.
ietadm --op new --tid=[id] --lun=[lun] --params Path=/path/to/exported/file,Type=fileio
[id] must be an already existing Target ID.
[lun] is the LUN id, typically 0 (zero) for the first LUN.
HA iSCSI Cluster
Why build it yourself if you don't have to. There are appliance distributions available to allow you to create a full blown iSCSI target in an hour (vs several). I chose to not do this for my own reasons, probably more related to being a control freak than anything else. However, they are reported to work very well in many environments. I did have a problem where I lost my configuration with one of these during an upgrade (don't remember which one), but that was many years ago.
They all (as far as I know) offer Windows File Sharing, NFS file sharing and a nice web based configuration utility.
- openmediavault - http://openmediavault.org/
- freenas - http://www.freenas.org/
- openfiler - http://www.openfiler.com/