The last couple of weeks I've been traveling for work, and I spent a long flight reading some publications I had saved up about mainframe and Fibre Channel. I will talk about some good sources of information later, but for now know that FICON runs over Fibre Channel, and indeed a lot of the reasons behind why Fibre Channel is known to be really reliable is that it more or less came out of IBMs requirements for accessing mainframe disk storage.
Enterprise disk storage has this annoying issue that the disk arrays are:
- Heavy
- Power-hungry
- Ridiculously expensive
An example: DS6800 is widely known to be an unreliable platform, and I would certainly never run a mainframe on it based on my experiences repairing mine. However it is as small as disk arrays get, but it is still 50 kg empty, and consumes 500W of power!
However the DS6800 is kind of a black sheep in reliability. What if we look at something bigger and better? The big brother DS8000 is known to be rock solid for example. The downside is of course that it weighs a bit more than one metric ton and consumes 6 kW of power. Not great for hobbyists.
Ideally I would like to have something that allows me to use the power efficiency of x86, the speed of SSD, at the cost of reliability. I do not need 99.999% uptime, I'm OK to use a conventional server, and I think fellow hobbyists would agree with me.
What we need is a storage system that runs in software, using the economies of scale to our advantage. That's why I started Fikonfarm. The goal is to implement Fibre Channel and FICON in software and somehow hook the mainframe to it. The keyword being "somehow".
I mentioned before that I have 2x Brocade 7800 switches. They are like a conventional Fibre Channel switch, but some of the ports are internally routed to an FPGA that converts the FC traffic (FICON or SCSI, doesn't matter) to TCP/IP. The idea is to figure out how the protocol they talk works so I can make Fikonfarm look like one of those switches.
That brings me to what I've been up to the last couple of weekends. Just that - figuring out the Brocade 7800 protocol. I've come so far to be able to establish an FCIP tunnel with the switch and keep it up. I can also receive FC traffic, as well as send some very very basic frames. Nothing usable yet, but it's pretty exciting :-).
 |
| Figure 1: Wireshark dissector |
What you see in Figure 1 is the Wireshark dissector plugin for the Brocade 7800 protocol that I wrote to be able to map my thesis to real traffic. While I could implement a thesis in Fikonfarm straight away and see what happens, it is useful to be able to take a working session dump and apply the thesis on straight away. That way the outliers are visible, and it is harder to overfit a thesis. E.g. if a byte is 0x10 somewhere in the protocol - you could assume it is static, but it could also be e.g. the hour of the day. Without an extensive dump to test against it is hard to see what values never change, which do, and which seem interesting.
Anyway. When I've come so far as to be able to send and receive frames somewhat reliably I will write up the protocol details and hopefully contribute the dissector upstream. For now it's available on Github if you want it, you can try it on this PCAP if you're curious. Of course, any help is always welcome :-).
Now then, where would one go to learn more about these things? I have two sources I can recommend.
Brocade Reference
 |
| Figure 2: Brocade's reference for mainframe connecitivty |
The only downside I can think of is that it is only talking about Brocade solutions when discussing implementations. Keep that salt handy when it speaks about how good Brocade is compared to other vendors.
You can download the book legally for free here.
INCITS Standard Documents
I was about to add the number of pages in "Brocade Mainframe Connectivity Solutions" as a con, but given that my second recommendation is the standard documents themselves which are 10x as long, that seemed silly.The Fibre Channel standard is divided in many many substandards. The owner of these standards have changed over the passage of time, so it can be hard to figure out what the documents are called. Right now it seems INCITS is the main owner of the standard however.
Here are some good standards to look at:
- FC-SB-3 - this is classical FICON (before zHPF entered the game)
- FC-FC-5 - this is the layer over the physical media, it explains how FC is framed and assembled
- FC-SW-6 - explains how FC switches connect to eachother
- FCP-4 - covers SCSI for FC
The list goes on, but that will give you an idea of how many standards there are. The number in the end is the version of the standard and while the latest probably gives you the most details about the protocol, it can also omit references to obsolete features that may well be in use in system that you would like to integrate with. FC-SW-6 for example contains 4 fields that are not described more than that they have been obsoleted in previous standards.
Another issue is that a single PDF of these standards, e.g. FC-SB-3, costs $60 to get. If you look at the Brocade Standard Compliance page you will find a lot of references to various versions of these standards.
However, there is another way. One trick I wish to share is that in a lot of cases you can search for the INCITS number as well as "public review" (e.g. "incits 545-2019 public review"). This will in the majority of cases give you a drafted version of the standard that was close to being finalized. It is nothing to base a commercial product on, but it is absolutely fine to use to study the behavior of the protocols as well as e.g. maintaining the Wireshark dissectors.
Summary
I certainly have a lot of work in front of me to build Fikonfarm and getting mainframe DASD up and running without extra hardware. I expect this to take a few years realistically, which means this will not block my normal mainframe tinkering - it is merely my long-term strategy to allow myself to not have to care about broken disk controllers, and focus on what I enjoy: unraveling the mysteries of the mainframe.
Thanks for reading,
https://storage.googleapis.com/bluecmd/fcip-reset-w-target.pcap
Comments
Post a Comment