Skip to main content

System z on contemporary zLinux



IBM System z supports a handful of operating systems; z/VM, z/VSE, z/OS, z/TPF, and finally zLinux. All the earlier mentioned OSes are proprietary except for zLinux which is simply Linux with a fancy z in the name. zLinux is the term used to describe a Linux distribution compiled for S390 (31 bit) or S390X (64 bit). As we are talking about modern mainframes I will not be discussing S390, only S390X.

There is a comfortable amount of distributions that support S390X - more or less all of the popular distributions do. In this list we find distributions like Debian, Ubuntu, Gentoo, Fedora, and RHEL. Noticeably Arch is missing but then again they only have an official port for x86-64.

This is great - this means that we could download the latest Ubuntu, boot the DVD, and be up and running in no time, right? Well, sadly no. The devil is, as always, in the details. When compiling high level code like C/C++/Go the compiler needs to select an instruction set to use for the compiled binary. Every new CPU generation released adds more instructions, and retains the previous instructions added by the previous generation. Side note: this means that to get the best performance out of your system you should recompile everything to target that particular instruction set of your CPU - this is what Gentoo is so known for. This targeting of a particular CPU instruction set is commonly referred to as ISA level (or sometimes ABI level).

For x86-64 platforms you commonly use the lowest level ISA that was defined when the 64 bit architecture was created. This means that even an old AMD Opteron 64 could run the latest Debian release. For 32-bit x86 platforms you can run on most of the older CPUs as long as they have the i686 ISA, although Fedora is thinking of deprecating support for pre-SSE2.

To recap: For a distribution to be usable on a computer we need to be at or above the ISA they are compiling for.

Oldest model supportable

We can tell that Debian uses the ISA level equivalent to a z196 from this email thread. That email references another email thread that makes the case that other compilers than GCC have been ported and are simply just not capable of emitting instructions that is compatible with a lower ISA level than z196. This makes sense if you think about it. IBM has noticeably increased their bet on zLinux over the years and they would have contributed patches to projects using an ISA that gives them the most bang for the buck. The oldest supported platform is z196, so it all fits neatly together as far as motive goes.

I am not a fan of planned obsolescence but there is not much choice for the maintainers here. I understand that customers buy System z because it is a reliable system with a massive support contract securing the investment. When the particular system is no longer supported, you buy the latest one on a new support contract. This means that there are most likely only a few generations of System z in production use. This survey somewhat backs this claim up.
Our research suggests that, traditionally, users upgrade on a regular basis to the most recent hardware to take advantage of capacity increases and cost benefits. 
This means that the corporate funds available for support zLinux is centered around supporting these current platforms. Compare that with x86-64 which has a more diverse ecosystem and massive interest in supporting many many generations of platforms.

So now we know the ISA of Debian, and given the reason behind the particular ISA it is unlikely that any distribution will ever support a lower one. We will soon see however that sadly there are many distributions that have chosen to bump the ISA even higher. I have tried finding a reason as to why one would abandon a currently supported model (z196 is still supported as of this writing) but the folks I have talked to have not been able to cite a convincing reason as to why.

What are the ISA changes anyway?

To not come of as rant-y I want to make sure I am not missing anything revolutionary that would explain why the ISA needs to be bumped. We already know the reason why z196 is the minimum ISA feasible, namely that some compilers do not support the earlier ISA levels, so what has been added in the higher levels?

IBM has good documentation on what the ISA levels mean, this is what it says:
  • Level 9 - 2817-xxx (IBM zEnterprise® 196 (z196)) and 2818-xxx (IBM zEnterprise 114 (z114))
    • High-word facility
    • Interlocked-access facility
    • Load/store-on-condition facility
    • Distinct-operands-facility
    • Population-count facility.
  • Level 10 (default in z/OS 2.3) - 2827-xxx (IBM zEnterprise EC12 (zEC12)) and 2828-xxx (IBM zEnterprise BC12 (zBC12))
    • Execution-hint facility
    • Load-and-trap facility
    • Miscellaneous-instruction-extension facility
    • Transactional-execution facility.
  • Level 11 - 2964-xxx (IBM z13® (z13)) and the 2965-xxx (IBM z13s (z13s))
    • Vector facility
    • Decimal floating point packed conversion facility
    • Load/store-on-condition facility 2
  • Level 12 - 3906-xxx (IBM z14)
    • Vector enhancement facility 1
    • Vector packed decimal facility
    • Miscellaneous instruction extension facility 2
There are undoubtedly some nice things in here but I am not convinced the performance gain of these extra features will do anything for the common system. You probably want your main workload compiled with the latest architecture level, but do you really gain much by having e.g. coreutils compiled with it? Is it worth alienating the hobbyist community over pushing the ISA level higher and higher?

Hercules, the freely available and open-source System z emulator, supports running z196 level code without any issues. For the zBC12 facilities there are some that are currently not implemented making the case even weaker for bumping the ISA level higher.

Models supported by zLinux distributions

Armed with the knowledge about the feasible minimum ISA level and what ISA changes give us, we can look at what the various distributions compile against and thus what models they support.

As of this writing (2019-05-22) these are what the latest released version of various distributions support in terms of IBM System z model.

Distro Minimum model
Debianz196[1]
Gentoo (stage3)z196 (unconfirmed)
UbuntuzEC12[2]
FedorazEC12[3]
CentOSDiscontinued[4]
RHELz13 (unconfirmed)
Alpinez196[5]
SLESzEC12[6]

The move to make zEC12 the minimum model does rhyme pretty well with what IBM is doing in general for their mainframe operating systems. If you look at z/VM 7.1 and z/OS 2.3 you will see that both of those releases require zEC12 or above to run. I cannot help but to wonder if IBM is pushing distribution maintainers to adopt zEC12 as a minimum ISA level to push customers to newer platforms? I really hope I am wrong though.

What about z114 in particular?

As I have an z114 in transit I am interested on what the z114 can run. Looking at the table above and the documentation for the distributions, I can choose between the latest version of Debian, Gentoo, and Alpine. I can also run the previous versions of RHEL (version 7) and SLES (version 12) if I feel like getting some enterprise feeling going. Over all it is not a bad list, but I am slightly worried about the bumping of ISA level without any publicly stated reason. I wish I will be able to run my z114 for at least another decade with an up to date Linux, but maybe that is hoping for too much.

Closing words

I take solace in knowing that there will likely be at least one zLinux distribution that runs on my z114 for years to come. Even if the mainline kernel for some reason drops support for z114 running a system with a couple of years old kernel is pretty common in the real world. For example SLES 12 and RHEL 7 has support until 2024 in general and for extended contracts seems like they extend to 2027 and afterwards. At that time the z114 would be 13 years old, which is pretty good.

It seems like my love for running the latest software version is just not compatible with wanting to run on older hardware. I will have to learn to like slower releases and long maintenance contracts. That does rhyme pretty well with mainframes though, and to be honest it sounds pretty relaxing.

Comments

Popular posts from this blog

Buying an IBM Mainframe

I bought an IBM mainframe for personal use. I am doing this for learning and figuring out how it works. If you are curious about what goes into this process, I hope this post will interest you.

I am not the first one by far to do something like this. There are some people on the internet that I know have their own personal mainframes, and I have drawn inspiration from each and every one of them. You should follow them if you are interested in these things:
@connorkrukosky@sebastian_wind@faultywarrior@kevinbowling1 This post is about buying an IBM z114 mainframe (picture 1) but should translate well to any of the IBM mainframes from z9 to z14.

What to expect of the process Buying a mainframe takes time. I never spent so much time on a purchase before. In fact - I purchased my first apartment with probably less planning and research. Compared to buying an apartment you have no guard rails. You are left to your own devices to ensure the state of whatever thing you are buying as it likely…

Powering a mainframe

The last few days have been eventful. I was contacted by the datacenter that the mainframe's cage is now ready for moving in, and the power has been made available. Very exciting! I grabbed my home-made power cables (more on that later) and my best screwdrivers and set off to the datacenter.


The datacenter staff, not needing a forklift in their day-to-day, had managed to solicit the services of a forklift, the associated operator, and some very handy folks to help navigate the mainframe from the storage space to its final location.



After some intense period of fighting the inclination of the road between the storage facility and the cage (and a door that was a bit too small) it was finally in place. Incidentally we were forced to trust the wheels on this pretty rough floor. I did not expect it to roll that well on raw concrete, I was pleasantly surprised. This thing is a tank!

Now, everybody wanted to see if it was working. My machine did not come with a power cable so I had to so…

Open Datacenter Hardware - Leopard Server

Introduction The Leopard is an OpenRack v1 compliant 12V server commissioned by Facebook to offer compute power. It consists of 2x Intel Xeon E5-2678 v3 and is available with either DDR3 or DDR4 memory. The model is manufactured by two vendors primarily: Quanta and Wiwynn.

Leopard features a 24x PCIe slot which can fit either a PCIe card with low profile, or a riser card with 1x 16x and 1x 8x slots. The server also supports a 3.5" SATA drive as well as either an mSATA or an M.2 drive mounted on the motherboard.

Connectivity wise Leopard has a mezzanine card slot allowing for example 10Gb/s or 25Gb/s Ethernet.

Figure 1 and figure 2 shows the server layout. The server is made to fit inside an OpenRack v1 enclosure, at which point it looks something like figure 3. Due to power constraints an OpenRack v1 can fit 30 of these servers before power starts to become an issue. The Leopard servers that the organization Serverfarm Upstate provides are all fitted with 256GiB DDR3 RAM and 2x …