By Frank Kyne, Technical Consultant, Watson and Walker

System Management Facility (SMF) has been with us for as long as most of us can remember. No matter what your job is, no one that works with z/OS can imagine life without SMF. Whether you are responsible for performance, availability, security, storage management, CICS, DB2, MQ, network, WebSphere or any other role, there is at least one SMF record type to help you. Perhaps an even bigger testament to SMF’s key role is the number of non-IBM products that create SMF records. Now, there are more tools to process SMF records than ever before.

For as long as we have had SMF, we have had the venerable SYS1.MANx data sets to act as the repository for the newly created SMF records. However, much as we love them, they have their drawbacks, which can include:

• All record types from each system are sent to a single MANx data set, which is not ideal for security in these data privacy-sensitive times.
• They use “old VSAM,” not Media Manager. This means that they can't benefit from more recent performance enhancements such as striping.
• They are a finite size and you can only have so many of them, meaning that it is possible for them to fill, at which point you are facing the loss of SMF records.
• The maximum buffer size is 1 GB for all SMF records at one time, which is often not large enough for today's volume of activity.

To address these shortcomings, IBM introduced the ability to send SMF records to System Logger log streams rather than the SYS1.MANx data sets in z/OS 1.9. The first incarnation of SMF Logger support delivered scalability that could not even be imagined with the old SYS1.MANx data sets. However, the manageability fell short, resulting in many installations saying “thanks, but no thanks.” Those shortcomings were addressed with a combination of PTFs and new z/OS releases. SMF logstream mode quickly provided manageability that was at least equivalent to dataset mode, as the use of SYS1.MANx data sets is now known.

Even though a lot of discussion of logstream mode has focused on its superior performance compared to dataset mode, I believe that logstream mode offers benefits to installations of all sizes, including:

• Massively improved scalability. You can have as many log streams as you wish, compared to a single SYS1.MAN data set.
• Flexibility. You can create log streams based on the team that will use the records in that log stream, or you could run in SYS1.MANx-imitation mode and direct all records for each system to a single DASDONLY log stream. The SMF world is your oyster.
• SMF data availability. Because you can split the SMF records across many log streams and each log stream can have up to 2 GB of buffers, the chances of all SMF buffers filling and records being discarded is tiny in logstream mode compared to dataset mode.
• Critical record protection. There is an old saying: “Just because you're not paranoid, doesn't mean that everyone isn't out to get you.” If you subscribe to that outlook, you could direct critical record types to multiple log streams. For example, you could send your RACF SMF records to two log streams, each in a different, failure-isolated CF, and place the offload data sets in separate storage groups, thereby ensuring that no single failure could ever result in the loss of those critical SMF records.
• SMF record compression. SMF in z/OS V2 exploits the new zEDC capability to compress SMF records before they are written to the log stream. This requires that you have a zEC12 or later, with zEDC cards installed. However, the cards are cheap and customer experience to date has been that they achieve compression ratios between 8 and 10 to 1. Given the volume of SMF data that many systems create, it can result in significant savings. SMF record compression with zEDC is only available when SMF is in logstream mode. If you currently compress your CICS or DB2 SMF records (as many sites do), you could turn off that compression (and save CPU time in CICS and DB2) and use zEDC instead. This is likely to give you double the compression that you are achieving today but with nearly none of the CPU cost.
• SMF record signing. This is a new capability delivered in z/OS 2.2. The objective is to provide the ability to identify SMF records that have been tampered with. This is a concept that didn't exist when I first entered the field, but it is a sad reality in today's world. This feature is also only available if SMF is running in logstream mode.

SMF logstream mode has been available since 2007 — more than eight years now. I think it is now safe to say that it is not a fad that will pass. Quite the opposite — it is obvious that future enhancements to SMF are going to be limited to SMF in logstream mode. Cheryl Watson and I recently presented to a user group conference in Germany, and she asked how many of the attendees are running in logstream mode. Ninety percent of the attendees raised their hands. That many customers can't be wrong, so it's time to step up and make the move. You'll be glad you did.

Frank Kyne, a technical consultant at Watson and Walker, is the editor of Cheryl Watson’s Tuning Letter and works with Watson on classes and consulting engagements. Prior to Watson and Walker, Kyne was a project leader in IBM’s Redbooks organization where he was responsible for Redbooks about Parallel Sysplex, availability and GDPS.