Application vitality is a key to long-term business success and developers are the gatekeepers. How can we help them be as productive as possible? There are ways to ensure developers focus less on lower value, time-consuming activities that are common in the AppDev environment and allow them to focus on coding. In “Celebrating Mainframe Programmers,” I explored some of the ways developers can automate necessary, but time-consuming tasks so they can focus on issues that require experimentation and problem solving.

The DevOps movement has addressed one of the biggest culprits: manual testing. Manual testing is essentially throw-away work that carries significant business risk and cost in terms of developer cycles (both hard cost and opportunity cost). The solution to these lost developer cycles is automation.

The “shift-left” mantra is all about empowering developers to automate testing to make themselves and their colleagues more productive over the long term. For a small upfront investment, the payoff accrues over the life of the application and, as we in mainframe know, that can be a very long time.

Taking a cue from hybrid cloud adoption, mainframe teams are now actively embracing unit test automation. Let’s review the opportunity from the most important perspective: the developer’s. Unless mainframe developers embrace automation, testing will continue to be a manual and risky process.

Unit Testing Options

The developer experience is critical to successful unit testing. Older tools attempted to bring test automation to mainframe development using awkward GUI-driven interfaces. Unsurprisingly, poor adoption followed because developers felt constrained by these pre-DevOps era tools designed to be used by testers and developers. The new generation of tools, like Test4z for mainframe, offers a developer-centric combination of code-based test automation combined with APIs and runtimes. Coding is what developers do best, and APIs and runtimes are familiar tools-of-the-trade.

New-to-mainframe talent is likely already familiar with similar distributed unit testing tools, such as JUnit, TestNG, and Pytest. As an architect recently said about Test4z, “Finally, a tool for the mainframe that works like the distributed world.”

Choose the IDE Developers Prefer

Developers are most productive and happy when they’re in their preferred development environment. While many know and love Visual Studio Code, others prefer Eclipse, ISPF, or other IDEs so the testing tool should accommodate all of these options. Developer adoption is job No. 1 so all developers need to be on board with unit testing.

Whether an IDE extension, Command Line Interface (CLI), or native 3270, the solution should fit into the developers’ preferred work environment and workflow.

Here’s a short demonstration of the developer experience using VS Code.

Unit Tests and Language Choice

Unit tests are typically written in the same programming language as the program being tested, which for most mainframe shops is COBOL.

Creating unit tests that are in the same programming language as the applications/programs provides the next generation of developers with knowledge of the code base. For example, these unit tests also would enable new talent to improve their COBOL skills.

Source code and unit tests also have a very symbiotic relationship. Including test files alongside source code makes managing and maintaining both easier. Unit tests are a form of living documentation. They provide insights into the expected behavior of individual components, making it easier for developers to understand and collaborate on the codebase. You wouldn’t expect a textbook written in English to have footnotes in Spanish. Unit test “documentation” should be the same language as the source code.

However, the decision regarding which language depends on many variables, including skills availability, performance, culture, etc., which is why unit testing tools should offer a choice.

What Code Should We Unit Test First?

With the tools and process in place, adoption can begin but with hundreds or thousands of applications and potentially millions of lines of code, where do you begin?

There’s no way around it, test automation requires effort, but capacity is limited. How do you roll out unit testing in an efficient way (i.e., without negatively impacting productivity in the short term)?  Several approaches address the capacity dilemma:

• Automate as-you-go: Adopt a policy that requires the changed section (or sections) to have an active unit test before promoting to production. The developer responsible for the code change could also be responsible for creating and passing the test(s). This approach will increase code coverage for applications that require changes/fixes (i.e., self-selecting adoption) without having a material impact on productivity. If a developer is changing code, they should be knowledgeable enough to create tests for the relevant sections (e.g., COBOL paragraphs).

• Business risk-based build-out: Most businesses would benefit from creating unit tests proactively for the most critical code (i.e., before a change is identified). Partner with product owners to determine which applications/sections would benefit most from unit test automation (i.e., faster changes, higher confidence). Those applications with significant business risk and relatively frequent updates should be the focus here (i.e., the low-hanging fruit).

• Risk-based build-out: Capture the “tribal knowledge” of soon-retiring staff by pairing them with next-gens, who are often already experienced with unit testing, to focus on applications within their responsibility. This win-win scenario enables the veterans to educate the next generation of talent while the next-gens create the tests.

• Mitigate risk with record/replay: For applications that change infrequently, mitigate unit test voids with regression tests using record/replay. This stopgap provides a simplistic way of determining if a subsequent change to the application has had an impact by recording test scenarios and then replaying them as needed with automatic validations. This technique is also valuable as a prerequisite for the application to deploy between dev/test, pre-prod, and prod environments. These tests can also be run, for example, before/after a compiler or middleware (e.g., CICS) version upgrade as an early warning system.

Culture Change and Developer Empowerment

Successful test automation, more than anything else, depends on culture change.

By empowering developers with familiar tools and languages in their day-to-day work environment, we open the door to what the father of the quality movement, Edwards Deming, referred to as “pride of workmanship.”

Let’s change the culture of mainframe application development thoughtfully and purposefully while keeping sight of the overall goal: greater code change confidence and application vitality.

Peter Wassel is the Director of Product Management, DevOps for Broadcom’s Mainframe Software Division. In this role, Peter partners with clients to realize the value of their investment in the Mainframe by leveraging modern development practices and open tooling, to further their innovation.