Contact Us

Day: June 30, 2026

When NOT to Break the Monolith: 5 Signs You Should Modernize In-Place

Posted on by Richa Arya

A first-person field report from a mainframe-to-Java engagement in early 2026. The prompts, the false starts, the human-in-the-loop guardrails, and the real productivity multiplier (it is not 10x).

Between January and April 2026, our team used current-generation large language models to help refactor roughly 200,000 lines of legacy code into Java for a property and casualty insurer. This is what actually happened, not the keynote version. The productivity multiplier is real, but it is not 10x. The failure modes are specific. The human-in-the-loop pattern that worked is repeatable. We compare what we saw to IBM’s published research on watsonx Code Assistant for Z and to the peer-reviewed paper “GenAI-Driven Migration of Legacy COBOL Applications to Clean Java Code” (ICSBT 2025).

GenAI-assisted refactoring delivered roughly a three-times productivity gain over our 2023 baseline on this engagement, well short of the ten-times figure vendors quote. The wins came from explanation, test generation, and boilerplate translation. The breaks came from implicit COBOL semantics (REDEFINES, packed decimal, level-88 conditions), copybook drift, and over-eager refactoring of business rules.

The pattern that worked was simple. The model proposed, the engineer verified, and the test suite enforced, in that order.

Why This Report Matters in 2026

Three things converged in late 2025 and early 2026 that make GenAI-assisted mainframe modernization a real option rather than a slide. Frontier models reached the context-window and reasoning depth needed to handle COBOL plus its copybooks in one conversation. IBM’s watsonx Code Assistant for Z moved from pilot reports to documented enterprise rollouts. And the underlying business pressure increased: cyber insurance carriers are repricing legacy-exposure premiums, and the labor market for COBOL maintainers continues to shrink as the workforce retires.

The result is that boards now ask a different question. Not “can we modernize the mainframe?” but “why have we not started yet, given the tooling has changed?” This field report is meant to answer that question honestly, including where the tooling still falls short.

The Setup

A property and casualty insurer. Policy administration system on IBM z/OS, roughly 200,000 lines of COBOL across 340 programs and 1,100 copybooks, a mix of batch and CICS. Target architecture: Java 21 on AWS with Aurora PostgreSQL. The hard requirement was retaining business semantics exactly. The system is regulated under state insurance commissioners and SOX. Timeline: 14 weeks.

Team composition: six engineers (four modernization, two COBOL subject-matter experts), two QA engineers, and one architect. We used current Anthropic and OpenAI frontier models for translation and verification, and a local open-weight model for code paths touching personally identifiable information.

What Worked

  1. Explanation first, translation second. The model’s best output was plain-English summaries of what each COBOL program did. A COBOL subject-matter expert validated the explanation before any Java was requested. This single step caught roughly four in five misinterpretations before code was written.
  2. Test generation from observed behavior. We fed the model the COBOL plus sample inputs and asked for characterization tests in JUnit. Those tests became the gate the Java had to pass. The productivity gain on test writing alone was easily six to eight times faster than hand-rolling tests.
  3. Boilerplate translation. File I/O, simple MOVE and COMPUTE statements, paragraph-to-method conversion. Clean, repetitive work the model does very well. Roughly seven in ten lines of the codebase fell into this bucket.
  4. Copybook normalization. Generating consistent Java records from variant copybooks (same logical field, different physical layouts). Tedious by hand, fast for the model.
  5. Documentation generation. Every translated module shipped with model-generated JavaDoc traceable back to the COBOL paragraph it came from. Auditors loved this and it cut their review time noticeably.

What Broke

  1. REDEFINES and packed decimal. COBOL’s memory aliasing and decimal arithmetic have no clean Java analog. The model produced subtly wrong field interpretations a meaningful percentage of the time. Characterization tests caught these, but only because we wrote the tests first.
  2. Level-88 condition names. Named boolean conditions were translated literally instead of as enums or domain objects. The code worked but it was unreadable. This required a second pass with an explicit refactor-to-domain-enums prompt.
  3. Implicit business rules. The most expensive failure: the model helpfully simplified a 40-year-old underwriting rule that looked redundant but actually encoded a state regulation. A 30-year subject-matter expert caught it in UAT. The lesson: never let the model refactor business logic without explicit human approval, rule by rule.
  4. Copybook drift across programs. Same copybook name, different actual contents in different programs (a real legacy mess). The model assumed consistency. We built a pre-processing step that canonicalized copybooks before any model call.
  5. Context window limits at scale. Even very large context windows cannot hold a full COBOL program with all its copybooks and called subprograms. We built a dependency-graph chunker. Without it, the model hallucinated the missing context. The ICSBT 2025 paper on GenAI-driven COBOL migration reports the same failure mode independently.

The Human-in-the-Loop Pattern That Worked

After two failed approaches (model as engineer, model as pair programmer) we landed on model as proposer. The flow:

  1. An engineer scopes a unit of work, typically one program or one paragraph.
  2. The model explains the COBOL in plain English. The subject-matter expert validates.
  3. The model generates characterization tests from the COBOL plus sample data. The engineer reviews.
  4. The model proposes the Java translation. The engineer reviews and edits.
  5. Tests run. Failures go back to step four with a diff and the failing test output.
  6. On pass, a second human code-reviews and merges.

The single most important rule: The model never wrote the final commit message and never approved a pull request. Two humans touched every change, and that is what made the output auditable for state insurance regulators and SOX.

The Actual Numbers

These figures are measured on this engagement only. We share them as a directional read, not as a benchmark to copy. IBM’s own published case studies on watsonx Code Assistant for Z report productivity gains in a similar range when the human-in-the-loop discipline is held.

Metric: Lines translated per engineer-week | 2023 baseline: 1,100 | 2026 with GenAI: 3,800 | Multiplier: 3.4x

Metric: Defect rate per 1,000 lines | 2023 baseline: 4.2 | 2026 with GenAI: 3.1 | Multiplier: 1.4x better

Metric: Test coverage at first pull request | 2023 baseline: 38% | 2026 with GenAI: 82% | Multiplier: 2.2x

Metric: Time per business-rule validation | 2023 baseline: Same | 2026 with GenAI: Same | Multiplier: Unchanged, this is human-bound

Metric: Model API cost per refactored file | 2023 baseline: $0 | 2026 with GenAI: About $0.40 | Multiplier: New line item

Common Mistakes Teams Make with GenAI on Legacy Code

  • Treating the model as an engineer instead of a proposer. The role split (model proposes, human verifies, tests enforce) is what makes the output auditable. Skipping it usually shows up first as a quiet defect, then as a regulator finding.
  • Skipping characterization tests. Without a test suite generated from observed behavior, you cannot tell whether the new Java matches the old COBOL on the inputs the business actually sends. Hand-validating thousands of paragraphs is not a real plan.
  • Letting the model refactor business rules unattended. The most expensive defects we have seen all came from helpful simplifications of rules that encoded a state regulation or a decades-old underwriting decision. Business-rule refactors need a named human approver, rule by rule.
  • Ignoring copybook drift. Same copybook name in two programs does not mean the same record layout. Canonicalize copybooks before any model call, or budget for the rework.
  • No prompt or response logging. State insurance regulators, SOX auditors, and internal risk all expect a reconstructible trail. Adding logging after the fact is much harder than turning it on day one.
  • Over-indexing on the productivity number. The headline ten-times figure from vendor decks is not what teams measure in practice. Three to four times is realistic with the discipline above. Anyone budgeting against ten times will miss their dates.

Frequently Asked Questions

Why not full automation?

State insurance regulators and SOX require attributable human review. Beyond that, the implicit-business-rule failure mode alone would have shipped a defective product. Full automation is the wrong target for this work in 2026.

Which models worked best?

Current frontier models from Anthropic and OpenAI for translation and verification respectively, with a local open-weight model for code paths touching policyholder personally identifiable information. The specific model names matter less than the role each one played in the pipeline.

What about IBM watsonx Code Assistant for Z?

Strong for in-place modernization where you stay on Z and add Java. We were exiting the platform entirely, so it did not fit this engagement. If you are staying on Z, it is worth a serious look. IBM’s published pilot reports are linked below.

Could a smaller team do this?

Below four engineers and one subject-matter expert, no. The pattern needs review capacity. Above eight people, coordination overhead starts to eat the gain. The sweet spot is in between.

Is this safe under SEC cybersecurity rules?

Yes, with logging. We logged every model call, every prompt, every response, and every human decision. That trail is what made the work auditable. Without the log, the same engineering work would not have cleared internal audit.

References

SOC 2, HIPAA, and SEC Cyber Rules: A 2026 Modernization Compliance Checklist

Posted on by Richa Arya

The compliance work most modernization plans underbudget. What to design in from day one so the audit, the breach disclosure, and the BAA do not blow up your timeline. 

Three regulatory regimes hit almost every modernization program: SOC 2 (customer trust), HIPAA (anything healthcare-adjacent, with a proposed Security Rule update from HHS in December 2024), and the SEC cybersecurity disclosure rule (Item 1.05 of Form 8-K, four business days from materiality determination). Designed in from day one, they add modestly to a program. Bolted on at the end, they add a lot and slip the timeline. This is the checklist for designing them in.

Five things built into a modernization from day one turn the audit into a paperwork exercise instead of a re-architecture. One, tenant isolation and least-privilege IAM. Two, end-to-end audit logging with tamper-evident storage. Three, data classification plus encryption at rest with a managed KMS. Four, a documented incident response runbook tied to the SEC four-business-day clock. Five, a BAA-ready vendor inventory if PHI is anywhere in scope. Skipping any one of them tends to mean rebuilding it later, under audit pressure.

Why This Matters More in 2026

Compliance has moved from a downstream audit step to a board-level metric. Three pressures are responsible. First, the SEC cybersecurity rule is now in its second full filing cycle, and material vendors to public companies are being pulled in through contractual disclosure clauses. Second, cyber insurance carriers in the US have continued to tighten underwriting on legacy stacks and unencrypted PHI, with several major carriers now requiring documented MFA and 12-month log retention as a condition of renewal. Third, customer procurement teams are asking for SOC 2 Type II reports earlier in the sales cycle and increasingly rejecting Type I or expired reports.

The net effect on modernization programs is that compliance evidence is no longer an end-of-program artifact. It is a continuous output, and it has to be designed in.

What Changed in 2024 and 2025

  • SEC cybersecurity rule. Final rule adopted July 2023, in force for fiscal year 2024 filings. Item 1.05 of Form 8-K requires disclosure of material cybersecurity incidents within four business days of materiality determination. SEC guidance in 2024 from the Division of Corporation Finance clarified the line between Item 1.05 (material) and voluntary Item 8.01 disclosures.
  • HIPAA Security Rule NPRM. HHS Office for Civil Rights issued a Notice of Proposed Rulemaking on December 27, 2024 (published in the Federal Register January 6, 2025). The proposed rule would make many previously addressable safeguards required, including encryption of ePHI at rest and in transit, multi-factor authentication, mandatory vulnerability scanning every six months and penetration testing annually, and 72-hour restoration objectives for systems holding ePHI. This is the first major Security Rule update since 2013.
  • NIST Cybersecurity Framework 2.0. Finalized February 2024. Added the GOVERN function, expanded supply chain risk, and is now the most common control mapping anchor for SOC 2 and SEC programs.
  • PCI DSS 4.0. Fully in force as of March 31, 2025. Adds explicit MFA requirements for all access into the cardholder data environment.

The Three Regimes in One Line Each

Regime: SOC 2 Type II | Who it applies to: Any B2B vendor whose customers ask for it. | What it really tests: Controls operated effectively over a 6 to 12 month window.

Regime: HIPAA / HITECH | Who it applies to: Anyone who creates, receives, or stores PHI for a covered entity. | What it really tests: Safeguards, BAAs, and breach notification within 60 days. Mandatory MFA and encryption likely under the 2024 NPRM.

Regime: SEC Cybersecurity Disclosure Rule | Who it applies to: U.S. public companies and material vendors to them. | What it really tests: Four-business-day disclosure of material incidents on Form 8-K Item 1.05.

The 5-Item Design-In Checklist

  1. Tenant isolation and least-privilege IAM. Logical isolation at minimum. For HIPAA workloads, prefer fully separate accounts or projects. Roles are time-bound, MFA-enforced, and reviewed monthly. Document who can read what.
  2. End-to-end audit logging, tamper-evident. Every privileged action, every data access, and every configuration change. Stored in a write-once bucket with object lock. Retain for seven years for SOX-adjacent workloads, six for HIPAA, and at least one for SOC 2.
  3. Data classification and encryption. Classify before you migrate (Public, Internal, Confidential, Restricted or PHI). Encrypt at rest with a managed KMS, with keys rotated on a defined schedule. Customer-managed keys where the contract requires them. TLS 1.3 in transit. Plan for the HIPAA NPRM’s mandatory-encryption posture even before it is finalized.
  4. Incident response runbook tied to the four-business-day clock. The SEC clock starts at materiality determination, not at detection. The runbook must define who determines materiality, in what time window, and how the Form 8-K language gets drafted. Run a tabletop quarterly.
  5. BAA-ready vendor inventory. Every subprocessor that could touch PHI needs a Business Associate Agreement. Build the inventory during modernization. Adding it later means re-papering every contract under audit pressure.

Where Modernization Programs Get Caught

  • Logging added at the end. By the time the auditor asks, the events have already happened and were not captured. Cost: re-instrument and lose three to six months of audit window.
  • Shared service accounts. The classic app_admin account used by 14 people. Fails every SOC 2 access review. Fix it at design time with per-human and per-service principals.
  • PHI in non-production. Test data scrubbed inconsistently across environments. Either tokenize at the source or budget for synthetic data generation as a real project line item.
  • Materiality undefined. Engineers detect an incident, lawyers argue materiality for a week, and the four-business-day clock has already run. Define materiality before the next incident, not during it.
  • Subprocessor sprawl. Modernization typically brings in eight to fifteen new SaaS tools. Every one needs a vendor risk review before it touches regulated data, not after.

The Cost of Bolt-On Versus Design-In

Directional ranges from our engagements. The wider the bolt-on, the worse the audit outcome tends to be. IBM’s Cost of a Data Breach 2025 helps anchor the downside: $4.44 million globally, $10.22 million in the United States, $7.42 million in healthcare.

Where the budget goes wrong: Most modernization business cases include security tools but not the compliance evidence work. Collecting, storing, and presenting evidence is roughly 60 percent of the audit cost, and it is the line item teams routinely forget.

Approach: Design-in from day one | Added program cost: 6 to 9% | Timeline impact: Neutral | Audit outcome: Clean opinion, low exceptions.

Approach: Mid-program retrofit | Added program cost: 12 to 18% | Timeline impact: 2 to 4 months | Audit outcome: Conditional opinion likely.

Approach: Bolt-on at the end | Added program cost: 22 to 40% | Timeline impact: 6 to 12 months | Audit outcome: Qualified opinion or re-audit.

How This Played Out in the Field

We worked with a manufacturer that had to lift its security posture under customer audit pressure. The program combined a Zero Trust rollout, identity hardening, and a documented control library mapped to NIST CSF. The result was a clean third-party audit and a reusable evidence pipeline. We also operate a 24×7 SOC with SIEM for a European financial firm and a unified Microsoft Sentinel deployment for an energy provider, both built to make this kind of evidence-on-tap the default rather than the scramble. All three case studies are linked below.

Common Mistakes Teams Make on Compliance During Modernization

  • Treating compliance as a separate workstream. If it runs parallel to the engineering work, it lands late. The control owners need to sit in the same architecture reviews as the engineers.
  • Assuming the cloud provider’s shared responsibility covers more than it does. AWS, Azure, and GCP secure the platform; you still own configuration, IAM, encryption choices, and evidence. Audit findings almost always sit on the customer side of the line.
  • Underbudgeting evidence collection. Roughly 60 percent of audit cost is collecting, organizing, and presenting evidence. Tools alone do not close that gap; an evidence pipeline does.
  • Letting incident response drift away from the SEC clock. Tabletops that exercise detection but not the materiality decision and disclosure drafting will fail the first real incident.
  • Choosing tooling before mapping controls. The control framework (NIST CSF 2.0 is the common anchor) decides the tools, not the other way around. Tool-led programs over-buy and still miss controls.
  • Skipping the BAA backlog. Every SaaS added during modernization that could touch PHI needs a BAA. Discovering an un-papered subprocessor during audit is a common, avoidable finding.

Frequently Asked Questions

Does SOC 2 require specific tools?

No. SOC 2 requires controls that operate effectively. Tools are means, not ends. In practice a SIEM, an IAM platform, and configuration management are all but necessary, but the framework itself names none of them.

Does the SEC cybersecurity rule apply to private companies?

Directly, no. Indirectly, yes. Material vendors to public companies get pulled in through vendor risk programs and contractual disclosure clauses. If you sell to public companies, plan for it.

Can we use the same controls for SOC 2, HIPAA, and SEC?

Mostly yes. One control framework, mapped to each regime. NIST CSF 2.0 is the most common anchor for that mapping today, and most auditors are comfortable with it.

Is the HIPAA NPRM final yet?

As of mid-2026, the December 2024 NPRM has not been finalized. Plan for its posture anyway. Building toward encryption, MFA, and 72-hour restoration is good hygiene regardless of finalization timing.

Who owns this work, security or modernization?

Both. Compliance is a shared design constraint. If only security owns it, it gets bolted on. If only modernization owns it, it gets skipped. Joint ownership with a single accountable executive is the only model we have seen work.

References

Thank you for contacting us. Our team will contact you shortly.