Why a Checklist
Mainframe source code reveals business rules, account structures, security controls, and operational procedures. In a regulated organization it deserves the same governance as any other sensitive technical asset, and an AI initiative does not lower that bar.
The questions below are the ones that come up in nearly every enterprise security review of an analysis tool. The answer to each should be a document or a control you can test, not a reassurance.
The Checklist
Where does the service run?
Ask for the deployment diagram. For a private SaaS model, confirm the application runs in your cloud tenant and identify every network path out of it.
Which AI models are used, and who approves them?
You should be able to select or approve the models. Read the provider terms for prompts and responses, and confirm in writing whether your data may be used for training.
What data is in scope?
List the source, configuration, and test artifacts that will be analyzed. Test data in particular may contain production-like personal information and needs an explicit decision.
How is data encrypted, and who holds the keys?
Confirm encryption in transit and at rest. Establish whether the keys belong to you, the vendor, or the cloud provider, and what that means at contract exit.
Who can access the environment?
Require single sign-on and role-based access. Ask how vendor staff access the system, whether that access is logged, and whether it requires your approval each time.
How are generated findings governed?
Diagrams, summaries, and dependency maps can expose system structure even without source code. Apply the same access, export, and retention rules to them as to the source.
What is logged, and for how long?
Ask for the audit event list: access, configuration changes, analysis runs, exports. Confirm log retention and who can read the logs.
How is data deleted?
Cover source, indexes, prompts, model responses, and outputs. Ask what evidence of deletion you receive when an evaluation or contract ends.
Who does what in an incident?
Agree notification responsibilities and timelines before the first file is uploaded, not after something goes wrong.
Can findings be traced for an auditor?
Each finding should reference the source it came from. If it cannot be traced, it cannot be defended in an audit or a regulatory review.
How to Use It
Treat unanswered items as open risks, not as blockers to be argued away. A vendor who can answer all ten with documentation has done the work. One who answers with adjectives has not.
No checklist makes a design universally secure. Its purpose is to give your team enough control and evidence to evaluate the service against your own security, privacy, and regulatory requirements.
Bring a real application question
Work with us to evaluate how Comprehend can support a defined mainframe application or application family.
Explore the Comprehend Design Partner Program