A
Adam Bertram
Guest
AI governance succeeds or fails at the data path rather than the model. A governed file transfer layer can help make paths defensible, from partner feeds to core banking systems.
Suppose a bank’s AI model approves a loan it should have flagged. In the review, data scientists defend the training set while the vendor defends the model, and everyone hunts for the failure in the math.
The real failure happened hours earlier, and in a file transfer nobody was watching. An unmonitored feed lands late or malformed and the model consumes whatever shows up: a credit bureau’s scheduled file, a fintech partner’s overnight batch or a nightly extract from the core banking system. The model didn’t fail; the data that reached it did, and no one can prove otherwise. If AI governance starts at the model, that blind spot sits outside it, and every feed you add widens it.
Beyond whether the AI is auditable, assess whether the data feeding it can be traced and reviewed as part of governance processes.
Onboarding the first fintech partner is easy: an SFTP connection, a shell script, and you move on when it works. Six months later, seven partners each deliver their own way—one over a REST API, another as CSV drops into an S3 bucket. That sprawl is now your data layer, and when an auditor asks how you govern those paths, the honest answer is tribal knowledge.
An auditor cares less that a feed is connected than that it is well-documented and traceable: can you show what it delivered, when it arrived, who could change it and where it was routed? Ad-hoc scripts are built for connectivity; provability was never a requirement any was written to meet.
Provability fails quietly, which makes it dangerous. A credit-bureau feed lands late one Tuesday, partially written and using an unexpected delimiter. The model runs anyway, scoring applications on incomplete data. Weeks later, an auditor asks for a chain of custody, and the only record is standard output (stdout) from a shell script nobody captured.
That uncaptured output may cause a gap in the decision record. The moment the auditor asks where data entered, ad-hoc scripts stop being an inconvenience and become a governance failure. The National Institute of Standards and Technology (NIST) AI Risk Management Framework frames AI risk as governable. Here the management question is concrete: who controlled the data path, and where did it go?
Answering it means replacing ad-hoc scripts with transfers you can govern.
Warning: If your AI governance program can describe how the model was trained but cannot reconstruct how yesterday’s input file arrived and who controlled its path, it may not provide sufficient visibility into an important governance layer.
The instinctive fix is to funnel every transfer through a central relay so the logs live in one place. Centralizing control is right; routing the data through a central hub is not. Sensitive financial data picks up an extra hop, and the whole network leans on one bottleneck.
Separating the control plane from the execution plane avoids that trade. Progress Automate MFT (Managed File Transfer) uses a cloud management console to help organizations define and monitor workflows, each saved as a task with source, destination, schedule and processing steps, while self-hosted agents, lightweight services behind your firewall, execute those tasks locally.
Because agents execute near the source, transfers move directly between endpoints without sending payload data through the console, while the console can provide monitoring and reporting for transfer activity. That is the part worth paying for: centralized visibility into transfer activity across hundreds of feeds. The agent initiates its management connection outbound, so keeping the console out of the data path costs no visibility.
“Near the source” is what lets one governance layer span more than partner feeds. A model scoring a loan rarely consumes third-party data alone: the same run pulls from branch systems and the core banking application. Put an agent beside each, and the console can provide centralized management and visibility across internal and external feeds. Govern only the partner boundary while a core banking extract still moves by unwatched script, and the model consumes ungoverned data one layer in.
The same split can help improve operational resilience while supporting audit and review activities. An agent pool—a group of agents the console load-balances across—routes retried work to another when one goes unresponsive, subject to the task’s retry behavior.
Centralizing control only helps if the console captures information relevant to audit and review processes. Four artifacts are worth checking before you trust a feed:
Two honest gaps are worth naming. Access control answers authorization and version history preserves prior task configuration and supports rollback. However, evidence that expires before the audit arrives is not evidence, so confirm your activity retention window against your audit lookback period before designing around it.
Those artifacts establish the configuration and execution history, they do not establish that the transferred data was complete for downstream use. A file lands late or arrives half-written. Adding explicit checks inside the task workflow before the destination step can help prevent downstream movement from being reported as successful. These checks could be native workflow conditions, custom PowerShell or an external validation.
A skeptic might ask whether a management console is really any better than an ad-hoc collection of scripts with a nicer interface. The difference is insight: a script has no versioned, access-controlled statement of expected behavior that predates the run, so nobody can compare intent against outcome afterward. Before you trust a model’s decision, you need to be able to demonstrate what data went into it and how that data got there.
In the loan review example, if the data feed were governed through the console, reviewers could trace the scoring run back to the exact file and timestamp used, then see how that file was delivered. Connecting those records still takes work, but it is worth doing before you need to investigate a problem.
So run the test that matters. Pick one feed and check whether you can show where it came from, who could change its path, what happened on the last exception and which configuration was live when it ran.
If you can answer those questions, you have a foundation for governance. If you cannot, fix the data path first. The model may be able to explain how it reached a decision. The harder question is often whether you can demonstrate which data it was given.
Request a Demo
Continue reading...
Suppose a bank’s AI model approves a loan it should have flagged. In the review, data scientists defend the training set while the vendor defends the model, and everyone hunts for the failure in the math.
The real failure happened hours earlier, and in a file transfer nobody was watching. An unmonitored feed lands late or malformed and the model consumes whatever shows up: a credit bureau’s scheduled file, a fintech partner’s overnight batch or a nightly extract from the core banking system. The model didn’t fail; the data that reached it did, and no one can prove otherwise. If AI governance starts at the model, that blind spot sits outside it, and every feed you add widens it.
Beyond whether the AI is auditable, assess whether the data feeding it can be traced and reviewed as part of governance processes.
Every Partner Is a New Path You Now Have to Govern
Onboarding the first fintech partner is easy: an SFTP connection, a shell script, and you move on when it works. Six months later, seven partners each deliver their own way—one over a REST API, another as CSV drops into an S3 bucket. That sprawl is now your data layer, and when an auditor asks how you govern those paths, the honest answer is tribal knowledge.
An auditor cares less that a feed is connected than that it is well-documented and traceable: can you show what it delivered, when it arrived, who could change it and where it was routed? Ad-hoc scripts are built for connectivity; provability was never a requirement any was written to meet.
The Failure You Won’t See Until the Auditor Asks
Provability fails quietly, which makes it dangerous. A credit-bureau feed lands late one Tuesday, partially written and using an unexpected delimiter. The model runs anyway, scoring applications on incomplete data. Weeks later, an auditor asks for a chain of custody, and the only record is standard output (stdout) from a shell script nobody captured.
That uncaptured output may cause a gap in the decision record. The moment the auditor asks where data entered, ad-hoc scripts stop being an inconvenience and become a governance failure. The National Institute of Standards and Technology (NIST) AI Risk Management Framework frames AI risk as governable. Here the management question is concrete: who controlled the data path, and where did it go?
Answering it means replacing ad-hoc scripts with transfers you can govern.
Warning: If your AI governance program can describe how the model was trained but cannot reconstruct how yesterday’s input file arrived and who controlled its path, it may not provide sufficient visibility into an important governance layer.
Centralize the Control, Distribute the Execution
The instinctive fix is to funnel every transfer through a central relay so the logs live in one place. Centralizing control is right; routing the data through a central hub is not. Sensitive financial data picks up an extra hop, and the whole network leans on one bottleneck.
Separating the control plane from the execution plane avoids that trade. Progress Automate MFT (Managed File Transfer) uses a cloud management console to help organizations define and monitor workflows, each saved as a task with source, destination, schedule and processing steps, while self-hosted agents, lightweight services behind your firewall, execute those tasks locally.
Because agents execute near the source, transfers move directly between endpoints without sending payload data through the console, while the console can provide monitoring and reporting for transfer activity. That is the part worth paying for: centralized visibility into transfer activity across hundreds of feeds. The agent initiates its management connection outbound, so keeping the console out of the data path costs no visibility.
“Near the source” is what lets one governance layer span more than partner feeds. A model scoring a loan rarely consumes third-party data alone: the same run pulls from branch systems and the core banking application. Put an agent beside each, and the console can provide centralized management and visibility across internal and external feeds. Govern only the partner boundary while a core banking extract still moves by unwatched script, and the model consumes ungoverned data one layer in.
The same split can help improve operational resilience while supporting audit and review activities. An agent pool—a group of agents the console load-balances across—routes retried work to another when one goes unresponsive, subject to the task’s retry behavior.
What the Console Has to Prove, Feed by Feed
Centralizing control only helps if the console captures information relevant to audit and review processes. Four artifacts are worth checking before you trust a feed:
- What ran, and when. Task and file-activity records show when a run occurred, whether it succeeded and how the transfer progressed. Paired with configured schedules, they answer the timeliness question a status screen cannot.
- Who could change it. Centralized policy and role-based access control helps organizations manage which users can view or change tasks and libraries, with PGP and SSH keys managed centrally.
- What the run itself checked. Configure each task to identify and respond to stale or incomplete input based on configured workflow logic and flag files that never arrive, with conditional steps deciding what happens next and notifications alerting the team.
- Which configuration was live. The console keeps a rolling history of up to 200 prior versions of every task and lets an administrator save up to 100 named milestones, so a broken feed rolls back to a configuration you can point an auditor at.
Two honest gaps are worth naming. Access control answers authorization and version history preserves prior task configuration and supports rollback. However, evidence that expires before the audit arrives is not evidence, so confirm your activity retention window against your audit lookback period before designing around it.
Those artifacts establish the configuration and execution history, they do not establish that the transferred data was complete for downstream use. A file lands late or arrives half-written. Adding explicit checks inside the task workflow before the destination step can help prevent downstream movement from being reported as successful. These checks could be native workflow conditions, custom PowerShell or an external validation.
Make the Data Path Auditable Before You Trust the Decision
A skeptic might ask whether a management console is really any better than an ad-hoc collection of scripts with a nicer interface. The difference is insight: a script has no versioned, access-controlled statement of expected behavior that predates the run, so nobody can compare intent against outcome afterward. Before you trust a model’s decision, you need to be able to demonstrate what data went into it and how that data got there.
In the loan review example, if the data feed were governed through the console, reviewers could trace the scoring run back to the exact file and timestamp used, then see how that file was delivered. Connecting those records still takes work, but it is worth doing before you need to investigate a problem.
So run the test that matters. Pick one feed and check whether you can show where it came from, who could change its path, what happened on the last exception and which configuration was live when it ran.
If you can answer those questions, you have a foundation for governance. If you cannot, fix the data path first. The model may be able to explain how it reached a decision. The harder question is often whether you can demonstrate which data it was given.
Request a Demo
Continue reading...