“Indirectly.”
“Geography?”
“Yes.”
“Language?”
“Preference settings.”
She nodded.
“Can a branch employee override it?”
“Yes.”
“Do they?”
“Rarely.”
“Why?”
“The system is highly accurate.”
There it was.
Authority.
Different costume.
She asked:
“What is accuracy measured against?”
The executive frowned.
“Customer completion rates.”
“What does completion mean?”
“The transaction is successfully completed through recommended channel.”
“Does the model measure whether the customer felt pressured into that channel?”
“No.”
“Does it measure whether someone abandoned the attempt and went elsewhere?”
“Some attrition.”
“Does it measure whether the customer asked for a human and was redirected?”
The executive hesitated.
“Not consistently.”
The granddaughter wrote one sentence in her notebook.
A system can be accurate at predicting what people tolerate and still be wrong about what they prefer.
That became a research agenda.
The commission ordered studies.
Results were uncomfortable.
Customers labeled “digital-first” were indeed more likely to complete digital transactions.
But lower-income customers were also more likely to accept self-service after being told human assistance would take longer.
Was that preference?
Or adaptation?
Elderly customers with limited digital literacy sometimes abandoned transactions entirely.
Small-business owners in lower-income neighborhoods were less likely to receive proactive banker outreach because their short-term balances looked less promising, even when transaction patterns suggested growing businesses.
No explicit racial variable.
Still, geography and wealth history carried patterns.
The granddaughter thought of the branch manager’s words decades earlier.
Spend human minutes where the money is.
The algorithm had learned a cleaner version.
No insult.
No smirk.
No navy suit.
Just probabilities.
One commission member said:
“Isn’t this simply rational resource allocation?”
The granddaughter replied:
“Maybe.”
The room paused.
She continued.
“The question is not whether allocation is rational.”
“The question is what goal we told the system to optimize.”
“If we optimize immediate revenue per human minute, it will rationally deprioritize people whose future value, vulnerability, or complexity is not captured.”
“Then the unfairness may be perfectly rational relative to the wrong objective.”
That became one of the commission’s key findings.
Not biased machine.
Misdefined success.
The same lesson as her grandfather’s bank.
Metrics were never neutral merely because they were mathematical.
They encoded what someone chose to reward.
The commission recommended several changes.
Customers could request human assistance without hidden service penalties.
Banks had to audit whether channel-prioritization models produced systematically different access to human help.
Small-business opportunity models had to be reviewed for under-identification in historically underserved areas.
Fraud flags could trigger verification but not automatic accusatory treatment.
And the most controversial recommendation:
Banks should distinguish customer profitability from customer service entitlement.
A customer might be unprofitable.