Building Agency Before Agents

Author perspective: Personal thought leadership; not an official policy statement by any employer or affiliated institution.

Abstract

Artificial intelligence is moving from answering questions to taking action. AI agents can retrieve information, use software tools, coordinate workflows, and perform increasingly complex tasks with limited intervention. That creates real opportunities to improve productivity, service, and institutional resilience. It also exposes a weakness that technology alone cannot repair: organizations may gain the ability to automate work before they possess the knowledge, judgment, authority, and learning practices necessary to delegate that work responsibly.

This paper argues for building agency before agents. Human agency is the capacity to understand, choose, question, act, and remain accountable. Institutional agency is the collective ability to sustain purpose, trustworthy knowledge, decision discipline, legitimate authority, and learning across people and time. The two are related but not interchangeable: a technically sophisticated organization may still weaken the agency of its employees, members, or citizens.

The paper proposes two working constructs: the Agency Gap, the mismatch between an organization's ability to deploy agentic technology and its capacity to direct it responsibly; and the Agency-First Maturity Model, a five-gate approach to readiness—Purpose, Knowledge, Judgment, Governance, and Bounded Delegation. These are proposed integrative frameworks, not validated standards or claims of inventing agency as a concept. They build upon established scholarship in organizational learning and human–automation interaction and align with published guidance from NIST, OECD, and ISO.

A case illustration from The Way and the Dang Genome examines knowledge stewardship under human authority. A public pension application shows how the argument translates into member service, administration modernization, and long-horizon institutional memory. The resulting position is not “never automate until everything is perfect.” It is more practical: experiment to learn; delegate consequential authority only to the extent the institution can explain, supervise, correct, and own what follows.

Executive Summary

There is a tempting equation in the current AI conversation: more capable agents will produce more capable organizations. Sometimes they will. But the relationship is neither automatic nor guaranteed.

An agent can find a policy, summarize it, and trigger a workflow. It cannot, merely by doing those things, establish that the policy is current, that the requester is entitled to rely on it, that an exception should be granted, or that the proposed action serves the institution's purpose. Those determinations require evidence, authority, context, and judgment. If they are weak before automation, they can remain weak afterward—only faster and at greater scale.

The Agency Gap is the distance between technical capacity and institutional readiness. It is visible when an organization has agents but cannot identify a reliable source of truth; has approvals but no genuine review; has process maps but cannot explain the tradeoffs embedded in them; or has dashboards that report completed tasks without showing whether outcomes improved.

Human and institutional agency require separate attention. Human agency preserves the ability to disagree, refuse, appeal, learn, and make consequential choices. Institutional agency enables many people to work from coherent purposes, governed knowledge, clear decision rights, and corrective feedback. Neither justifies paternalism: an institution's desire for consistency does not grant it the right to predetermine the choices of people it serves.

The Agency-First Maturity Model has five gates:

1. Purpose: Define the human and institutional outcome the work is meant to serve.
2. Knowledge: Establish authoritative, current, traceable, accessible information.
3. Judgment: Expose decision criteria, uncertainty, alternatives, and responsible owners.
4. Governance: Enforce permission, oversight, evaluation, escalation, and revocation.
5. Bounded Delegation: Grant the least autonomy needed, evaluate actual outcomes, and revise.

These are gates of sufficiency, not a linear procurement program. A low-risk prototype may begin immediately and help reveal which foundations are missing. A production agent changing a person's entitlement, moving money, modifying an official record, or making an irreversible external commitment should face a substantially higher threshold.

For executives, the practical shift is from asking “What can we automate?” to asking “What can we responsibly delegate, and what must become stronger in us first?” Success should be measured not only by time saved, but by decision quality, traceability, member or customer outcomes, correction time, employee expertise, and effective human control.

Research supports a balanced view. In a large customer-support setting, access to generative AI assistance increased issues resolved per hour by about 15% on average, with uneven effects across worker skill levels [9]. A separate randomized field experiment found measurable reductions in email time, but not in meeting time—an important reminder that individual productivity improvements do not automatically transform coordinated organizational work [15]. Research on automation and appropriate reliance shows why oversight design matters [6–8]. More recent work on agent autonomy and security documents both increasing independence and concrete attack surfaces [12–13].

This paper's proposal is therefore not to slow innovation for its own sake. It is to make innovation durable: human in the lead, AI as an extraordinary amplifier, and governance as the structure that allows trust to be earned.

1. The Inversion: Our Agents Are Advancing Faster Than Our Agency

Imagine a newly deployed agent assisting with the processing of retirement applications. It retrieves a member record, reads a policy document, compares service credits, drafts a response, and queues an administrative update. Every step appears efficient. Each has a successful technical execution log.

Now imagine that the policy it retrieved had been superseded. An experienced analyst would recognize the exception because the language changed after a prior court decision or board interpretation. The agent has no such context because that decision was never preserved in the repository it searched.

The workflow succeeded. The institution failed.

This is a hypothetical example, but it illustrates the central distinction in this paper. We tend to evaluate automation by whether the system can complete a task. We should also ask whether the organization can explain why the task is valid, who authorized it, which evidence governs it, and how an error would be discovered and repaired.

A traditional digital tool often waits for instruction. A modern agent can plan intermediate steps, call other applications, and adapt to changing information. The significant shift is not that software has suddenly acquired moral responsibility. It is that software can now execute a longer chain of actions within authority granted by people and organizations.

The consequences of a mistake vary by task. An inaccurate draft email is usually reversible. An incorrect statement of a benefit entitlement, an unauthorized financial transfer, or a change to a production data set can be harder to unwind. Autonomy and consequence should not be treated as synonyms, but their intersection determines the need for stronger controls.

Empirical evidence suggests that human relationships with agents are changing. Anthropic's February 2026 analysis of its own coding-agent and API usage found that more experienced users were more likely to permit automatic approval while also intervening more often during execution [13]. The study is vendor-specific, uses proprietary product settings and observational measures, and does not establish a universal trend. It does highlight a plausible direction: oversight may move from approving every action to managing boundaries and intervening at the right moment.

That requires a different kind of organizational competence.

Before building systems that act on our behalf, we must strengthen our ability to decide what they should be permitted to do.

2. Two Forms of Agency That Must Advance Together

2.1 Human agency is more than a signature

Human agency is the ability to understand the situation, evaluate alternatives, choose a course, and retain meaningful power to question and redirect it. It is not merely being designated the person who clicks “approve” at the end of an opaque AI process.

Consider an employee asked to approve a benefit determination produced by AI. If the employee cannot inspect the source provisions, cannot see the calculation steps, cannot understand the uncertainty, or faces a workload that makes review impossible, the approval may be procedurally human but substantively hollow.

A meaningful human role has at least five properties. The person can understand the relevant basis for action, challenge the proposed result, choose another legitimate course, intervene before material harm, and answer for the resulting decision within the authority of their role. Each property can be designed, tested, and supported. None is guaranteed by putting a person into a workflow diagram.

That distinction matters for people served by institutions as well. Members, patients, students, employees, and citizens need understandable explanations, appropriate opportunities to correct records, and access to human review when decisions materially affect them. An internal approval chain does not substitute for the agency of the affected person.

The OECD's updated AI Principles explicitly connect trustworthy AI to human-centred values, individual autonomy, and appropriate oversight [1]. The principle is not a blanket requirement for a human to check every trivial action. It is a requirement to preserve the conditions under which human choice and accountability remain meaningful.

2.2 Institutional agency is the capacity to remember, decide, and learn

Individuals make choices. Institutions must make decisions that survive changes in personnel, vendors, technology, and leadership.

I use institutional agency here to describe a connected set of organizational capabilities: continuity of purpose; usable knowledge; explicit judgment; legitimate decision rights; coordination; and the ability to learn from consequences. The term has broader uses in scholarship. The contribution proposed here is its application as an operational readiness lens for agentic AI, not a claim to have coined the phrase.

An institution with strong agency can answer questions that a sophisticated software inventory cannot:

• What are we trying to protect or improve?
• Which version of our understanding is authoritative, and why?
• Who can decide, recommend, authorize, override, and appeal?
• What was known at the time of a past decision, and what changed afterward?
• What evidence would cause us to revise a rule or stop an agent?

These capabilities are familiar in organizational learning. Nonaka's account of knowledge creation emphasizes interaction between tacit and explicit knowledge, including the role of organizations in articulating and amplifying what individuals know [4]. March examined the organizational tension between exploiting established knowledge and exploring new possibilities [5]. Neither study examined contemporary large-language-model agents. Both illuminate the institutional problem that agents now make more urgent.

Institutional knowledge is not just a collection of files. A file may show what was written without preserving why it was written, who challenged it, how a decision was reached, or whether it still applies. An organization can digitize every document and still lose its reasoning.

2.3 The relationship must not become paternalism

Strong institutional agency should make a person better equipped to choose—not simply more compliant with institutional expectations. Organizations need consistent rules and fiduciary controls, but those obligations do not authorize AI to define a person's values, suppress legitimate disagreement, or disguise policy judgments as objective calculations.

That is the most important link between the human and institutional forms of agency: the institution should be capable enough to support informed human freedom, not powerful enough to replace it.

3. What the Research Establishes—and What It Does Not

The ambition of this paper is integrative. Several ideas behind it are already well established; the proposed frameworks put them into an operational sequence for the agentic era.

3.1 AI can improve performance and help transfer expertise

Brynjolfsson, Li, and Raymond studied a generative AI assistant introduced to 5,172 customer-support agents and reported a 15% average gain in issues resolved per hour. Gains were especially pronounced among less experienced and lower-skilled workers, while more experienced workers showed smaller productivity gains and some quality tradeoffs [9]. This is meaningful evidence that assistance can capture and distribute useful practices. It is not proof that autonomous agents improve every workflow or that the same benefits will occur in pensions, healthcare, or government.

A cross-industry randomized field experiment involving approximately 6,000 knowledge workers found changes in time spent on email and some document tasks, but not a significant reduction in meeting time [15]. That distinction matters. Individual productivity can improve before the organization redesigns coordination, authority, or decision processes.

The goal should therefore be broader than replacing labor with faster software. A more durable result would be better-informed people and stronger institutional capability.

3.2 Automation can create dependency and misplaced reliance

Lisanne Bainbridge's 1983 examination of the ironies of automation described the difficulty of keeping people skilled and prepared for unusual conditions when routine operations are automated [6]. Parasuraman, Sheridan, and Wickens later provided a model for deciding which functions to automate and to what extent [7]. Lee and See examined appropriate reliance, arguing that trust must be calibrated to the system and its operating context [8].

These insights long predate generative AI. Their relevance has increased, not diminished. A fluent system can make a weak recommendation appear coherent. A fast workflow can leave too little time for checking. A highly reliable process can make rare failures harder for people to anticipate.

A 2025 survey of 319 knowledge workers found associations between high confidence in generative AI and lower self-reported critical-thinking effort [11]. Because the study is survey-based, it does not prove that AI causes cognitive decline or generalized deskilling. It nevertheless supports the need to design interaction around verification, challenge, and continued skill development.

3.3 AI capabilities are uneven, and threats cross trust boundaries

The jagged technological frontier research on consulting tasks showed that AI could improve performance on some assignments while harming results on others that appeared related [10]. Organizational readiness must therefore be task-specific. Success on a benchmark or a neighboring workflow does not establish reliability on a new, consequential process.

The security dimension is equally concrete. NIST has reported that indirect prompt injection can hijack agents by placing adversarial instructions in material they are supposed to read as data [12]. An agent that can act through enterprise applications becomes an attractive vehicle for crossing boundaries between untrusted content and privileged actions.

An access-control failure is not fixed by a model's eloquence. It requires architectural controls: least privilege, source trust classification, scoped tool permissions, approval gates, monitoring, and the ability to revoke access.

3.4 Existing governance frameworks provide a foundation

This proposal does not seek to replace established AI governance. NIST's AI Risk Management Framework organizes risk work around Govern, Map, Measure, and Manage, with governance applied throughout the system lifecycle [2]. Its Generative AI Profile identifies additional risks and management actions relevant to generative systems [16]. The OECD's principles articulate human-centred values, accountability, transparency, robustness, and oversight [1]. ISO/IEC 42001 specifies requirements for an organizational AI management system [3].

These frameworks provide substantive controls. What this paper adds is an executive sequencing question: What must an organization develop in itself before the capabilities of an AI system become delegated authority?

The answer cannot be determined by policy adoption alone. Readiness must be demonstrated in work.

4. The Agency Gap: A Proposed Diagnostic Framework

An institution's capacity to deploy AI and its capacity to govern AI may develop at different speeds. I call the resulting mismatch the Agency Gap.

As illustrated in Figure 1, the label is offered as a working construct in this paper. Similar phrases appear in other fields, and this paper does not claim exclusive ownership of the term. Nor is the gap a scientifically validated single-number index. Its value is diagnostic: it forces leaders to look at what technology amplifies and what it cannot supply on its own.

View full-size, accessible diagram ↗

Text description

Conceptual comparison of technical capabilities such as retrieving data, recommending, using tools, and executing; against institutional prerequisites including purpose, authoritative knowledge, judgment, decision rights, oversight, and correction. A not-equal symbol divides the two. No numeric comparison is implied.

Five dimensions deserve attention.

The purpose gap. The organization can specify a task but cannot say what human outcome the task is meant to serve. A member-service agent may optimize call closure while giving people incomplete understanding. Speed improves; service quality does not.

The knowledge gap. The agent can retrieve documents, but the institution has not established which sources govern, when they became effective, which exceptions apply, or how conflicting records should be reconciled. Retrieval is mistaken for authority.

The judgment gap. Important decision criteria are embedded in unwritten practices or scattered across individuals. An agent can repeat historical outcomes without knowing which tradeoffs were legitimate or which facts would justify an exception.

The authority gap. A workflow allows an agent to write to a system even though no accountable owner has clearly delegated the power to make that class of change. Technical access exceeds institutional permission.

The learning gap. The organization records tasks completed but not errors discovered, appeals upheld, reversals made, or changes in the rule base. Automation accelerates yesterday's assumptions into tomorrow's decisions.

These dimensions reinforce one another. A weak purpose makes it difficult to define quality. Weak knowledge obscures evidence. Weak judgment hides the reason for action. Weak authority makes it unclear who must intervene. Weak learning allows the same error to recur.

An organization can be strong in some dimensions and weak in others. A mature cybersecurity program does not automatically confer well-governed policy interpretation. A respected legal team cannot compensate for poor data lineage. Nor should a strong score in one low-risk workflow be generalized to all AI deployments.

Diagnostic implication: the Agency Gap should be assessed by workflow and authority scope, not declared as a single enterprise label.

A practical diagnostic can ask each process owner to produce five artifacts: a purpose statement, an authoritative source map, a decision and exception map, an authorization matrix, and an error-to-correction pathway. Their absence is not proof that automation will fail. It is evidence that the organization may not yet know how to recognize or own a failure.

5. The Foundations of Institutional Agency

5.1 Purpose: begin with a North Star

Technology strategy often begins with a capability inventory: what the software can do, which processes appear repetitive, and how much labor might be saved. Those are useful questions, but they come second.

A North Star tells an institution why it exists and what it must not sacrifice along the way. For a public retirement system, the enduring obligations include accurate administration, prudent stewardship, fair treatment under applicable rules, and public trust. The precise legal duties vary by jurisdiction and governing documents, but the central point holds: efficiency serves a purpose; it is not the purpose itself.

A meaningful North Star must be operational. It should help resolve competing priorities. If an AI application reduces average response time but increases incorrect member guidance, the institution must be able to say which result governs. If a system improves staff productivity while making members less able to understand or contest determinations, the outcome is not unambiguously better.

Purpose gives autonomy its legitimate direction. It should also define non-negotiables: privacy, nondiscrimination, security, authorized decision rights, and the dignity of those affected.

5.2 Knowledge: preserve the difference between information and authority

Organizations preserve enormous quantities of information. Far fewer can reliably identify which information should control a decision at a specific point in time.

A responsible knowledge architecture needs more than search. It needs provenance, versioning, effective dates, ownership, access controls, conflict handling, and mechanisms for correction. It must distinguish a source statement from an inference, an approved policy from a draft, a documented fact from an uncertain recollection, and a past decision from a rule applicable today.

Consider a retrieval system that finds three plausible descriptions of a benefit provision. One is a current administrative rule. Another is a superseded training slide. The third is a historical memorandum explaining why the rule changed. All three may be useful. Only one may govern the immediate determination, and even that may depend on dates and individual circumstances.

A retrieval score is not an authority decision. Embedding similarity is not legal validity. The fact that an agent can quote a document does not give the document permission to govern an action.

Knowledge readiness therefore means that the agent can be grounded in authorized information and the institution can explain why those sources were selected. The architecture must allow old knowledge to remain available for historical understanding without silently treating it as current instruction.

5.3 Judgment: preserve reasoning, not merely decisions

Many organizations keep records of what they decided. Fewer preserve enough context to understand why.

That is a significant weakness when decisions are transferred across leadership changes or automated through new technologies. An agent trained on prior outcomes may reproduce patterns without understanding the reasons that once justified them. Some patterns reflect sound judgment. Others reflect compromises, incomplete information, obsolete policies, or errors.

A durable decision record should answer a small number of consequential questions: What problem were we solving? What did we know? Which alternatives were credible? Which tradeoffs did we accept? Who had authority? What was uncertain? What would cause us to reconsider?

The point is not to turn everyday work into a compliance exercise. Routine decisions need routine controls. High-impact or difficult-to-reverse decisions deserve a more durable memory.

Judgment also requires separating decision quality from outcome quality. A well-reasoned decision can produce a disappointing result because important uncertainty was unavoidable. A poor process can occasionally succeed through luck. Learning requires examining what was reasonably knowable at the time, not rewriting the past with hindsight.

That distinction becomes essential when evaluating AI. We should examine both the decision process and the eventual outcome, with enough traceability to learn from each.

5.4 Governance: make trust operational

Governance is often treated as the obstacle that appears after innovation. Properly designed, it is what allows organizations to innovate with confidence.

NIST, OECD, and ISO provide established ways to frame risk, oversight, and continuous improvement [1–3]. Translating those ideas into agentic operations requires practical answers to questions such as these:

Who owns this use case? What information may the agent access? What may it modify? Which actions require approval? What external instructions are untrusted? How are outputs evaluated? Which failures trigger escalation? Who can suspend the agent, and how quickly? How does the institution notify and remediate affected people?

A policy document without technical enforcement can be ceremonial. Technical restrictions without legitimate business authority can be arbitrary. Effective governance combines written authority, real permissions, tests, operational monitoring, and a clear path to stop or correct action.

The ideal is not maximal friction. It is proportionate control: the safeguards become stronger as uncertainty, impact, and irreversibility increase.

5.5 Learning: close the loop from action to correction

The final foundation is the capacity to learn. An organization that deploys AI but cannot detect drift, evaluate complaints, and revise its knowledge base has increased execution capacity without building institutional intelligence.

Learning must be designed into the workflow. Capture sources and decisions. Observe outcomes. Compare expected and actual results. Preserve exceptions, corrections, and challenges. Update the authorized knowledge and controls. Reevaluate whether the agent's permissions are still justified.

The strongest organizations will not be the ones that never make an error. They will be the ones that make errors visible early, reduce the harm, learn from them, and do not conceal the history of how their understanding changed.

6. The Agency-First Maturity Model

The five foundations suggest a practical readiness model, illustrated in Figure 2. The Agency-First Maturity Model is proposed here as an original integrative tool for leaders. It has not yet been empirically validated. It should be tested against real workflows before anyone treats its stages or measures as benchmarks.

View full-size, accessible diagram ↗

Text description

Five readiness gates: Purpose (outcome and harm criteria), Knowledge (sources dates and permissions), Judgment (criteria exceptions owners), Governance (constraints audit interruption correction), and Bounded Delegation (scoped authority pilot evidence). Experiments may occur throughout, but consequential delegation requires relevant gates to be met and revisited.

These gates do not require five sequential projects. Purpose, knowledge, judgment, and governance can mature together. Early prototypes can expose gaps and inform each gate. The word before refers to the moment of consequential delegation, not a ban on AI-assisted learning.

Gate 1 — Purpose

Select a specific workflow and articulate the outcome in human terms. Name whose interests may be affected and what tradeoffs are unacceptable. Define conditions under which the organization should refuse to automate. If the use case cannot be explained without naming a product, the team may not yet understand the actual problem.

Gate 2 — Knowledge

Test whether authoritative sources can be located, distinguished, and updated. Do not rely solely on retrieval accuracy. Evaluate whether a superseded rule can be mistakenly cited as current, whether a draft can override approved guidance, and whether access permissions survive through retrieval and agent execution.

Gate 3 — Judgment

Map decisions by rule-based, discretionary, and consequential components. Ask domain experts to examine difficult cases. Identify where human discretion is legally required, where organizational policy has not resolved ambiguity, and where an agent should stop rather than improvise.

Gate 4 — Governance

Translate policy into enforceable boundaries. Separate the system's access to information from its permission to act. Test prompt-injection scenarios, unauthorized data requests, exception handling, approval failures, incomplete logs, and emergency suspension. An agent that behaves well in normal circumstances but cannot be controlled during abnormal ones is not ready for consequential autonomy.

Gate 5 — Bounded Delegation

Authorize only a defined action class with a named owner, measurable outcomes, and an expiration or review mechanism. Expand authority based on evidence from actual operations—not because a vendor added a feature or the model benchmark improved. Where an action cannot be readily reversed, require greater assurance before execution.

The model's principle: earn autonomy through demonstrated organizational competence, not technological optimism.

7. Designing Responsible Delegation

There is no universal level of autonomy appropriate for all AI systems. The same agent can be low risk when drafting a test message and high risk when sending binding communications or changing production records.

An agency-first organization considers at least six factors: the consequence of error; reversibility; uncertainty; affected people's rights; exposure of sensitive information; and the organization's ability to observe and interrupt the process.

The resulting delegation modes can be practical rather than ideological.

A high degree of task complexity does not automatically justify the last mode. Conversely, a routine action with reliable guardrails may need little step-by-step review. The appropriate mode is conditional on consequence and control.

Figure 3 makes the central permission boundary visible: humans and institutions retain legitimate decision rights and enforceable controls; the agent receives only a defined scope of action.

View full-size, accessible diagram ↗

Text description

Three layers: human authority sets purpose, values, consequential decisions, appeals, and stopping power; institutional controls provide trusted sources, access limits, evaluation, monitoring and audit; authorized agents may retrieve, recommend, execute within scope, and report exceptions. Actions in scope may execute with logs; out-of-scope actions must stop and escalate.

Two design rules follow.

First, separate capability from permission. An agent may technically be able to write a record, send a message, or call a financial API. That does not establish the legitimacy of doing so in a given situation. Authority must be granted by an accountable institution through enforceable mechanisms and constrained to the intended context.

Second, make oversight substantive. Human-in-the-loop is not sufficient if the human lacks the time, information, or authority to challenge the result. Oversight may take the form of preapproval, sampling, anomaly detection, intervention, appeals, or retrospective audit. Whatever the arrangement, it must be capable of affecting outcomes.

An important special case is the right to stop. Agent design should include safe interruption and suspension, especially when the task touches sensitive information or irreversible action. Where possible, systems should preserve a complete execution record and a tested rollback or remediation procedure.

The question is not whether to trust AI. It is how to make reliance appropriate to the task, evidence, and consequences—the problem described in longstanding automation research [8].

8. What We Should Measure

Most AI business cases begin with familiar measures: cost per transaction, time saved, number of tasks automated, and employee adoption. Those measures matter. They are insufficient.

If a pension system answers questions twice as fast but introduces more uncertainty about benefits, the productivity improvement may be real while the service outcome is worse. If an AI drafting assistant makes staff more efficient but gradually erodes the organization's ability to explain its own policies, it may create a hidden long-term liability.

An Agency-First Scorecard should balance at least five categories:

Some measurements should be numerical; others require structured expert review and feedback from affected people. An institution should resist turning human agency into a single numerical score. The quality of an appeal mechanism, for example, cannot be reduced to the number of appeals filed. Too few appeals may mean excellent service—or an inaccessible process.

For higher-risk applications, evaluation must test realistic cases, not just average-case demonstrations. Review ambiguous inputs, historical exceptions, conflicting sources, malicious instructions, incomplete information, and people whose circumstances do not fit the dominant pattern. Measure system behavior and human behavior together.

This changes the executive dashboard. The number of agents deployed is a technology inventory. The institution's ability to use them without losing understanding or control is a leadership outcome.

9. A Case Illustration: The Way and the Dang Genome

The ideas in this paper have also been shaped by a personal question: How do we preserve what experience has taught us without turning that experience into instructions that limit those who come after us?

That question became central to The Way, a body of writing and reflection concerned with passing forward a way of thinking. Its governing purpose is to leave future generations a stronger starting point and the freedom to go farther. The emphasis is deliberate. Wisdom should enlarge another person's capacity to judge; it should not prescribe the outcome of that judgment.

Over time, a collection of stories and writings led to a broader problem of knowledge stewardship. How should source material be preserved? How should a later correction relate to an earlier memory? What happens when AI identifies a plausible connection that no human source has endorsed? How can useful insights remain retrievable without confusing a private interpretation with public truth?

The emerging response has been an architecture referred to as Dang OS and the Dang Genome. At a high level, Dang OS is the learning and stewardship process; the Dang Genome is a governed body of memory, sources, relationships, decisions, and creations. The architectural ambition is to support a recurring cycle of capture, connection, judgment, action, observation, learning, and stewardship.

Figure 4 illustrates the learning loop: experience becomes evidence, evidence informs judgment, and consequences become material for future learning.

View full-size, accessible diagram ↗

Text description

Cycle: Capture evidence; Connect context; Judge choices; Act or Create; Observe consequences; Learn from results; Steward and pass forward; Repeat with better evidence and greater human capacity. A conceptual adaptation of the governed Dang OS learning loop, not a claim of empirical validation.

Several decisions are especially relevant to institutional AI.

Source and interpretation are kept separate. A historical record, a person's recollection, an AI inference, and an approved conclusion do not become equivalent simply because all four appear in a searchable system. This distinction resembles a basic requirement for institutional reasoning: know not only what has been found but what kind of evidence it represents.

Current authority is distinguishable from historical content. Older interpretations can be preserved for context without becoming current instructions. A retrieval result cannot make itself authoritative through relevance alone.

Approval is a human act. AI can assist with research, synthesis, drafting, challenge, and implementation. It cannot manufacture a person's consent, declare an unapproved belief canonical, or convert an inference into personal testimony.

Publication is a separate choice. Knowledge may be useful and accurate yet remain private. Preserving a source does not automatically confer permission to expose it. That boundary matters in every institution entrusted with sensitive information.

Corrections remain part of the record. Strong learning is not the appearance of having always been right. It is the ability to show how understanding changed and why.

These are observations about the project's documented design and governing intention, not evidence that it has already been validated as an enterprise AI implementation. Nor does a personal and family knowledge system provide a direct substitute for regulated institutional controls, independent audits, legal obligations, or production security engineering.

The transferable lesson is narrower and more useful: a knowledge system becomes more trustworthy when it knows what it knows, how it knows it, who may authorize changes, and what remains open to question.

That is a strong foundation for agents. It is also valuable even without them.

10. Why Public Pension Systems Make the Stakes Clear

Public pension systems offer a useful test of the argument because they operate over long horizons. Their obligations can outlast careers, vendors, leadership teams, software platforms, and administrative interpretations. An error may affect a benefit determination years after the original decision or data entry.

A small 2026 NCPERS survey illustrates the sector's caution. Among 33 respondents, 96% described human judgment as primary where AI was used, 63% cited a lack of internal AI evaluation or oversight expertise as a barrier, and 43% identified AI-related cybersecurity vulnerabilities as their leading risk concern [14]. These percentages reflect the respondents, not a representative estimate of all public pension systems. Yet they sharpen the question: What does it take to maintain human judgment when the systems beneath it become more autonomous?

Three illustrative applications show how an agency-first approach could work.

10.1 Member service: the difference between an answer and a determination

An agent may be very useful in helping a member locate forms, understand general benefit provisions, or prepare a question for staff. Those tasks can reduce friction and improve access to information.

But a personalized estimate or determination may depend on facts and provisions that the member cannot verify from a generic answer. Source effective dates, service history, eligibility exceptions, and authorized calculation rules matter.

The institution should therefore distinguish informational assistance from an official determination. An advisory agent can cite controlling material, disclose uncertainty, and guide a member to qualified assistance. A supervised process can prepare calculations and identify discrepancies. A consequential determination should follow the governing legal and administrative authority, with review and appeal channels appropriate to the decision.

Success is not merely whether the answer sounds clear. It is whether the member receives accurate, understandable, actionable information and can correct an error.

10.2 Administration modernization: automation must not hide migration risk

Modernization projects often require reconciliation across old and new data structures, benefit rules, interfaces, and operating procedures. AI may help identify mismatches, produce test cases, trace requirements, summarize defects, and accelerate documentation.

Those contributions can be significant. Yet the migration of an incorrect service history remains incorrect even if the reconciliation report is elegant. An agent that proposes a mapping cannot authorize an unreviewed change to pension data merely because its technical confidence is high.

An agency-first program would preserve source lineage, define acceptance criteria, document exceptions, retain independent testing, and establish owners for unresolved issues. AI could broaden the team's ability to see patterns and anomalies while accountable professionals retain authority over acceptance, deployment, and remediation.

The broader point applies to any institutional modernization: an agent can accelerate verification, but should not be allowed to replace the meaning of verification.

10.3 Institutional memory: protecting knowledge across decades

Retirement organizations, like many public agencies, depend upon experienced people who have lived through rule changes, litigation, system replacements, and exceptional member cases. Not all of that knowledge is captured in policy manuals.

A governed institutional memory could connect official documents, decisions, process changes, rationales, and correction histories so that successors have a better starting point. AI could help retrieve relevant evidence and surface apparent contradictions. Experienced staff could challenge, correct, and contribute what the sources do not explain.

This would be more than a chatbot trained on a document repository. It would require source stewardship, legal and records-management policies, role-based access, effective-date reasoning, and clear boundaries between historical explanation and current authority.

It would also change the purpose of knowledge capture. Instead of preserving the appearance of certainty, the institution would preserve enough context for future employees to exercise better judgment.

For a pension board or executive team, this is a compelling strategic question: what institutional understanding should survive the next major technology transition—and what must remain open for successors to reconsider?

11. The Counterargument: Shouldn't We Deploy Agents to Learn Faster?

Yes—often we should.

One of the strongest arguments against an agency-first position is that institutions cannot fully understand a new technology until they use it. Early pilots can uncover undocumented practices, reveal contradictions in official sources, and show where existing governance is more symbolic than effective. Requiring complete readiness before beginning any experiment would preserve ignorance.

The Agency-First Maturity Model should not become a new form of organizational paralysis. It should distinguish learning permission from operational authority.

A low-risk sandbox that works with synthetic data, or a read-only assistant used by trained staff, can help build knowledge and judgment. The governance threshold should match the limited consequences. At the other end, an agent that modifies a member's official record or communicates a binding determination must meet much stronger conditions.

Another objection is that humans are fallible. That is true. Human decisions can be inconsistent, biased, slow, and poorly documented. The argument here is not that human judgment is always superior to automation. It is that legitimate responsibility must remain located in accountable people and institutions, and that AI performance must be compared with realistic human baselines rather than an imagined perfect human.

A third objection is that sophisticated agents could eventually monitor themselves, detect errors, and recommend governance improvements. They may contribute materially to these tasks. But technical capacity to monitor a process does not establish authority to redefine its purpose, grant itself new permissions, or decide that a stakeholder's rights are expendable. Independent controls and human institutional legitimacy remain relevant, even when models become far more capable.

The refined thesis is therefore this:

Build agency before consequential delegation. Use bounded agents to help build that agency.

That is not a contradiction. It is a disciplined learning cycle.

12. A 90-Day Agenda for Executive Teams

Organizations do not need a multiyear transformation before taking a useful first step. They need one well-chosen workflow, a clear purpose, and a willingness to examine the work honestly.

Days 1–30: Discover what the institution actually knows

Choose a workflow that matters but is bounded enough to study. In a pension system, it might be responding to common member questions, reconciling a defined class of data exceptions, or locating approved policy guidance.

Map the entire process: whose outcome it serves, the decisions it contains, the authoritative sources, the people with decision rights, the exceptions, and the failure modes. Ask experienced staff where the written procedure differs from lived practice. Include input from those affected by the workflow when feasible and appropriate.

Produce five artifacts: a purpose and harm statement; a source-authority map; a decision-and-exception map; an authorization matrix; and a correction pathway. Do not automate undocumented ambiguity as though it were a rule.

Days 31–60: Design an agent that can be governed

Build a bounded prototype against authoritative or appropriately simulated data. Establish traceability from outputs to sources and from action requests to authorization. Create realistic evaluation cases, including conflicting documents, superseded guidance, missing evidence, unauthorized tool calls, and exceptions.

Develop meaningful oversight rather than a reflexive approval click. Define when a person must review, what information they see, how they challenge an output, and what causes automatic escalation or suspension. Consider the security risk of untrusted material reaching an agent with tool permissions.

Record the baseline human process so that later claims of improvement can be tested fairly.

Days 61–90: Run, observe, and decide what has been earned

Conduct a controlled pilot. Measure task quality, source accuracy, user understanding, exception handling, staff effort, and the ability to detect and reverse errors. Review not only average outcomes but also difficult cases and low-frequency failures.

Conclude with an evidence-based delegation decision: remain advisory, authorize supervised execution, permit bounded autonomy, or stop and repair the foundations. Name the owner who accepts the decision. Set a review date and clear conditions for expanding or withdrawing authority.

A successful 90-day pilot may produce a production system. It may instead produce something more valuable: a reliable map of what the institution must improve before deployment.

Neither outcome is a failure. The failure would be to deploy authority that the institution cannot understand or control.

13. The Deeper Opportunity: AI as Rocket Fuel for Human Intelligence

I see AI as an extraordinary amplifier of human capability. It can help us see connections across information that would otherwise remain fragmented. It can accelerate analysis, expose unfamiliar perspectives, and help small teams tackle work that once required much greater resources.

But amplification depends on what is being amplified.

If an institution has a clear purpose, credible knowledge, sound judgment, and accountability, AI may allow those strengths to reach further. If it has fragmented records, confused authority, and poor feedback, AI may amplify those weaknesses just as readily.

This is why the conversation about the frontier organization should extend beyond the technology itself. A frontier organization is not simply one that buys the newest tools or deploys the most agents. It is one that learns how to combine human judgment and machine capability without losing the ability to explain, correct, or choose.

The best future is not one in which people compete with agents to perform every task. Nor is it one in which people surrender judgment because machines can move faster.

It is a future in which machines extend what people can discover and accomplish, while institutions become more capable of remembering what matters, deciding responsibly, and learning from reality.

That future requires deliberate design. Human agency cannot be an afterthought added to an autonomous workflow. Institutional agency cannot be purchased as a product feature. Trust cannot be declared by a communications campaign. Each must be earned through practice.

Conclusion: What Should Become Stronger?

The defining opportunity of agentic AI is not merely that work can be performed with fewer manual steps. It is that people and institutions may gain an unprecedented ability to understand complex problems, preserve knowledge, test possibilities, and act with greater confidence.

That opportunity carries a responsibility.

We should ask whether an AI system leaves employees more capable of recognizing a good decision. Whether it leaves people better able to understand decisions that affect them. Whether it preserves the reasons behind institutional choices so future leaders can learn from them. Whether it makes errors easier to detect and correct. Whether it increases the freedom to reconsider rather than merely the speed of execution.

The strongest institutions will not be measured by how many agents they have deployed. They will be measured by whether those agents help the institution remember better, see more clearly, make better decisions, learn from consequences, and serve people more faithfully.

Before we build systems that can act on our behalf, we must strengthen our capacity to know what matters, judge what should be done, and remain responsible for what follows.

Build agency before agents.

Appendix A. Distinguishing Established Knowledge from Proposed Contributions

Appendix B. Executive Diagnostic: Ten Questions Before Delegating

1. What human or institutional outcome does this agent serve, and how would we recognize harm?
2. Who may decide to deploy it, authorize each kind of action, and suspend it?
3. Which sources are authoritative, by whom, and as of what effective date?
4. How will the system distinguish data, historical context, instructions, and untrusted content?
5. Which decisions require professional discretion or legal authority rather than mechanical execution?
6. What will the reviewer see, and can that person genuinely challenge the result?
7. What can the agent change, disclose, commit, or transmit—and how are those permissions technically enforced?
8. How will people affected by a decision understand, correct, or appeal the outcome?
9. What real-world tests, difficult cases, and baseline comparisons justify deployment?
10. How will an error become a corrected institutional understanding rather than a repeated mistake?

No single score answers these questions. The purpose is to expose the boundaries that must be made explicit before autonomy expands.

References

[1] Organisation for Economic Co-operation and Development (OECD). OECD AI Principles (adopted 2019; updated 2024). https://oecd.ai/en/ai-principles

[2] National Institute of Standards and Technology (NIST). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1 (2023). https://www.nist.gov/itl/ai-risk-management-framework

[3] International Organization for Standardization / International Electrotechnical Commission. ISO/IEC 42001:2023, Information technology—Artificial intelligence—Management system (2023). https://www.iso.org/standard/81230.html

[4] Nonaka, I. (1994). “A Dynamic Theory of Organizational Knowledge Creation.” Organization Science, 5(1), 14–37. https://doi.org/10.1287/orsc.5.1.14

[5] March, J. G. (1991). “Exploration and Exploitation in Organizational Learning.” Organization Science, 2(1), 71–87. https://doi.org/10.1287/orsc.2.1.71

[6] Bainbridge, L. (1983). “Ironies of Automation.” Automatica, 19(6), 775–779. https://doi.org/10.1016/0005-1098(83)90046-8

[7] Parasuraman, R., Sheridan, T. B., and Wickens, C. D. (2000). “A Model for Types and Levels of Human Interaction with Automation.” IEEE Transactions on Systems, Man, and Cybernetics—Part A, 30(3), 286–297. https://doi.org/10.1109/3468.844354

[8] Lee, J. D., and See, K. A. (2004). “Trust in Automation: Designing for Appropriate Reliance.” Human Factors, 46(1), 50–80. https://doi.org/10.1518/hfes.46.1.50_30392

[9] Brynjolfsson, E., Li, D., and Raymond, L. (2025). “Generative AI at Work.” The Quarterly Journal of Economics, 140(2), 889–942. https://doi.org/10.1093/qje/qjae044

[10] Dell'Acqua, F., et al. (2023; revised 2026). “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality.” Working paper. https://doi.org/10.2139/ssrn.4573321

[11] Lee, H.-P., et al. (2025). “The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers.” Proceedings of CHI 2025. https://doi.org/10.1145/3706598.3713778

[12] NIST Center for AI Standards and Innovation. “Strengthening AI Agent Hijacking Evaluations” (January 17, 2025). https://www.nist.gov/news-events/news/2025/01/technical-blog-strengthening-ai-agent-hijacking-evaluations

[13] Anthropic. “Measuring AI Agent Autonomy in Practice” (February 18, 2026). Vendor observational research, not a representative cross-platform study. https://www.anthropic.com/research/measuring-agent-autonomy

[14] National Conference on Public Employee Retirement Systems (NCPERS). AI Adoption and Strategy in Public Pension Administration (August 2026), preliminary online survey of 33 respondents. https://www.ncpers.org/ai-adoption-and-strategy-in-public-pension-administration

[15] Dillon, E., Jaffe, S., Immorlica, N., and Stanton, C. “Shifting Work Patterns with Generative AI” (Microsoft Research, April 2025). https://www.microsoft.com/en-us/research/publication/shifting-work-patterns-with-generative-ai/

[16] NIST. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1 (2024). https://www.nist.gov/itl/ai-risk-management-framework