Educational Technology and Change Journal

By Jim Shimabukuro (assisted by Copilot)
Editor

Overall picture

As of 22 February 2026, there is no single, unified “grand” hierarchical developmental theory that simultaneously and symmetrically models how humans learn to work with AI, how AI learns to work with humans, and how AIs learn to work with other AIs. What we do have is a patchwork of emerging, partly hierarchical frameworks: (a) stage‑based models of human–AI teaming and human learning with AI; (b) hierarchical and curriculum‑based training schemes for AI systems adapting to humans; and (c) hierarchical multi‑agent reinforcement learning and curricula for AI–AI cooperation. Together, they form the beginnings of a developmental science of human–AI–AI collaboration, but they are still fragmented, domain‑specific, and rarely integrated into a single cross‑species developmental theory.

1. Humans learning to work with AI: emerging stage and literacy frameworks

On the human side, the clearest hierarchical developmental work is “Developing teamwork: transitioning between stages in human–agent collaboration,” which explicitly adapts Tuckman’s classic forming–storming–norming–performing model to human–AI teams.¹ The authors treat human–AI collaboration as a staged process in which teams move from initial orientation and role clarification (forming), through conflict and negotiation around capabilities and expectations (storming), toward stabilized patterns of coordination (norming), and finally to fluent, high‑performance joint activity (performing). This is not a lifespan developmental theory of individuals, but it is a hierarchical, stage‑based account of how collaboration quality develops over time in human–AI teams, with empirically grounded indicators of transition between stages.¹ Its strength is that it connects a well‑known human team‑development model to concrete design and evaluation of human–AI teaming; its weakness is that it focuses on team‑level dynamics rather than individual cognitive development, and it does not explicitly model how AI itself develops across these stages.

A complementary line of work comes from AI literacy and metacognition in education. Atchley and colleagues, in “Human and AI collaboration in the higher education environment: opportunities and concerns,” argue that effective human–AI collaboration in learning depends on metacognitive knowledge and skills—students must learn to plan, monitor, and evaluate when and how to use AI tools, and this metacognitive sophistication can be scaffolded in stages as curricula evolve.² Hutson and Plate’s chapter “Human‑AI Collaboration for Smart Education: Reframing Applied Learning to Support Metacognition” similarly frames human–AI collaboration as a progression from basic tool use toward more reflective, co‑regulative learning, where learners increasingly understand AI’s strengths, limitations, and biases and adjust their strategies accordingly.³ A related conceptual piece on “Reconceptualizing AI Literacy: The Importance of Metacognitive Thinking” explicitly proposes that metacognitive thinking should be central to AI literacy, implying a developmental trajectory from naïve, unreflective use of AI to sophisticated, self‑regulated collaboration with AI systems.⁴ These works collectively sketch a hierarchical progression in human capabilities—from low metacognitive awareness and uncritical reliance on AI, through intermediate stages of guided use and reflection, toward expert‑level, strategic collaboration. Their strength is that they connect cognitive and educational theory to concrete instructional design; their weakness is that they remain more conceptual than formal, and they do not yet specify detailed stage models with validated developmental assessments.

Another strand focuses on organizational mentoring as a developmental scaffold. Lin and Chen’s “Mentoring for effective human‑AI collaboration: an integrated theoretical framework” integrates Affordance Actualization Theory, Event System Theory, and transformational/transactional leadership theories to describe how mentors can guide employees through stages of awareness, experimentation, appropriation, and routinization of AI tools.⁵ While not framed as a strict stage theory, the model is implicitly hierarchical: individuals move from perceiving AI affordances, to experimenting with them, to integrating them into work practices, with different mentoring styles being more or less effective at each phase.⁵ This is valuable because it links individual learning trajectories to organizational structures, but it is limited by its conceptual nature and by the fact that it does not model AI’s side of the developmental relationship.

2. AI learning to work with humans: hierarchical and curriculum‑based training

On the AI side, the most directly relevant hierarchical work is technical rather than psychological. Loo, Gong, and Meghjani’s “A Hierarchical Approach to Population Training for Human‑AI Collaboration” proposes a hierarchical reinforcement learning (HRL) method in which an AI agent learns multiple low‑level best‑response policies to different partner behaviors and a high‑level policy that selects among them based on the current partner.⁶ The “population training” idea exposes the agent to a diverse set of partners (including humans and other agents) so that it can generalize to novel collaborators. Hierarchy here is explicit: low‑level controllers encode different collaboration styles, while the high‑level controller learns when to deploy which style. This is a genuine developmental structure for AI: the agent first acquires a repertoire of behaviors, then learns to meta‑select among them. Its strength is that it directly targets robustness to new human partners; its weakness, from a developmental‑theory perspective, is that it is framed purely in terms of performance optimization, without a broader account of social or ethical development, and it does not attempt to mirror human developmental stages.

More broadly, recent methodological frameworks for evaluating human–AI collaboration implicitly assume developmental trajectories for AI systems. Fragiadakis and colleagues’ “Evaluating Human‑AI Collaboration: A Review and Methodological Framework” surveys metrics and experimental paradigms for assessing how collaboration quality changes as AI systems are iteratively improved and adapted to human users.⁷ While not hierarchical in the sense of discrete stages, the framework emphasizes longitudinal evaluation and the idea that both humans and AI systems co‑evolve in their collaboration patterns. Its strength is its comprehensive synthesis of evaluation methods; its weakness is that it stops short of proposing a formal developmental hierarchy or curriculum for AI adaptation to humans.

In design research, Song, Zhu, and Luo’s “Human‑AI collaboration by design” offers a unified scheme for classifying AI roles (assistant, peer, coach, etc.) and mapping expected capabilities, interaction attributes, and trust enablers across scenarios.⁸ Although not explicitly developmental, this role taxonomy can be interpreted hierarchically: as AI systems gain more sophisticated capabilities and interaction skills, they can move from narrow assistant roles toward more autonomous, peer‑like or advisory roles. The paper’s strength is its systematic mapping of roles and capabilities; its weakness, relative to your question, is that it does not specify how an AI transitions between roles over time or how such transitions should be staged or governed.

3. AIs learning to work with other AIs: hierarchical multi‑agent cooperation

For AI–AI collaboration, the most developed hierarchical work lies in multi‑agent reinforcement learning (MARL). Cao and colleagues’ “Hierarchical multi‑agent reinforcement learning for cooperative tasks with sparse rewards in continuous domain” propose a two‑level architecture in which a high‑level meta‑agent selects subgoals or macro‑actions, while low‑level agents execute detailed control policies to achieve cooperative tasks.⁹ Similarly, Xu and colleagues’ “HAVEN: Hierarchical Cooperative Multi‑Agent Reinforcement Learning with Dual Coordination Mechanism” introduces a hierarchical structure that separates global coordination from local decision‑making, enabling agents to learn both high‑level cooperation strategies and low‑level actions.¹⁰ These architectures are explicitly hierarchical and developmental in the sense that agents learn progressively: they first acquire low‑level skills, then learn to coordinate them under high‑level policies. Their strength is that they provide concrete, scalable mechanisms for emergent cooperation among many agents; their weakness is that they are largely agnostic about human partners and about social or ethical dimensions of development.

Curriculum learning adds another developmental layer. Bhati and colleagues’ “Curriculum Learning for Cooperation in Multi‑Agent Reinforcement Learning” propose training agents with a curriculum of increasingly challenging cooperative tasks and partner behaviors, so that agents gradually acquire more robust cooperative policies.¹¹ This is a clear developmental idea: agents progress from simple to complex cooperation scenarios, with the curriculum shaping the trajectory. The strength of this work is its explicit focus on staged difficulty and cooperation; its weakness is that it is still primarily a training heuristic rather than a theory of developmental stages with interpretive, psychological content.

A related line, “Hierarchical Consensus‑Based Multi‑Agent Reinforcement Learning for Multi‑Robot Cooperation Tasks,” extends hierarchical MARL to real‑world multi‑robot settings, again using hierarchical structures to bridge centralized training and decentralized execution.¹² This reinforces the pattern: in AI–AI collaboration, hierarchy is a practical tool for scaling coordination, but it is not yet framed as a general developmental theory of how artificial societies mature over time.

4. Cross‑cutting perspectives and what is still missing

Across these strands, we can see the outlines of a three‑part developmental picture. For humans, there is a move from naïve, tool‑centric use of AI toward metacognitively sophisticated, strategically reflective collaboration, scaffolded by education, mentoring, and organizational structures.^2,3,4,5 For AI systems, there is a move from single‑policy, brittle behavior toward hierarchical, population‑trained, curriculum‑shaped repertoires that can adapt to diverse human and artificial partners.^6,9,10,11,12 For AI–AI collectives, hierarchical MARL and curricula provide mechanisms for emergent cooperation and division of labor. However, these pieces are not yet integrated into a single, general hierarchical developmental theory that jointly models human cognitive, social, and ethical development with AI’s algorithmic and representational development.

The main strengths of the current landscape are its empirical grounding and technical specificity: stage‑based human–AI teaming models can be tested in real teams; metacognitive and AI‑literacy frameworks can be embedded in curricula; hierarchical and curriculum‑based MARL methods can be benchmarked in simulations and robotics. The main weaknesses, relative to your question, are fragmentation and asymmetry. Human‑focused theories draw heavily on psychology and education but rarely model AI’s internal development; AI‑focused work optimizes performance but rarely engages with human developmental theory; and AI–AI cooperation work often ignores humans altogether. There is also limited attention to normative questions: how should developmental trajectories be shaped to promote fairness, trust, and shared agency, rather than merely efficiency?

In that sense, the field is ripe for exactly the kind of integrative, hierarchical developmental theory you are asking about—one that treats human learners, AI systems, and multi‑agent collectives as co‑developing over time, with explicit stages, transitions, and design levers at each level. The existing theories and frameworks provide strong building blocks, but the synthesis into a unified developmental architecture for human–AI–AI collaboration is still an open research frontier.

References

1. Developing teamwork: transitioning between stages in human–agent collaboration (Frontiers in Computer Science, 2024). https://www.frontiersin.org/articles/10.3389/fcomp.2024.1455903 (frontiersin.org in Bing)
2. Human and AI collaboration in the higher education environment: opportunities and concerns (Cognitive Research: Principles and Implications, 2024). https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-024-00573-9 (cognitiveresearchjournal.springeropen.com in Bing)
3. Human‑AI Collaboration for Smart Education: Reframing Applied Learning to Support Metacognition (IntechOpen, 2023). https://www.intechopen.com/chapters/1001832
4. Reconceptualizing AI Literacy: The Importance of Metacognitive Thinking (preprint / open‑access chapter, 2023–2024). https://osf.io or similar open repository entry titled “Reconceptualizing AI Literacy: The Importance of Metacognitive Thinking”
5. Mentoring for effective human‑AI collaboration: an integrated theoretical framework (Total Quality Management & Business Excellence, 2025). https://doi.org/10.1080/14783363.2025.2504603
6. A Hierarchical Approach to Population Training for Human‑AI Collaboration (IJCAI 2023; arXiv:2305.16708). https://arxiv.org/abs/2305.16708
7. Evaluating Human‑AI Collaboration: A Review and Methodological Framework (arXiv preprint, 2024). https://arxiv.org/abs/2407.19098
8. Human‑AI collaboration by design (Proceedings of the Design Society, DESIGN 2024). Open PDF via Cambridge Core.
9. Hierarchical multi‑agent reinforcement learning for cooperative tasks with sparse rewards in continuous domain (Neural Computing and Applications, 2024; open access). https://link.springer.com/article/10.1007/s00521-023-08940-0 (link.springer.com in Bing)
10. HAVEN: Hierarchical Cooperative Multi‑Agent Reinforcement Learning with Dual Coordination Mechanism (AAAI 2023; open proceedings). https://ojs.aaai.org/index.php/AAAI/article/view/25863 (ojs.aaai.org in Bing)
11. Curriculum Learning for Cooperation in Multi‑Agent Reinforcement Learning (arXiv:2312.11768, 2023). https://arxiv.org/abs/2312.11768
12. Hierarchical Consensus‑Based Multi‑Agent Reinforcement Learning for Multi‑Robot Cooperation Tasks (arXiv:2407.08164, 2024). https://arxiv.org/abs/2407.08164

[End]

Filed under: Uncategorized |