Lighthouse Letter VI

Spreading practices > scaling products

By: Xinman (Yoyo) L.
AI Architect – Minds Studio Practice Lead 

I love oxymorons. i remember i first came across this concept in my middle school English class, and i was intrigued. bittersweet. deafening silence. virtual reality. wise fool.

the idea of an “oxymoron”, to me, was expansive. expansive in a way that it juxtaposes all the contradictions at once, yet reconciles and holds them together through this very juxtaposition. it showed me that many things in this world—from how society operates down to my own identity—do not need to be defined black or white. rather, much of our sense-making and discoveries stem from liminal, grey spaces.

for the past few months, i’ve been deliberating on a new oxymoron: engineering serendipity.

More than a happy accident.

some gut reactions to this might be: how can you systematically plan out unanticipated “luck” and “accidents”?

in a decade-old TIME article, How To Engineer Serendipity,1 Greg Lindsay writes that serendipity is a bountiful source of good ideas, a state of mind, an emergent property of social networks, and a process through which we discover unknown unknowns.

so, serendipity is not just happy accidents or sheer luck. it’s a perception of those—underpinned by a coordinated set of circumstances and interactions.

There are many innovations and discoveries that resulted from serendipity: Post-it notes, Velcro, the benzene ring, Play-Doh… today, more companies are designing open office environments with cafes and micro-kitchens to foster casual collisions between colleagues.2 cities, too, are dissolving their walls of conventional facilities and morphing spaces into hybrids where offices, homes, playgrounds, and streets blur into one another.3

indeed, serendipity cannot be engineered into a system. rather, it is an organic, emergent byproduct of a system—one intentionally architected with structural holes4 and negative spaces that enable transdisciplinary collaboration, learning, and innovation.

As Dr Stephann Makri, during his time at UCL, put this nicely:5

The notion of ‘designing for serendipity’ is an oxymoron because once we try to ‘engineer’ it into a system, users may no longer perceive the experience as serendipitous… Designers of interactive systems shouldn’t try to offer ‘serendipity on a plate.’ Instead, they should design tools that create opportunities for users to have experiences they might perceive as serendipitous.

That direct pursuit defeats the very thing pursued shows up more broadly in what’s called the principle of obliquity (which, fittingly, i learned in a random conversation on a bus): our goals are best achieved indirectly without intending them.6 this is because in complex systems with unpredictable dynamics, direct pursuit narrows focus and forecloses better paths that only become visible sideways. when you obsess over a metric, attention shifts from genuine engagement and curiosity to external validation; then the pressure to achieve becomes self-defeating: students who chase grades tend to learn less; companies fixated on profit often underperform those focused on serving customers well.

serendipity, like these, cannot be pursued. it can only be built toward.

Products, probes, possibilities.

What steered me into thinking about engineering serendipity lately is my experience with my education data science capstone project. the work is now available here,7 and i have previously written on how i navigated parts of this very serendipitous process. but TL;DR: i co-designed a tool with teaching assistants (my users) supposedly to use AI to “augment” their feedback rather than “automate” it away—only to later realize that augmentation is not a property of the intended tool’s design, but of the user’s emergent relationship with the tool during actual use. in this sense, the tool becomes not just a product, but a probe for new possibilities.

to put it in my capstone’s context:

the value of human-AI collaboration in feedback provision lies not just in the feedback [the AI system] automates or augments, but in what it allows teachers, researchers, and designers to notice, examine, and ultimately reconstitute about their own practices.

for example, one teaching assistant rejected a piece of AI-generated feedback as “inaccurate”. but when asked why in a follow-up interview, she traced the gap not to a model failure but to the course rubric being underspecified (she knew the AI was grounded to the rubric). this probed her into revising the rubric for the next school year.

Similarly, partway through using the tool, another teaching assistant stopped using a feature where she’d draft feedback and AI would “enhance” it. “enhance by what standard?” she questioned, arguing that feedback is inherently personal and relational, not something with an external bar to be raised toward. this probed me as a researcher and system designer into reflecting upon ongoing iterations: what actually makes feedback good, and who gets to decide?

most efforts on AI in education today ask how we might “innovate” tools to change user practice. but both moments above highlight how intended design ≠ actual use. not that the tool itself is broken, but that using it surfaced questions the design had never thought to ask in the first place.

in other words, it is not just the tool design, but the user’s own agency and interaction (i.e., what they do with the tool, what they push back on, what they notice because of it) that decide what the tool actually becomes in use, versus what it was designed to be. this is precisely where serendipity emerges, and where it can be easily stifled if system designers stick to tunnel vision and preconceived assumptions.

I found the same intuition articulated by my former colleague Mubarak (Moby) Marafa (best designer!), who wrote a substack earlier on agentic product design as world-building.8 he describes how product design today is no longer about predefining features and user journeys, but scoping out possibility spaces: setting up the right rules and asking what emergent behaviors a system should be steered toward or away from.

In regular product work, I am used to mapping a user journey, finding the pattern, designing the feature and solving the problem. That starts to break when you are designing a system that can remember, communicate, and change over time. You cannot draw every journey in advance.

when we design for possibility spaces, we start thinking about how we might engineer the affordances, boundaries, and interpretations of a tool’s current state to be porous—in ways that probe and give room to the process of becoming and co-evolving, toward future states of serendipitous unknown unknowns.

Octopuses spreading pockets of innovation.

But i think much of this notion of possibility space design applies beyond products and even probes, and more broadly to spreading practices.

Jana Werner and Phil Le-Brun, in their piece on what they call “octopus organizations”,9 draw a useful distinction here: most companies try to scale what works by running a pilot, measuring the pre- and post-outcomes, and mandating it top-down, institution-wide, or even across institutions. this is a Tin Man model that treats innovation as an isolated product developed by an isolated department, handed off for others to implement. in fact, edtech does this constantly: a vendor validates a tool through an RCT in one or a few classrooms, then packages it for district-wide rollout. it assumes contexts are fixed and interventions are universal.

octopus organizations work differently. an octopus doesn’t coordinate its arms through a central command issuing instructions to each one. instead, most of its neurons live in the arms themselves, sensing and acting with independence, then feeding back into the body that still somehow moves as one. innovation, similarly, does not happen within an “innovation” department; rather, it’s distributed to whoever is closest to the problem. they spread, not scale.

As Werner and Le-Brun put it:

a complex world, however, is more like the ocean, the home of the octopus—where a small shift in the current can create unpredictable effects miles away. The shifts and unpredictability make it difficult to navigate; you can only sense, respond, and learn from the flow.

This is, i think, why RCTs struggle so much with teaching and learning as a research object. i had a conversation a few months ago with my advisor Prof Candace Thille on this. in short, RCTs work beautifully for mechanistic systems, where you can isolate a causal chain through control. BUT! education is a complex, adaptive system—and context varies! we have different institutional structures, class cultures, rubric formats, assessment values, curriculum design, student competencies, teacher attitudes, and so on… even under tight experimental constraints, people don’t behave mechanistically. they are people—embedded in complex adaptive systems that exhibit these features:10

1. emergence: higher-level patterns arise from countless lower-level interactions, rather than being designed in directly. e.g., in my capstone, i defined and built four modes for the teaching assistants to collaborate with AI; but when i clustered their actual behavior, entirely different behavioral patterns emerged instead, cutting across my predefined modes.
2. feedback loops: those emergent patterns loop back and reshape the very structure that produced them. e.g., when one teaching assistant rejected the AI-enhance mode outright (“enhance by what standard?”), her pushback fed directly into the next iteration’s design.
3. attractor states: recurring patterns that persist across different starting conditions, shaped as much by the people inside the system as by any structure imposed on it. e.g., regardless of whether the teaching assistants were randomly assigned the modes or were allowed to choose, they kept settling into the same behavioral clusters.
   

all of this is to say that educational contexts are too complex and emergent for a single validated product to install cleanly everywhere top-down. instead, we should create the conditions, parameters, and possibility spaces that facilitate each context arriving at its own version, often in a scaffolded yet organic and serendipitous way.

put another way, we should shift our mindsets from scaling a product to spreading practices. and if we do build anything at all for “education innovation”, it should serve two processes:

1. divergence—probes for more generative practices that give rise to new possibilities (pockets of local innovation)
   • does it prompt a user to reflect and develop a new POV on their current approaches to teaching, learning, or educational support?
   • does it create room for the user to push back on the system design, rather than simply comply or ignore?
2. convergence—infra that documents evidence, synthesizes insights, and enables others to fork approaches (fabric of shared practices)
   • are user overrides recorded, and what do they tell us? are they failures or discoveries?
   • are we documenting the variations and the greatest common divisor across how a tool gets adopted in different contexts through different practices? what are the shifting parameters? what are the invariant parameters?
     

In this way, we engineer serendipity by allowing the system design to meet, morph, and co-evolve with practices—which, btw, i think informs us on how we might rethink the role of AI.

much of what i wrote above is more possible now than it’s ever been with AI. We can build probes and infra fast by iterating, prototyping, and testing them again and again with users across different contexts.

However, our perception or measure of “innovation” still hinges largely on how much AI we can stuff into a product. I don’t think it should be about that. rather, innovation is about new ways of seeing, doing, imagining. And that takes more generative practices than generative AI.