What other researchers found

Why this page

Most of our research is our own tests, and many of those lean on a model judge. This page puts our work next to what others have measured, so the claims the product rests on can be checked against more than us. Each finding names its source. None of it shows that our machine works.

Models give the same answers

Jiang and colleagues built Infinity-Chat, 26,000 open-ended questions with no single right answer, and ran more than 70 models on it. They describe an “artificial hivemind” in two parts. Each model repeats itself, and different models give strikingly similar answers to each other. They also found that models, reward models and model judges are less well calibrated when people hold different, personal preferences, which is one reason we treat our AI judge as a screen.

NoveltyBench, from Carnegie Mellon, asks models for several different valid answers to creative and subjective requests. All 20 frontier models it tested showed weak diversity, and larger models within a family tended to be less diverse than smaller ones. Our own version, thirty answers to one skyscraper brief, found 26 named for the tide.

It costs the set, not the single piece

Doshi and Hauser let writers use story ideas from GPT-4. Readers rated the AI-assisted stories as more creative one by one, and the gain went almost entirely to the least creative writers. As a set, the stories were more similar to one another. The authors call it a social dilemma. Each writer gains while the range of new work narrows. That is the risk we build against.

Far inspiration raises novelty and lowers quality

Design by analogy has decades of experiments on inspiration at different distances from a problem. Chan and colleagues gave engineering students patent examples that were near or far, and common or rare. Far, rare examples raised novelty above the control group that saw none. Fu and colleagues found that far analogies tend to produce more innovative solutions, but there is such a thing as too far, when the stimuli become harmful.

Across 105 student designers, novelty rose with analogical distance and quality rose as it fell (Venkataraman et al.). With 96 participants, near stimuli improved feasibility and usefulness and distant stimuli improved uniqueness (Goucher-Lambert et al.). One experiment with 84 graduate students found the opposite for designers without analogy training, who did better with near analogies on both novelty and quality (Jia et al.). So “near” and “far” are relative, and results depend on the designers. The overall pattern matches our own finding that far costs fit.

Taste is personal, most of all in architecture

Vessel and colleagues compared shared taste across kinds of images. Faces and landscapes showed high agreement between people. Exterior architecture, interior architecture and artworks showed strong individual differences, and in a follow-up agreement was higher for landscapes than for exterior architecture. That is the case for choosing by person.

On the machine side, work on pluralistic alignment builds reward models that serve individual users and localise a new user’s preferences from a few examples, such as PAL (Chen et al.) and variational preference learning (Poddar et al.). Both work on language and image tasks. Neither has been shown on design concepts, which is the gap our selector aims at.

The problem is real and measured

Architects work long hours. In Parlour’s 2021 survey of Australian architects, more than a third worked over 45 hours a week, and architects scored below average on every measure of personal wellbeing. A 2024 RIBA survey of more than 1,450 architectural professionals found people contracted for 37.5 hours working 44, with 90% working more than their contract. A 2026 study of 642 Spanish architects found 65% reporting severe exhaustion, with long hours and weekend work among the main predictors of burnout.

Competitions add the cost of the search. In Architectural Record in 2012, one firm said it spent about $25,000 on a proposal it won and about $25,000 on one it regretted. Another put a lost Dallas entry at about $16,000 out of pocket and around $50,000 counting time and lost work.

What these do not show

That our machine works, or that people prefer what it makes. The analogy experiments used engineering students and patents, not architects and buildings. The diversity studies measure text. The wellbeing surveys show long hours, not that the hours go to finding ideas, which is our reading from inside studios. The preference-modelling papers are on language and image tasks. What these sources do is make the problems we target real and independently measured, and show where our results agree with them.

Next: Forty concepts, three ideas

All papers