Are You the Right Fit?

Introduction

The authority of knowledge gets away with a lot. Like a man in a white lab coat, assumptions are made when a study is referenced to support an argument. This is not the fault of Science. If anything it's the fault of Science exiting the lab, without care instructions, and colliding head first with transactional arrangements. The attention economy, for example, does not have time for the nuance and caveats written into the Discussion section of an academic paper. Opaque paragraphs do not fit neatly into a snappy headline, nor a tick tock. Similarly, in more traditional transaction spaces - the sort in which a product has been designed and now needs the marketing department to help drive sales - the same problem crops up. How do you fit the line "In a sample size of 128 men who spend less than 3 hours a week in green spaces, self-reported surveys suggested participants had low scores on a scale measuring trait XYZ with an effect size of d."

I've never worked in marketing, but as a perpetual customer the lack of a catchy sales pitch is evident - as are its limits.

Shaping an idea into a size and image that will respond well to several markets and be scalable means cutting off the dead weight. And so, the question becomes: What does the potential buyer need to know? Do they need to understand how statistical assumptions can be wrong but also useful? Do they need to know the history upon which this particular approach to assessment was established? Should another bother to check in on the critique of this methodology and its real world application?

That seems like a lot of dead weight - better just keep the Unleash Your Hidden Strength caption, then.

The dead weight of history

Despite what springs to most minds upon hearing the word "eugenics", it wasn't as limited as all that. Long before Germany thought it was a good idea to completely dehumanise people in concentration camps, Francis Galton was hard at work measuring craniums and setting up research programmes to explain why rich white men like him were so awesome. His scientific contributions include statistical concepts like correlation, regression toward the mean, surveys to collect data about humans, and establishing the Galton Laboratory of National Eugenics at the UCL. If you were interested in learning how he fits into the history of science, find the incredible work of Subhadra Das who is a Science Curator and is also very good at calmly making podcasts that discuss Galton. [1, 2]

It's about at this moment we pause to offer our yeah-buts in support of how much has changed since the turn of the 20th century. While awkward, embarrassing and horrific, it remains locked away in the past.

Well, not quite. Much of the 'analysis' of the human mind and behaviour uses the same methodology today. The personality tests that contemporary organisations are so stuck on are carried out using the same statistical concepts. Latent variables (i.e., variables a researcher cannot observe) continue to be represented by observed actions that have been grouped together. There remains an over-reliance on and obsession with effect sizes while correlation is still conflated with causation. While we're on correlation, "Galton's fascination with genetics and heredity provided the initial inspiration that led to regression" and Pearson's r . [3] Indeed, Pearson took up the post of Chair of Eugenics upon Galton's passing and created the Department of Applied Statistics to bring together human genetics and stats in holy, academic matrimony.

How odd that this didn't feature in my Applied Statistics courses.

The dead weight of epistemic authority

As a passive recipient of the phrase, "Recent studies show…" demand much from the general public. At a minimum, it demands an assumption of objectivity and thus, population wide application.

Is there an objective truth that we can locate with the correct collection and analysis of data? For that to be possible, the data, the methods of collection, the tests, the people involved, the funding, the formal learning supporting all of the above, and the interpretation would also need to be objective.

Zuboff writes about the division of learning in her book Surveillance Capitalism. [3] She explains how three questions help to resolve the dilemmas of knowledge, authority, and power. 1) Who knows? 2) Who decides? 3) Who decides who decides?

This is a useful exercise in coming to think critically about how we 'know' what we know, and how we got to it. Or, how it came to us.

Which brings us back to another familiar voice of authority in the 'knowing' of things - statistics. In a recent paper that was a delight to read, authors Van Rooij and Guest considered the danger in combining AI with psychology - a discipline that "wants good theories and good statistical practices but whose scientific practitioners predominantly lack advanced theoretical, computational, and statistical skills". [4] Ouch.

The authors argue that there is too frequently a conflation of scientific inference with statistical inference. That is, the reductive process of proving a hypothesis (via demonstrating acceptable effect sizes) is seen as enough evidence to support a scientific claim, despite the complexity involved if one wants to know why and how something happens.

If you are struggling to think of this in a practical manner, consider tests that assess your behaviour in a job interview. A team (hopefully) working in the People Department has designed or purchased a test that will tell them which applicants demonstrate the Behaviours required to do this job. Why does the hiring panel use it? Because Science is the knower, and Science will objectively explain which candidates demonstrate what desirable behaviour, and thus who will be a 'good fit' for the role. While the labels on the categories have changed over time, the methods and purpose have not. The tests are still trying to identify who is Fit and who is Unfit.

NB. I am amused by the idea of believing someone's supposed innate conscientiousness levels are going to overcome burnout-inducing poor work design choices.

The dead weight of responsibility

Inevitably in any given statistics course, the professor will trot out the George Box quote "All models are wrong, but some are useful." It's less of a direct quote and more of an un-ironic reduction of his writing on the role of mathematics in science. In a paper published in 1976 he wrote,

In applying mathematics to subjects such as physics or statistics we make tentative assumptions about the real world which we know are false but which we believe may be useful nonetheless. [5]

The previous two sections of the paper begin with, "Since all models are wrong…", so you can see how the shortened version came about. It is interesting to note that in the same paper, Box impresses upon the reader how important it is to engage in an iteration between theory and practice, along with a feedback loop so that science can contribute to the advancement of learning. Although, I suspect bad replication studies on a loop were not what he had in mind. As one critical paper describes them, studies in which "success is identified as finding a statistically significant result in the same direction as the original," and then combining those into meta-analyses.[6] Copy, paste. Looking into the current evolution of automated application for things like management, the design must be humane. [7]

I appreciate we take comfort in handing over authority to the 'knowers', and in this instance they are the mysterious bright minds who made the tests that tell us who is suitable for work and who isn't, but what is our responsibility in that? After having spent the time, energy and money required learning about the use of stats in the theory and practice of organisational psychology, I've concluded I have a responsibility to question the collision of historical bias, theories of psychology, and applied statistics. Likewise, when profit-making and automation join the club, I feel an additional responsibility to question why we are still assessing and categorising humans based on unobservable traits.

Final thoughts

The Victorian love affair with eugenics assumed that much of what we do comes down to what exists inside of us. Race, gender, class were not considered social constructs but data to be analysed and used to tell us who was and wasn't a good fit. [8]

Looking for the right fit is of deep concern to many companies. Some devote entire departments and not an insignificant spend on testing the 'good fit' of employees, potential and employed. I wonder why we still test humans at all. What is the point in using psychometrics? It is to categorize, measure, order, and assign value to humans. It is to predict behaviour and to make decisions based on that prediction before the behaviour has occurred. It is to decide who is and isn't desirable for hiring or promoting or leading, lest they do the job badly and lower ROI. Interestingly, by using psychometric products, the implication remains that issues within an employee are the result of individual lack, not an organisational one. And so, the systems, structures, and environments which consistently inform an individual employee's behaviour are left firmly in place to shape the next Right Fit.

Works Cited

[1] UCL podcast: Living With Eugenics

[2] BBC Radio 4 You're Dead to Me, Francis Galton: Victorian scientist and pioneer of eugenics

[3] Jeffrey M. Stanton, Syracuse University: Journal of Statistics Education Volume 9, Number 3 (2001) <https://jse.amstat.org/v9n3/stanton.html>

[4] Van Rooij, I., & Guest, O. (2026). Combining psychology with artificial intelligence: What could possibly go wrong? Current Directions in Psychological Science, 35(3), 193–200. https://doi.org/10.1177/09637214261438379

[5] Box, G. E. P. (1976). Science and Statistics. Journal of the American Statistical Association, 71(356), 791–799, p. 792. https://doi.org/10.1080/01621459.1976.10480949

[6] Irvine, E. (2021). The role of replication studies in theory building. Perspectives on Psychological Science, 16(4), 844–853. https://doi.org/10.1177/1745691620970558

[7] Röttgen C, Herbig B, Weinmann T and Müller A (2024) Algorithmic management and human-centered task design: a conceptual synthesis from the perspective of action regulation and sociomaterial systems theory. Front. Artif. Intell. 7:1441497. doi: 10.3389/frai.2024.1441497

[8] Yakushko, O. (2019). Eugenics and its evolution in the history of western psychology: A critical archival review. Psychotherapy and Politics International, 17(2). https://doi.org/10.1002/ppi.1495

Cover image: Photo by William Warby on Unsplash