Becoming Comfortable with Black Box Engineering

In this article, Professor Andrew Garrard from the University of Sheffield and Dr Parakram Pyakurel from New Model Institute for Technology and Engineering discuss whether engineering education's traditional resistance to "black box" thinking remains fit for purpose. As simulation software, Industry 4.0 methods, and generative AI make it harder for engineers to understand the tools they use, is it time to take a pragmatic approach in education and consider black box engineering not as a threat to be resisted, but as an inevitability to be acknowledged?

Cartoon images generated by AI using the prompt “Please can you read this article and come up with a simple pencil drawing cartoon that makes light of the general themes of the piece?”. This image was created using Open AI’s ChatGPT 5.4.

There is a widely held belief amongst engineering educators that students should be discouraged from relying on a “black box” approach to engineering; that is, producing results using methods and tools with little or no understanding of how they work. Moreover, it can be considered dangerous to certify a graduate as competent if they accept, on faith, the validity of results generated by means they do not understand.

To what extent, however, is this view an anachronism, tightly clung to by academia but no longer reflective of real-world engineering practice?

Consider, for example, predicting how a system will behave through engineering simulation, such as by using the finite element method. In previous generations, it was conceivable that engineers could build, from the ground up, the software required to perform finite element calculations. This was possible because the limits of computational power meant that only relatively simple systems could be analysed using comparatively straightforward numerical techniques.

Fast forward to today, and modern engineering simulation software can consist of hundreds of gigabytes of code and mind-bogglingly complex algorithms designed to optimise every stage of the process. Take the generation of computational meshes. In the past, engineers constructed meshes manually, deciding the position of nodes and connecting them together with lines. Today, powerful automated meshing algorithms generate highly complex meshes as part of a largely turnkey process.

It is unrealistic to expect engineering students, who are learning multiple topics across a wide range of domains, to understand the detailed inner workings of every piece of software they use. As the ability to understand precisely what engineering simulation software is doing has diminished, it has been replaced by an increased emphasis on heuristics: I chose this turbulence model because experience suggests it performs well for this type of fluid flow.

The same “black box” approach underpins many aspects of Industry 4.0, and more recently Industry 5.0. Rather than fully understanding the physical mechanisms governing the dynamics of engineering systems, sufficiently large datasets can be used to predict outcomes accurately through sophisticated statistical methods.

The world is already full of black-box engineering, and this trend is only likely to accelerate with the rapid advancement of artificial intelligence. While it may be extremely challenging to fully understand all the methods used in modern, complex engineering software, the algorithms involved are at least deterministic. By contrast, the stochastic outputs of Generative AI, like large language models, means that determining how they will respond to any given input is not something anyone can reliably predict, even those who have built the models. As AI becomes increasingly embedded within software products, additional opaque layers will accumulate, making it ever more difficult to truly understand what the computer is doing.

Whether the adoption of a “black box” approach to engineering is desirable remains open to debate. What seems less debatable, however, is its inevitability in the modern world. We can either acknowledge this reality, or pretend it is not happening.

Once we admit this - what are the implications?

The increasing complexity of computational methods, together with the advent of AI systems whose behaviour can be difficult to fully understand, may appear to be a distinctly modern problem. However, making predictions about how systems will behave with limited understanding of the underlying governing principles has long been the standard approach of empiricists.

Often attributed to Heisenberg is the quote:

“When I meet God, I am going to ask him two questions: Why relativity? And why turbulence? I really believe he will have an answer for the first.”

As a species, we remain unable to analytically calculate turbulent flow. Yet engineers are able to model aeroplanes, pipelines, and wind turbines with impressively high precision. While imparting conceptual understanding of fundamental engineering principles is important, we may be overlooking opportunities to train students in data analysis, interpretation, and subsequent processing.

As educators, our task is to equip students with the understanding and tools necessary to have a positive impact on the world. In already crowded engineering curricula, content must compete for inclusion among the many topics considered essential for an undergraduate programme. If complexity, big data, and stochastic computation are becoming commonplace, is there a need to shift some emphasis away from traditional physical sciences toward statistics, empiricism, and validation?

When citing this work, please use the following citation:
Garrard, A & Pyakurel, P. (2026). “Becoming comfortable with black box engineering”. Centre for Engineering Education Blog, The University of Sheffield, Sheffield, UK. March 2026. https://www.ceesheffield.co.uk/2026/03/becoming-comfortable-with-black-box.html