Work is first and foremost a necessity. But if a person is fortunate, they can work at jobs in which necessity is not the directing force. Happily, because of my background in programming, the world of the humanities in general, and jobs within it in particular, were never a matter of necessity for me. I could afford, for instance, to “waste” three years editing two books (one of which, on the concept of the victim* in the modern humanities, has yet to see print). In any case, my work in the broad sense — which includes studying philosophy — raises a few reflections in me, and I ask the reader’s forgiveness in advance for not developing them here systematically, owing to time constraints. Still, I do think there is some value in committing a rawer version of these reflections to writing.
Since I edit nonfiction books — and not only that, but I am a substantive editor* rather than a language editor — my relationship to certain kinds of editing work is one that is not, at bottom, technical. Yes, a certain technical ability is required in order to understand how arguments function, but at its core, editing a nonfiction book — all the more so with people who from the outset see this process as part of writing the book rather than as the polishing of an existing product — is a process that, for lack of a better way to define it, I will call “dialogical.” Not in an egalitarian sense: the role here is part of a hierarchy in which the delivery of the author’s idea is the main thing, and my ideas that are not about the work itself are not supposed to make their way in. In this sense, this kind of work is far less vulnerable to AI, for obvious reasons. Authors who are in this place to begin with are not themselves doing it out of necessity, and so a situation arises in which, beyond the technical question of whether AI can do the same work, there is simply no need for it.
From here we can think about a certain generalization. My reasonable expectation is that new social norms, bad and good, will form around the use of AI in the humanities. We are not there yet. But allow me to guess that not only the matter of norms, but also the matter of institutions and the conditions within which the work grows, will play a major role. If we manage to create spaces that have no need at their foundation — careerist or material — then the humanities, or the humanistic pursuit, will manage to leap easily over the AI revolution. Academia, presumably, is not a factor in this context, because there necessity rules in more than one sense. There are pockets within academia around the world that have managed to create such spaces, but what most of them have in common — and this is the university’s advantage in the age of AI generally — is that they all sit around an ethos, or even an institution, whose origin is pre-modern. Either way, the absence of necessity of the kinds mentioned is not the absence of necessity altogether — true scholē. I assume that writers’ envy, competitiveness, etc., will continue to be a significant force, and a good thing too. But what is important to notice, in my view, is that the competitive space will not be algorithmic but organic to the project at hand: even if one removes the necessity of writing for a career or for money, writing in a space that contains thousands of competitors is not realistic. This produces, by default, entirely artificial attempts to shrink the competition to more natural dimensions, and in the process rewards flat writing, whose main advantage is a technical feature such as “is this something that can be cited in as many contexts as possible” rather than criteria internal to the work itself.
All of these things, I assume, have been said by others better than I could say them. But lately a few more thoughts have arisen in me, which concern, specifically, the architecture of AI. One of the interesting things about this architecture is that what is called a “transformer” is a kind of “neural network.” This is, of course, a misnomer: human neural networks do not resemble, in the way they work, those of AI. Let us take a brief look at why it still matters, following Dreyfus, that statistical learning, too, is based on a kind of mechanism of mathematical manipulation — and this even though I am speaking here of an algorithm of digital neurons, whereas Dreyfus’s original critique focused for the most part more on algorithms of higher-order symbol manipulation, a problem that was “solved” in directions Dreyfus was sympathetic to (back then, at the dawn of the algorithms that can be called “neural networks”).
As Hubert Dreyfus already wrote in his legendary book from the previous round of AI hype, in the 1970s, What Computers Can’t Do, viewing the mind* as a machine for manipulating symbols is a sort of analogy that got out of hand and does not reflect the reality of human cognition, and any attempt to imitate the way human beings perform higher-order symbol manipulation is doomed to failure. Dreyfus deserves to be written about in his own right, but I would not be very surprised if in our era, in which the AI hype is this time backed by extraordinary achievements, he receives little attention beyond small pockets of people who, on the one hand, understand the technical subject matter and, on the other, have a philosophical training serious enough to renew part of the discussion that was already conducted back then. It is worth noting here that I, and I assume many other Heidegger scholars, owe to Dreyfus our initial initiation into Heidegger’s philosophy.
Either way, I want to sharpen somewhat an argument of Dreyfus’s — from his book on Division One of Being and Time, of all places — in light of what I understand from the results of contemporary AI. One of the acute problems facing current LLMs is the problem of context (a broader version of what was once called the frame problem, a version Dreyfus dealt with). The problem is that AI does not know how to distinguish between the important and interesting (or essential) and the trivial or noise, especially in contexts that by their nature do not belong to a controlled environment. This problem, as I understand it, is the root of what popular jargon calls AI “hallucinations.” In a formulation that is slightly too cautious, and relevant to cases of discrete hallucinations (invented quotations, logical errors, etc.), let us say that this is the result of a built-in preference for the answer that is safe for the model over an answer that would be considered “correct” in a given human context. In a broader context, by “hallucinations” I mean not only the discrete cases, which are also relatively amenable to symptomatic treatment, but, as we shall see, a whole range of more “latent hallucinations” that stem from an incorrect distribution of weight across the conversation. Hence what is interesting is that, as far as the current successful architecture is concerned, it is the very same mechanism that produces the hallucination and the correct answer. There is no comprehensive solution to this problem. It stems, if I understand the problem correctly, from the fact that problems in ordinary human reality are a higher-order product of ordinary activity within a given world. With today’s AI, what we are in fact dealing with is the rewarding of guesses in a controlled environment. In this sense, not only does current artificial intelligence not “imitate” the human being, but its developers, quite rightly, adopted “machine learning” approaches, which by their nature learn in an entirely different way from human beings.
(The Sleep of Reason Produces Monsters, Francisco Goya, 1799~)
Now let us turn to why the new algorithm matters. Take, for example, the architecture of a digital neuron. The architecture dictates that the machine’s learning be done in relation to a very large quantity of digitally encoded data, and through dense statistical training that becomes rigid following a conscious design choice, not to say an economic one: the “weights” that have been fixed, and which determine the lion’s share of the AI’s response, will no longer change for the end user. This means that, unlike any human being, the AI in effect approaches every problem with a very long list of pre-encoded “presuppositions” regarding the problem it is asked to solve. In fact, the end user also has no real way to “correct” those presuppositions once the model has been trained, or to undo its prior learning if that learning brought it a certain measure of success in contexts entirely abstracted from the problems of human beings in the real world. Even NotebookLM, which is marketed to researchers in the humanities as a tool that completely neutralizes all “prior knowledge” and supposedly knows how to take into account only the data uploaded to it, is not capable of this, for two hypothetical reasons (since neither I nor anyone outside Google knows what the code is, but judging by the results in the field and familiarity with the industry, this can be framed as an educated guess). (a) What NotebookLM actually does is to map by itself, by means of the LLM, a geometric index of the data uploaded to it, and this mapping is done with exactly the same presuppositions we discussed. (b) Even after the mapping, when one asks it questions, what restrains it from deviating, supposedly, from the context defined for it is in fact a hidden background prompt that asks it to do so. But as with other “guardrails” of this kind, which all the major LLMs implement, this is an expression of a recursive problem: if you ask something of the AI and it errs, you can ask it not to err. But the request not to err is exposed, more or less, to the same likelihood of error. The attempt to correct errors by means of “more text” (or a further, renewed mapping of the entire conversation) will probably fail. Moreover: the more “protections” of this kind you add, the more the actual effectiveness of the AI declines. To the best of my understanding this is a paradox rooted in the way things are currently done, and it has no theoretical horizon for a solution, that I know of, which does not amount to symptomatic treatment. In fact, within what is called the “context window,” you are conducting a conversation with the AI, and the way it remembers what was said earlier in it is a prompt that contains, each time anew, the entire conversation, so that “your answers and the AI’s are fed” back into a new context that grows linearly, unlike the way an ordinary human conversation proceeds. There is also a compression mechanism here, which kicks in when the conversation grows overly long, and which likewise does no wonders for the humanity of the thinking involved. The model is in effect “memoryless” in the sense that is decisive for any serious analogy to the corresponding human concept: the temporal dimension, and the different weight that each move has at a given time relative to the context in which it was created, receive no real weight, and this is a problem that grows worse the longer the conversation goes on.
What is interesting here are the implications, because they are less clear-cut than one might have expected. It means, first and foremost, that the way contemporary artificial intelligence organizes information has a clear limit on relearning: it depends on the “context window.” One of the sure ways to enlarge it in a way that is meaningful for the end user is “more energy,” which, as is well known, costs money — and certainty, as is well known, is worth paying for. In fact there is a real suspicion that much of AI’s progress in recent years owes no small part to a rather childish approach of brute force, which, beyond its lack of economic (and some would say environmental) justification, is also unlikely to overcome a real problem in the way AI is able to conduct a “discussion” about something that is not already encoded in its entirety within some database or other. Anyone who engages in philosophy (from a certain level up) and has tried their hand at a discussion with AI understands this frustration, which is the truly paradigmatic form of hallucination, even though it is something subtle and less discrete. Assumptions that have already been agreed upon are casually discarded or forgotten, rather arbitrary tacit assumptions get added, and the AI’s ability to weigh new information in light of the center of gravity of the discussion, as opposed to the center of gravity of the existing literature, is rather limited. If we try to put this more clearly for people with other, more important skills, we might say that what Rawls called “reflective equilibrium” is something artificial intelligence struggles to maintain over time or to any real depth. In human beings, adjustments already made in the past retain their relevance in light of their relative position in the process of reflection. The center of gravity of artificial intelligence, by contrast, is maintained, as noted, against the literature and not at the center of gravity that forms naturally in a reasoned conversation between people. I am even willing to formulate this as a hypothesis: given that an AI has been trained on a large enough quantity of professional content, it will not manage, over the course of a long conversation, both to maintain for long a center of gravity that deviates from the professional literature in that field and to maintain the same level of performance that it has in an environment more similar to its training environment.
A further implication, almost trivial, is that this matters a great deal for the kind of philosophy AI is capable of producing. Under conditions of hundreds of thousands of existing data points from an existing field of knowledge such as philosophy, it is capable of performances that are not bad at all, which the vast majority of philosophy lecturers can only envy. But to a large extent this capability of its is tied to the kind of philosophy that became popular in advance in the Western world, as Dreyfus noted, following certain traditional preconceptions. What is meant is less these or those philosophical arguments and more the form in which philosophy is done. Anyone with a foothold in the history of philosophy understands well the dimension of philosophy that can be called “historical.” And yet, over the last hundred years the tendency in philosophy — and it makes no difference whether it comes from the Continent — has been to answer the challenge of a global environment, which is almost by definition an ahistorical environment. Articles are written so as to rely — supposedly — on as few intuitive presuppositions as possible that are not shared by a large part of humankind, in a process that is of course reflexive: the kind of presuppositions that are or are not worth putting into a published* article or book tends, by the very nature of the environment, toward presuppositions that are tied as little as possible to the concrete world in which the person is doing philosophy. This bias has for decades played a central role in the material infrastructure within which philosophy is done, and as a result it certainly looks as though AI is capable of producing a professional intervention in the accepted philosophical discourse and of overtaking its competitors with relative ease: the simple reason is that this kind of philosophy tends from the outset to encourage philosophy that can be encoded and transmitted without the need for some particular shared understanding.
In this sense, a few days ago an excellent piece appeared in Haaretz on how translators of literature from foreign languages are coping with the new technology. I recommend you go and read it, but what matters for our purposes is that it is said there, in passing, that nonfiction can today be translated almost automatically at a higher quality than what you would get from an ordinary translator. And this is both true and not true. It is very true with respect to nonfiction texts that were written, as noted, under the conditions we described above, that is, in recent decades. It is less true of historical philosophy: translating philosophy of this kind requires, as is well known, a heightened awareness of philosophical anachronism on the one hand, and thinking about relevance on the other. The latter itself depends deeply on the kind of conversation we are conducting at a given moment about historical philosophy; and accordingly, one can see, for example in ancient philosophy, translations that grow out of academic contexts in which the kind of problems of interest is of one sort, as against a translation of the same text in the context of other problems. Naturally, I am using “institutional” examples, but the real sting lies precisely in contexts that are not expected to be mapped any time soon, all the more so if the economic bias of AI development is toward large, available data that is already encoded in one way or another. I mean, of course, serious philosophical conversation as it is conducted in the human world.
As fate would have it, I happened to be under the necessity, of all things, of editing a nonfiction text translated by AI. I have already had occasion to work on several human translations, so this is an interesting opportunity to learn where we currently stand. As noted, the translated nonfiction text is from recent decades, and accordingly the AI’s translation in this context is far more successful than what a human translator would have done. In particular, more global problems — ones that do not require attention at the level of the individual line — can be solved relatively simply by means of the AI’s editing tools. But therein lies the rub. Even in translating texts that have already been edited in the past and belong to a relatively current context, the refining of the translation and its editing do not diminish. Yes, there are fewer human errors to correct, quantitatively. But most of an editor’s work is in fact a series of small decisions over time, according to a variety of considerations: scholarly, philological, political, and more. It is Sisyphean work, even if many of the decisions end in a ruling in favor of the AI’s relative arbitrariness — its “neutrality” — if only because even when it is not precise, that matters. Forgoing the stage of human review in a process of this kind, if it is being considered at present — and in my view also in the foreseeable future — is an act of intellectual irresponsibility of the first order. As far as I can see, AI is not going to “get the keys” in the foreseeable future, at least if a fairly basic social infrastructure of intellectual responsibility is maintained.
The heart of my claim, then, is that on all the various planes — from the economic, through the social and institutional, to the technological — the fundamental assumption of “Dreydegger” (the amusing portmanteau, once common, of Dreyfus–Heidegger), according to which it is an essential feature of the human being to work within the framework of a world given to it in advance, has not lost its force. The further we move away from this assumption, on each of the levels in question, the more we in fact create not only human problems but real technological problems. This holds especially for traditional spaces, practices, and hierarchies — such as the editor and the author — which make possible the occurrence of thinking that is freed, in effect, from limitations which are, to begin with, surprisingly similar to the limitations of AI that we went over. This is admittedly not my field, but as an observer from the sidelines it is interesting to watch the development of artificial intelligence technology, and above all the human response to it, as we strip away the futuristic hype and focus on what has already been achieved, what is going to be achieved, and what in all likelihood will never be achieved along this path.