I’ve been using search engines for nigh on three decades of my life at this point. A search engine is a tool you use to help you find something that is true. Truth, because it has meaning, cannot be delivered by a machine, it’s got to be arrived at via something that can perceive meaning, and as far as I know living consciousness is the only thing that can do that. But what a search engine can do (what good search engines do very well) is provide information that can be processed by the user into knowledge, which can then be used to seek out truth.

LLMs claim to provide truth, in the form of answers to questions. This rubs me the wrong way from the outset. I don’t want to imagine that I’m in conversation with another intelligence when I’m interacting with my computer, because I am not. When you use a search engine, you (whether you realize it or not) are operating a machine via the input of logic operators, and the output that you receive is a deterministic one, based on intentionally crafted elements of that machine’s algorithms. As a user of that tool, you can develop your skills and learn to use it better, until the tool becomes an invaluable part of your search for something real, but the tool doesn’t purport to provide you with truth. It serves you platters of information, based on what you ordered from its menu. It’s up to you to synthesize that information into knowledge and understanding.

The way you operate a large language model is different. So much so that I feel like I’m stretching the word “operate” to even describe what it is that you’re doing. Since you input instructions in natural language and receive your output in natural language, the results that you wind up with have gone through at least two black boxes on the way to your eyeballs. First, a complicated and unpredictable algorithm reads the natural language input that you provide, and then translates it into instructions for the machine to produce some kind of output. But what are the instructions that it takes into its database of training data? You, the user, don’t know, and even if you programmed the darn thing yourself, you still wouldn’t know, because the way the language is processed is always changing based on the other inputs the machine receives. Then, in order to provide your output in the form of natural language as well, the same basic black box word guessing machine is applied to the machine’s findings in reverse.

When I first started using the LLM search tools, I was not impressed with them at all. Sometimes they produced correct answers, and sometimes they didn’t, but the correct and incorrect answers tend to be delivered with equivalent levels of certainty, and even when you click the “show your work” button there are sometimes baffling leaps between what the machine claims its source is and the output that it gives you. But really, why wouldn’t there be? The machine doesn’t “understand” either inputs or outputs in a way that means what it means for a conscious human being. It just has billions of words and connections between them in its guessing engine, and selects the ones that it thinks would be most likely to please you, then does the parlor trick of arranging the ones you receive into tidy sentences. How, I thought, could anything worthwhile be gleaned from incorporating these into my workflow?

The answer turned out to be (much to my chagrin) playing the game they present on its own terms. By relentlessly badgering them with rephrasings, requests for clarifications, and most importantly challenges on what they got wrong or didn’t provide evidence for, I was able to take the complicated questions I arrived with and hammer out some actual answers, which I then used to synthesize knowledge in my brain, with my organic consciousness (whatever that is) and arrived at something approximating a true answer for the problem I was trying to solve, which I was finally able to write up for my customer.

Do I think that this workflow is better than what I was doing before? No, not really. But if my employer, like so many others, is going to require that these tools be a major part of how I get my job done, I can at least figure out a way to do it well.