Think about the 2 types of LLMs when crafting interview questions (reasoning and non-reasoning). Which type of model reflects the insight you need?

The non-reasoning LLMs act more like fast databases – great for quick, knowledge-based recall, summarization, translation and generally simple tasks. I call this the lexicon mode. When you are in the lexicon mode, you ask the interviewee, “What tools did you use in project X?”

The other mode is the architect mode. Think of it like the reasoning LLMs. You wouldn’t ask a reasoning LLM what is the capital of France. You are wasting too many ressources and it is slow. You use it when you want to analyze situations, connect concepts, evaluate options, and solve problems. You would ask “How did you decide which tool was best for that project, considering the constraints?”

In an interview you are interested in what is inside the candidates <thinking></thinking> tags. You want to assess how it generates intermediate tokens that will help them give a final answer.