Artificial intelligence is moving rapidly into professional domains once thought immune to automation. With Large Language Models (LLMs) like ChatGPT now capable of generating ideas, code, art and even full-length movies, it's tempting to imagine a future where AI acts as a co-designer—or even a lead designer. But for high-stakes disciplines like mechanical design, product engineering, and manufacturing, current models still fall short of what's required to be truly useful.
Design is rarely linear. It evolves over iterative steps, each dependent on the last. LLMs, however, operate in isolated interactions by default. They lack persistent memory and struggle to maintain design logic across a multi-step process. Without explicit prompts, earlier decisions can be lost, misinterpreted, or overwritten.
LLMs often speak with authority, even when they’re wrong. In creative tasks, that’s harmless. In engineering? It’s risky. The absence of built-in uncertainty modeling or self-assessment means users must always second-guess the output, especially when safety, cost, or compliance is on the line.
While LLMs excel at language, they’re less fluent in numerical precision and geometric reasoning. They may mishandle unit conversions, oversimplify trade-offs, or miscalculate spatial relationships. These flaws make them unreliable for tasks requiring tight tolerances or physical feasibility.
A 2024 research paper co-authored by two of our founders investigated the application of existing LLM models to design and engineering tasks - Gen AI models repeatedly hallucinated or gave outputs that were physically impossible. As shown above, ChatGPT outputs a floating tabletop when asked to design a table. (Source)
Despite being able to describe tools and concepts fluently, LLMs cannot yet interface directly with the files and software used in actual design workflows—like CAD, simulation platforms, or prototyping environments. This forces users to act as translators, reducing efficiency and increasing the chance of miscommunication.
Most AI models today lack the industry-specific knowledge needed for specialized design tasks. Their understanding of materials, safety standards, or regulatory constraints remains surface-level. As a result, their suggestions often need vetting by human experts familiar with the domain.
If AI is to become a serious player in the design and engineering space, it would need foundational upgrades. The requirements of this domain are unforgiving: any AI output must be accurate, precise and traceable. Not just that, it must deal with ambiguity and tradeoffs like actual engineers do.
LLMs must evolve beyond single-turn interactions. Design workflows depend on continuity—tracking design iterations, constraints, goals, and decisions over time. Without memory architectures that persist across sessions, LLMs will continue to make contradictory or regressive suggestions, eroding trust and slowing down iteration cycles.
Engineering doesn’t tolerate “close enough.” Any AI deployed in this space must understand laws of physics, respect tolerances, and preserve numerical accuracy across calculations. Integrating symbolic math engines or constraint solvers could help LLMs respect the strict numerical logic underpinning physical systems.
Perhaps the most overlooked requirement: being able to explain why. In engineering, every decision must be traceable—backed by rationale, standards, or calculations. Current AI systems offer no transparent way to audit how a conclusion was reached. Without a clear chain of logic or sources, these tools are unfit for critical design reviews or compliance-heavy workflows.
Design often involves ambiguity, but models shouldn't bluff through it. We need LLMs to express uncertainty, flag low-confidence outputs, and provide options rather than absolute answers. This would allow users to apply judgment where it matters most, instead of treating outputs as fact.
Neural-net based systems must be combined with domain-specific tools: geometry engines, physics simulators, finite element solvers, or CAD plugins. This hybrid approach allows LLMs to remain flexible while anchoring their suggestions in physically viable solutions.
General-purpose training is not sufficient. Models must be fine-tuned on specific - often proprietary - datasets, such as past designs, organizational best practices and even tribal knowledge residing inside senior engineers. Without this grounding, they remain generalists in a specialist world.
Artificial Intelligence has great potential for design and manufacturing but many structural weaknesses still remain. Right now, LLMs are an helpful assistant—great for brainstorming or quick research—but they lack the precision, consistency, and accountability required to own decisions in real-world product development. To move from “helpful” to “essential,” AI must evolve to reflect the specific needs of the domain.
At Foundation EGI, we’re building an AI-native platform purpose-built for engineering. Our platform combines generative AI models with a hybrid architecture that enforces the precision and transparency that engineers demand. No hallucinations. No black-box outputs. EGI is built to be a true copilot—supporting the real pilot, the human engineer—to vastly accelerate what engineers already do best: building better products, faster and more efficiently.
Come build with us!
If you are interested in learning more, please reach out to us at info@foundationegi.com.