Beyond Models – Why Domain Insight and Quality Data Will Define AI’s Future
posted: 10-Feb-2025 & updated: 13-Feb-2025
… the next competitive battlefield isn't in developing better base AI models, but in effectively combining them with domain-specific data through technologies like RAG and vector databases.
This evolutionary chain – represented by keywords from Big Data to Deep Learning, then to LLMs & genAI, and now to AI agents – represents more than just a progression of buzzwords. Each transition signifies a fundamental leap in our ability to solve complex problems with machines, redefining the boundaries of AI.
The future belongs not to those who can build the biggest models, but to those who can most effectively organize and leverage their domain-specific data.
From Mathematical Foundations to AI Revolution
In 2004, when I completed my PhD under the supervision of one of optimization theory’s most brilliant minds, few could have predicted how optimization mathematics would become the backbone of today’s AI revolution. Yet looking back, the threads connecting advanced mathematics to modern artificial intelligence (AI) were already being woven in the halls of Stanford’s Engineering department.
I earned my PhD in Electrical Engineering under Professor Stephen P. Boyd, arguably the most influential figure in the field of convex optimization – the mathematical discipline that would later provide the foundational framework for all modern AI algorithms. While many of my EE colleagues focused on semiconductor design, I was drawn to the pure mathematical beauty of optimization theory. Finding Prof. Boyd at Stanford rather than MIT or Caltech was one of the most fortuitous accidents of my academic career. Though my passion was mathematics, I still needed to ground my work in electrical engineering, which led me to focus my thesis on “power and delay optimization of digital logic circuits using convex optimization.”
This intersection of theoretical mathematics and practical engineering would shape my entire career trajectory, beginning with my role at Samsung Electronics in December 2004. This position also served as my mandatory military service, as the research center was government-designated. At Samsung, I developed numerous optimization tools that dramatically improved productivity across semiconductor chip design, manufacturing processes, and testing operations.
The Industrial Impact – Optimization at Scale
Among hundreds of projects, here I will just state the first three that came to my mind now. The first was a comprehensive circuit optimization tool that could simultaneously optimize hundreds of parameters to optimize tens of objectives such as power consumption, delay times, circuit area, and signal duties. While (psychologists proved) humans can optimize at most 5 or 6 parameters at the same time, this tool could optimize hundreds of parameters to find the true optimum.
The second project revolutionized the next-generation DRAM cell scheme design by replacing intuition-based approaches with mathematically proven optimal solutions. Instead of relying solely on the experience of seasoned engineers with decades of expertise—which was amazing beyond my pure imagination—we could now computationally determine the optimal cell scheme for next-generation DRAM products. This capability could make or break years of global business strategy, translating into tremendous monetary value. The impact of this work resonated years later when I was being vetted for the role of founder and CTO of Gauss Labs, Inc., SK Hynix’s industrial AI spin-off, as I heard from the Vice Chair of SK Group, one of the largest conglomerates of South Korea.
The third project was perhaps the most forward-looking: a general-purpose machine learning (ML) and optimization software platform dubbed iOpt – yes, I was so into Apple products at that time, no other principal engineers or managers dared to use iPhones at work; they showed their loyalty by using Galaxy phones, but I rebelliously used iPhones AND named my proud ML and optimization software platform iOpt! – This platform compressed months-long projects into weeks by automatically modeling problems mathematically and finding optimal solutions. This work, though we didn’t realize it at the time, was already laying the groundwork for what would become modern AI applications. that compressed months-long projects into weeks by automatically modeling problems mathematically and finding optimal solutions. This work, though we didn’t realize it at the time, was already laying the groundwork for what would become modern AI applications.
The Rise of Machine Learning, hence AI
Around 2010, terms like “big data” and “ML” began emerging in technical discussions. I distinctly remember Prof. Boyd advising me to explore the potential of machine learning – advice that initially puzzled me but now seems prescient. While most of us were still focused on traditional optimization and engineering problems, he had already recognized that ML would transform the technology landscape. The transformation accelerated in 2012 with the rise of deep learning, marked by AlexNet’s revolutionary performance at the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC). This breakthrough represented the culmination of Geoffrey Hinton’s decades-long work on neural networks, validating his unwavering belief in their potential despite widespread skepticism. AlexNet demonstrated that deep neural networks could finally outperform human-engineered features in computer vision, opening the floodgates for applications in image generation, recognition, classification, and recommendation systems. What we didn’t realize then was that this breakthrough was merely a glimpse of the transformative capabilities that would emerge in the following years.
The Acceleration – From Transformers to AI Agents
The pace of innovation continued to accelerate. Shortly after I joined Amazon in July, 2017, a bunch of Googlers published their landmark paper “Attention is All You Need,” introducing the transformative Transformer architecture. The paper’s ambitious title proved prophetic - Transformers began outperforming previous sequence models almost immediately, long before ChatGPT would bring this technology into the mainstream in 2022. Then the era of LLMs and genAI began.1 Ever since, there have been unprecedentedly fast development and progress in these areas marked by the names of new versions of products or new products such as AlphaFold 2, DALL-E, DALL-E 2, DALL-E 3, Mistral AI, Optimus, GPT-4o, Figure AI, Claude Sonnet, and AlphaFold 3. Literrally, you would see new things coming beating the performance of the predecessors every week (without any bit of exeggerations). And just months ago, I started seeing “AI agents” as new buzzwords everywhere, marking yet another paradigm shift in how we think about artificial intelligence.
This evolutionary chain – represented by keywords from Big Data to Deep Learning, then to LLMs & genAI, and now to AI agents – represents more than just a progression of buzzwords. Each transition marked a fundamental shift in how we approach problem-solving with machines as well as the progress of AI technology. While the technical details of each breakthrough could fill volumes (and perhaps will in future posts), what's crucial for today's business leaders is understanding the practical implications of this rapid technological acceleration. As someone who has straddled both the theoretical foundations and practical applications of these technologies, I want to offer a strategic perspective for business owners navigating the AI landscape – especially those looking to integrate or build upon these transformative technologies. The insights I’ll share next come from witnessing firsthand how mathematical theories transform into market-moving innovations.
The Commoditization of AI Technology
What we’re witnessing now in the AI landscape might seem like an endless frontier of innovation, but a crucial pattern is emerging. The core AI technologies - from LLMs to generative AI models to AI agents - are rapidly becoming commoditized. This isn’t to diminish their revolutionary impact, but rather to highlight an important reality: mastery of these AI technologies is no longer a differentiator but a baseline requirement for competition.
Consider the current state of LLMs. While OpenAI, Anthropic, and Google made headlines with their initial releases, we’re now seeing capable models emerge from smaller players like Mistral AI & DeepSeek and even open-source initiatives. The barrier to entry isn’t necessarily technical knowledge anymore, but data! – together with differentiated insight and experience with domain knowledge or subject matter expertises!
The Real Competitive Advantage – RAG and Domain Knowledge
As someone who has witnessed firsthand the evolution from basic optimization problems to today’s sophisticated AI landscape, I see a clear parallel between my early work and the current state of AI. Just as my optimization tools at Samsung gained their true value by incorporating deep domain knowledge of semiconductor design, today’s most powerful AI applications will emerge from the fusion of general AI capabilities with specialized expertise.
Retrieval-Augmented Generation (RAG) represents this perfect synthesis. By combining the general capabilities of Large Language Models with domain-specific knowledge stored in vector databases, organizations can create AI applications that are both powerful and uniquely tailored to their needs. Vector databases serve as the modern equivalent of the specialized optimization constraints we used to define in my early career – they provide the crucial domain-specific boundaries that transform general-purpose tools into precision instruments.
Looking Ahead – The Data-First Future
As I reflect on my journey from optimizing circuit parameters to witnessing the rise of autonomous AI systems, one truth becomes increasingly clear: the future belongs to those who can effectively bridge the gap between general-purpose AI and specialized domain knowledge. Just as my work at Samsung showed how domain-specific optimization could transform semiconductor design, the next wave of innovation will come from those who can effectively combine general-purpose AI with their specialized knowledge bases.
The parallels between my early career and today’s AI landscape are striking. Twenty years ago, I was working to optimize hundreds of parameters simultaneously in circuit design – today, we’re using similar mathematical principles to optimize billions of parameters in neural networks. The key difference is scale, not fundamental approach. This perspective gives me unique insight into where the field is heading.
For business leaders, this means shifting focus from trying to compete on AI model development to investing in:
- Robust data infrastructure, including vector database implementations
- Effective RAG architectures, both software and hardware, that can leverage existing LLMs
- Tools and processes for maintaining and updating knowledge bases
- Expertise in combining domain knowledge with AI capabilities
Just as my optimization work gained its true value through deep integration with semiconductor expertise, tomorrow’s AI solutions will derive their power not from raw computational capability, but from their ability to meaningfully incorporate domain-specific knowledge and expertise. This is where I see the field heading – not toward bigger models, but toward smarter integration of specialized knowledge with general AI capabilities.
As I look back on my journey from pure mathematics to the frontiers of AI, I’m reminded that the most powerful innovations often come not from revolutionary new technologies, but from finding novel ways to combine existing tools with deep domain expertise. This was true in my optimization work two decades ago, and it remains true in today’s AI landscape. The future belongs not to those who can build the biggest models, but to those who can most effectively bridge the gap between general AI capabilities and specialized domain knowledge.
Sunghee Yun
Mathematician, Statistician, Theorist, Futurologist, and Digital Philosopher
- LLM stands for a large language model and genAI for generative AI. ↩