7 Key trends shaping gen AI in 2025
Domain-specific models
Gartner predicts that by 2027, more than 50% of the gen AI models used by enterprises will be tailored to specific industries or business functions, massively more than just 1% seen in 2023.
While foundation models are trained with colossal amounts of general knowledge data to be proficient on a wide range of tasks, domain-specific models are fine-tuned to be specialized for a particular field or use case, and trained on more limited datasets.
Domain-specific models offer several advantages:
- They are significantly smaller in terms of parameters, storage, and memory requirements, and require less human annotation and validation to be trained.
- They are more accessible for companies because fine-tuning costs are more affordable.
- They are less general but deliver better performance for the tasks they have been tuned for, reducing the likelihood of hallucinations, and improving accuracy and reliability.
- Because they narrow the scope, these models are able to understand the nuances, context, and jargon of a specific industry, so the output is much more precise.
“We’ve seen how smaller models that are trained on higher-quality data can outperform larger models trained on lower-quality data.”
— Percy Liang, Computer Science Professor at Stanford University, “We could run out of data to train AI language programs,” MIT Technology Review
As the demand for smaller, domain-specific models grows, so does the evidence of their exceptional performance in specialized fields such as e-learning, aeronautics, or use cases like biomedical text translation.
However, they pose a new challenge. Domain-specific models often require extremely high data accuracy, highlighting the need for high-quality training data and domain experts in the data annotation process.
Improving model training efficiency
Cost and speed are two big roadblocks for companies aiming to train and deploy LLMs and gen AI models. These models demand substantial financial resources: For instance, training Open AI’s GPT-3 cost over US$4 million, while Meta’s LLaMA required an investment of more than US$2.4 million. Energy consumption is an additional concern. A MIT Technology Review Insights report highlights that training an AI model can consume more electricity than 100 average U.S. homes in a year.
“Companies will need to ramp up their attention to efficiency while startups and researchers are pioneering new ways to improve AI model efficiency.”
— Adam Green, “The great acceleration: CIO perspectives on generative AI,” MIT Technology Review
Advances in software and hardware can gradually reduce costs, making systems more efficient. However, speed is the key variable that can truly drive significant improvements.
“Speed is really accelerating thanks to generative AI, and scaling faster is a growing demand from companies,” says Jean-Claude Junqua, Executive Senior Advisor at Sigma AI. In the data annotation field, companies can achieve this by “creating fast annotation cycles, bringing automation into the workflow, and having the right processes, knowledge, and expertise in place to hit the ground running.”
Multimodal gen AI
Multimodal gen AI — that is, training gen AI models and LLMs with a combination of different data types — is gaining significant momentum. These models are capable of seamlessly integrating and creating outputs from images, text, video, audio, or other data sources.
As IBM highlights in “The most important AI trends in 2024,” “The most immediate benefit of multimodal AI is more intuitive, versatile AI applications and virtual assistants. Users can, for example, ask about an image and receive a natural language answer, or ask out loud for instructions to repair something and receive visual aids alongside step-by-step text instructions.”
In 2023, Google introduced Gemini, a native multimodal model capable of understanding and operating across different types of information. For example, it can extract text from infographic content, summarize a podcast, and even correct mathematical assignments based on visual input.
That same year, Meta launched ImageBind, which connects different forms of information such as images, video, audio, text, depth, thermal, and inertial measurement units (IMUs). For instance, ImageBind can suggest audio by using images or video as an input, or use audio prompts to generate images.
However, one of the major challenges in building multimodal models is the scarcity of multimodal data. Compared to the vast amount of text data available, the volume of data containing more than one modality is significantly lower. Plus, it is costly, particularly in some fields such as biomedical.
Synthetic data
Using gen AI to produce synthetic data is also a rapidly evolving field. This artificially generated data resembles real-world data and can help companies train machine learning models. Gartner predicts that “by 2026, 75% of businesses will use generative AI to create synthetic customer data, up from less than 5% in 2023.”
Synthetic data offers a valuable solution in scenarios where real data is costly, unavailable, imbalanced, or restricted by privacy concerns. This presents new opportunities for businesses developing gen AI models by lowering costs, providing more diverse samples for model training, and overcoming data limitations, especially in highly regulated industries.
“The creation of synthetic data is gaining traction in fields such as finance, healthcare, and defense,” explains Jean-Claude Junqua. “We are seeing more projects involving synthetic data and privacy. One of the main challenges of such projects is often to capture key characteristics of the real data and generate synthetic data that closely resembled the original data without compromising privacy,” he says.
The use of synthetic data is also key to addressing a major concern: the lack of data needed to train foundation models. The shortage of high-quality training data hinders the ability to build larger, more powerful models. As this data gap increases, companies like NVidia and xAI are exploring the use of synthetic data (generated from a computer simulation or gen AI) to overcome these limitations.
Autonomous AI agents
Autonomous agents are taking AI technology to a new level, by performing multiple tasks, making decisions, and engaging with their surroundings independently, without requiring any human supervision or assistance.
Unlike traditional AI systems, specialized in specific tasks, autonomous AI agents are versatile, learning from experience, planning ahead, and adapting to solve complex problems. Examples of autonomous agents include personal assistants (such as Siri or Alexa), self-driving cars, personal health assistants, delivery drones, and autonomous robots.
Many autonomous AI agents are built upon foundation models. For example, AutoGPT relies on GPT-4. Instead of requiring humans to input prompts, these agents are designed to execute tasks from end to end.
Data selection
Domain-specific models are emerging as a faster and more efficient solution for companies seeking to adapt LLMs to highly specialized tasks. These models require significantly less training data, prompting a crucial question: How can we select the most effective data to train these smaller models?
While data selection has always been a challenge in machine learning, it has become even more relevant in the era of large language models.
Understanding the task that a model needs to perform is crucial for effective data selection. This involves defining specific requirements to ensure the data fits the desired model outcomes.
Companies can use various methods to select the right data. For example:
- Distribution matching: This approach selects data that closely aligns with the desired target distribution, ensuring the model is well-equipped to handle real-world scenarios.
- Distribution diversification: By prioritizing heterogeneity, this method helps to prevent overfitting and improve model generalization by exposing the model to a wider range of data points.
Ethical concerns about gen AI
The rise of generative AI has sparked intense ethical debates, with concerns about bias, the spread of misinformation, manipulated data, copyright, and data privacy taking center stage. This ongoing conversation is pivotal in terms of how to build a responsible AI that truly serves humans.
“The fact that generative AI is a predictive technology — meaning that it can create things that do not exist — makes quality even more important. If training data doesn’t have the highest quality, the risk of hallucinations, misinformation, or propagating biases is much higher.”
— Jean-Claude Junqua, Executive Senior Advisor at Sigma AI
Globally, people are discussing emerging regulatory frameworks to establish governance strategies for the development and use of AI, mitigating potential risks. On August 1, 2024, the European Artificial Intelligence Act (AI Act) provided a uniform framework for safe AI across all EU countries.
This framework focuses on transparency, accuracy, and explainability, especially for high-risk AI systems, such as those used in private and public services, justice and democratic processes, employment, and other use cases involving the lives and rights of people.
These systems are subject to strict obligations, including:
- Having risk assessment and mitigation systems
- Requiring high-quality training data, to minimize risks and discriminatory outcomes
- Having appropriate human oversight to reduce risks.
All forms of AI, including generative AI, have the potential to transform our society, but only if trained and used ethically. “We must act responsibly to ensure AI benefits everyone,” Junqua explains. “This means considering not only customer needs but also the well-being of humanity.”