Amazon Web Services has unveiled Nova Forge, a specialized development environment designed to customize frontier models without the side effect of "catastrophic forgetting." When developers train an AI on specific company data—a process called supervised fine-tuning—the model often becomes a niche expert but loses its ability to follow basic instructions or reason through general logic. Nova Forge solves this by allowing users to mix their proprietary datasets with Amazon’s curated training data, ensuring the model remains versatile.

In a rigorous evaluation involving 16,000 customer comments, the AWS team tested Nova 2 Lite against open-source competitors. The challenge required categorization of feedback into a complex hierarchy of over 1,400 specific labels. Without customization, even powerful models struggled with such granular detail. However, after using Nova Forge’s full-rank fine-tuning, the model’s accuracy improved by 17%.

Crucially, the model retained its general knowledge. While standard fine-tuned models saw their scores on the Massive Multitask Language Understanding (MMLU) benchmark—a test spanning subjects like history and math—plummeted, the data-mixed Nova model stayed within one percent of its original performance. This allows enterprises to deploy a single model that understands both internal logistics and general professional communication, streamlining operations.

Amazon Web Services has unveiled Nova Forge, a specialized development environment designed to customize frontier models without the side effect of "catastrophic forgetting." When developers train an AI on specific company data—a process called supervised fine-tuning—the model often becomes a niche expert but loses its ability to follow basic instructions or reason through general logic. Nova Forge solves this by allowing users to mix their proprietary datasets with Amazon’s curated training data, ensuring the model remains versatile.

In a rigorous evaluation involving 16,000 customer comments, the AWS team tested Nova 2 Lite against open-source competitors. The challenge required categorization of feedback into a complex hierarchy of over 1,400 specific labels. Without customization, even powerful models struggled with such granular detail. However, after using Nova Forge’s full-rank fine-tuning, the model’s accuracy improved by 17%.

Crucially, the model retained its general knowledge. While standard fine-tuned models saw their scores on the Massive Multitask Language Understanding (MMLU) benchmark—a test spanning subjects like history and math—plummeted, the data-mixed Nova model stayed within one percent of its original performance. This allows enterprises to deploy a single model that understands both internal logistics and general professional communication, streamlining operations.