Will ‘AI Factory’ replace ‘Software Factory’ as predicted by Nvidia CEO Jensen Huang?

The COMPUTEX 2024 took place in Taiwan during the week of 4th June, and I witnessed an extremely interesting phenomenon.

The CEOs of three of the leading AI infrastructure enablers, Jensen Huang of Nvidia, Lisa Su of AMD, and Charles Liang of Supermicro, all born in Taiwan, delivered insightful and thought-provoking speeches about how AI, especially generative AI, will disrupt not just the IT industry but across all sectors.

Nvidia CEO Jensen Huang delivered the most notable keynote address, predicting an AI Factory will replace the current Software Factory:

1. Software Factory consists of pre-programmed software applications that retrieve defined information.
2. Using Nvidia GPU technology, which exponentially scales computation power, generative AI can dynamically generate information and output based on prompts. Therefore, Software Factory will no longer be relevant.
3. The Nvidia Inference Microservice (NIM) provides pre-trained models in optimized containers to accelerate the creation of generative AI applications.

As a result of Huang's keynote speech, I witnessed an interesting phenomenon.

During my subsequent meetings with CIOs of leading high-tech manufacturing companies, many told me that shortly after Huang delivered his keynote address, their CEOs called them with questions about whether their current IT investments, initiatives, operations, resource plans, etc., would be disrupted by generative AI as Huang predicted.

In fact, Huang’s view aligned with what we discussed in my prior post – why AI needs a new era of probabilistic adaptability?

1. Nearly all of today’s IT systems and software applications are inherently deterministic, i.e., build-to-spec based on well-understood requirements to deliver pre-defined outcomes.
2. With AI can continually uncover new findings and generative AI can start with a prompt and dynamically generate information or output, i.e., non-predefined and non-predictable, IT systems and applications must become probabilistic-capable to stay relevant in the era of generative AI.

This raises a few questions every business and tech executive should ask.

1. Can we protect our existing IT investment from being disrupted immediately, as Huang predicted, and still cost-effectively practice AI?
2. How can we avoid the need for rip-and-replace by making the current inherently deterministic IT systems smarter, i.e., becoming probabilistic-ready to work with the dynamic nature of generative AI and enabling the gradual build-out of Huang's 'AI Factory' vision?
3. How can we ensure all future applications, automation, and orchestrations are AI and generative AI-ready?
4. How can effective governance be used to ensure boundaries and goals when using generative AI? Let's brainstorm and share perspectives.