The practical question in enterprise AI is no longer whether a bigger model is available. It is whether each workload is being sent to the right combination of model, memory, processor and deployment environment. That matters for teams evaluating computer vision and natural language processing, but the immediate lesson is broader: AI infrastructure must be designed around workload fit, not a single default system.
Intel’s position at the Open Compute Project Global Summit was clear. Deep research may justify a frontier model, while routine coding, summarisation and operational tasks may be better suited to an open model running closer to enterprise data. The decision depends on required intelligence, context length, latency, data sensitivity, sovereignty, cost and energy use.
Why computer vision and natural language processing need workload-aware design
For several years, AI infrastructure planning has leaned toward larger models, larger clusters and larger capital commitments. That remains relevant for demanding workloads, but it creates a poor operating assumption when every request is treated as equally complex.
The alternative is an orchestration layer that evaluates the request before assigning compute. A request involving deep research can follow a different path from routine summarisation. Proprietary documents may need to remain inside the enterprise, while less sensitive work may use an open model. This is not simply a model-selection decision; it is a policy for deciding where intelligence runs and what constraints take priority.
Inference is central to that policy. Training produces a model, but inference is the repeated process of serving requests from it. As usage expands across enterprise workflows, serving economics, response time and data handling become architecture concerns rather than implementation details.
Open-weight models are also widening the available choices. Data cited from Vercel AI Gateway showed their share of observed token traffic rising from roughly 31% in early July to about 78% by late September. Figures mentioned are indicative and may vary depending on gateway coverage, workload mix, measurement method and market conditions.
That movement does not establish that an open model is suitable for every task. It does show why buyers need a routing policy and evaluation process instead of assuming that one model should receive every request.
The hardware trade-off is capacity, cost and deployment fit
Intel’s D-series GPU design, formerly known as Crescent Island, reflects this middle-ground approach. It is built on the Xe3P architecture and specified at 1.3 petaflops of FP8 compute and 1.5 terabytes per second of memory bandwidth in a 350-watt, air-cooled PCIe form factor. It supports up to 480 gigabytes of LPDDR5X memory, while Intel’s branded card carries 160 gigabytes. Figures mentioned are indicative and may vary by configuration, workload, cooling design and supplier assessment.
The important choice is not only peak compute. Intel selected LPDDR memory rather than pursuing a design centred on expensive, supply-constrained high-bandwidth memory. The trade-off is lower emphasis on peak bandwidth in exchange for memory capacity and cost considerations.
That trade-off is relevant to inference because the model and its context must fit into available memory. Intel says the design supports models up to 300 billion parameters on a single card and more than 1 trillion in an eight-GPU system. Figures mentioned are indicative and depend on model architecture, context requirements, quantisation and system configuration.
The deployment path is equally important. A company can begin with a card in a standard air-cooled server, move to an eight-way server as demand develops, and consider rack-level systems only when the workload requires that scale. This may simplify adoption, but buyers still need to test throughput, latency, software compatibility, cooling requirements and total operating cost on their own workloads.
Why agentic AI changes the CPU conversation
An AI agent is not limited to generating tokens. It may retrieve information, reason over context, call a tool, query a database and execute an application step. Those activities require general-purpose processing alongside model inference.
Intel’s rack strategy therefore places Xeon CPUs beside inference hardware. In the proposed division of work, CPUs handle orchestration, data processing and tool execution; the D-series handles open-model inference; and a specialised token engine can be added when response speed is the priority.
Intel showcased Xeon 6+ configurations scaling beyond 46,000 cores per rack at 133.45 kilowatts. Figures mentioned are indicative and may vary by rack configuration, cooling design, workload and deployment conditions.
This changes how infrastructure should be assessed. Core count alone may not describe the system’s usefulness. A more relevant operational question is how many concurrent agents the rack can sustain at the required latency, while the surrounding data movement, memory, model serving and application execution remain stable.
For agentic systems, the unit of analysis is the complete workflow, not the model call in isolation.
A practical framework for choosing an AI system
Founders and technology leaders should map the workload before choosing hardware or committing to a model strategy. A useful decision process is:
- Classify the work. Separate deep reasoning, retrieval-heavy requests, routine generation, data processing and tool execution.
- Identify the binding constraint. Decide whether answer quality, latency, privacy, sovereignty, energy use, capacity or operating cost is most important for that workload.
- Match the deployment environment. Check whether the data must remain within the enterprise and whether an air-cooled server is sufficient for the expected load.
- Measure the entire path. Include retrieval, database queries, tool calls, retries, CPU processing and application execution—not only token throughput.
- Plan the next step. Confirm that the software and operating model can move from one card to multiple cards or rack-level capacity when demand justifies it.
What caught my attention is the shift in the buyer’s unit of analysis. The decision is no longer just which model to call. It is how to distribute work across models, CPUs, GPUs, memory and enterprise applications while preserving the required controls.
What founders should build around the workload layer
The opportunity is not necessarily another general-purpose model. The more immediate problem is making mixed models and mixed hardware usable inside a real enterprise environment.
That points toward systems that evaluate requests using the constraints already identified: intelligence required, context length, latency, data sensitivity, sovereignty, cost and energy. The product value would sit in workload placement and operational control, while the underlying model or accelerator remains one component of a larger system.
Founders should be careful not to turn this into a simplistic “small models win” thesis. A frontier model may still be appropriate for complex research. An open model may be preferable for routine or sensitive work, but only after testing quality, context handling, throughput and compatibility with the surrounding application.
The strategic effect is straightforward: more varied enterprise inference creates more varied infrastructure requirements. More varied requirements increase the importance of orchestration, memory capacity, CPU processing and deployment flexibility. That gives builders a concrete problem to solve without assuming that every customer needs the same model or hardware configuration.
How to apply the framework beyond this decision
Teams searching for “prompt engineering” may be looking for ways to improve request quality, but better prompts do not remove the need to choose the right execution path.
People evaluating “ai agents” should measure tool calls, database access and application execution alongside generated tokens.
A buyer considering “penetration testing” should treat the workload’s data sensitivity and review requirements as design inputs rather than assuming that a general AI endpoint is appropriate.
For “reinforcement learning”, the relevant lesson here is not a claim about training methods; it is the value of defining the workload and its operating constraints before selecting compute.
If you typed “ci cd pipeline” into a search engine, you are probably looking for deployment automation. The same discipline still applies: separate the work involved, identify its constraints and avoid treating one default system as suitable for every step.
Explore more practical perspectives on AI, automation and product execution on the Yanisa Execution blog when comparing architecture options and execution priorities.
Rightsized AI does not mean choosing the least capable system. It means matching capability, memory, processing and deployment conditions to the work being performed. The teams that make that matching operational may shape the next phase of enterprise AI more than teams focused only on model scale.
Frequently Asked Questions
What does rightsized AI infrastructure mean?
It means matching a workload with the model, memory, processor and deployment environment suited to its required quality, latency, data handling and operating constraints. A complex research task may use a different system from routine coding or summarisation.
Why is inference important for enterprise AI?
Inference is the repeated process of serving requests after a model has been trained. As enterprise usage grows, the cost, latency, memory requirements and data-handling choices involved in serving those requests become important architecture decisions.
Why do AI agents need CPUs as well as GPUs?
Agents can retrieve information, call tools, query databases and execute application steps in addition to generating tokens. CPUs can handle much of that general-purpose processing while GPUs support model inference.
What is the LPDDR trade-off in Intel’s D-series design?
LPDDR reflects a design choice that places more emphasis on memory capacity and cost considerations than on peak bandwidth. Whether that trade-off is suitable depends on the model, context size, throughput target and deployment environment.
How should a company decide between a frontier model and an open model?
Start by classifying the workload and identifying its main constraint, such as reasoning quality, latency, privacy, sovereignty, energy use or cost. Then test the available options across the complete workflow, including retrieval, tool calls, CPU processing and application execution.