Ex-Meta scientists want to bring visual AI to the factory floor

Artificial intelligence has fundamentally reshaped our digital landscape, yet its influence on the physical world remains in its infancy. A new wave of startups is now working to bridge this gap, moving AI from screens and servers into the tangible, chaotic environments of the real world. Among the most promising of these is Perceptron, a startup founded in November 2024 by two former research scientists from Meta’s Fundamental AI Research (FAIR) division, Armen Aghajanyan and Akshat Shrivastava.

The company is setting its sights on the industrial sector, aiming to provide machines with the cognitive capacity to navigate and manipulate their surroundings. This week, Perceptron unveiled its latest breakthrough: Isaac 0.5, a frontier vision model engineered to help robots "perceive, reason, and act" within complex spaces like warehouses and manufacturing plants.

Bridging the Gap in Physical AI

The founders argue that the current state of robotics is hampered by a "false choice" between two extremes. On one hand, developers rely on generalist foundation models that demand massive, expensive cloud GPU resources for every operation. On the other, they utilize narrow, specialized models that handle either perception or control, but never both simultaneously.

Perceptron’s solution is designed to be a general-purpose, flexible intelligence layer. Rather than being hard-coded for a single, repetitive motion, Isaac 0.5 is built to adapt to the nuances of its environment.

"Physical AI today forces a false choice: generalist foundation models that need multiple dedicated cloud GPUs for every instance, or narrow models that handle perception or control, but never both," the company stated.

To illustrate the complexity of this challenge, Shrivastava points to the seemingly simple task of sorting boxes. A robot must perform a sequence of sophisticated actions: identifying labels, conducting spatial analysis to locate objects, determining the optimal pick-up order, and executing the physical movement. Isaac 0.5 is designed to orchestrate this entire workflow, providing a level of flexibility that has been largely absent from industrial automation until now.

Data-Driven Intelligence

The "algorithmic alchemy" behind Isaac 0.5 is fueled by a massive ingestion of video data. To teach the model how to interpret physical scenarios, Perceptron trained the system on over one million hours of general video content.

The training regimen also incorporates two specialized types of data:

• Ego video: Footage captured from the perspective of a human performing a task, often via wearable cameras or GoPros.
• UMI video: Recordings of repetitive human actions used to teach AI systems how to replicate specific physical movements.

While the company has kept the specific origins of its training data private, Shrivastava confirmed that Perceptron has developed petabyte-scale datasets internally. These datasets span multiple modalities, ranging from text and images to complex robotic trajectories.

Open-Weight Innovation

In a move to foster transparency and accelerate adoption, Perceptron is releasing Isaac 0.5 as an open-weight model. By making the parameters and training materials accessible, the startup invites the research community to inspect and build upon their work.

The company envisions its software being integrated into a diverse array of sectors, including:

• Logistics and Warehousing
• Manufacturing
• Security and Surveillance
• Mobility
• Media and Entertainment

"Nothing like this really exists out there," said Aghajanyan. "We’re really excited about it."

Future Outlook

Perceptron’s ambitions are backed by significant financial momentum. According to Pitchbook, the startup secured $16 million in 2024 from high-profile investors including Bessemer Venture Partners, The Explorer Fund, and SmartGateVC. Reports indicate that the company is currently in the process of finalizing an additional funding round to scale its operations.

As the industry moves toward more autonomous, vision-guided robotics, Perceptron’s focus on general-purpose intelligence could prove to be a critical turning point. By moving beyond the limitations of narrow, task-specific software, the team is positioning itself to become a foundational player in the next generation of industrial automation.