Researchers from Caltech, NASA JPL, UCLA, and Okean Solutions are developing AI Space Cortex, an AI-assisted autonomy framework for robotic sample handling on alien surfaces. The system integrates hierarchical control, active vision scene analysis, real-time telemetry assessment, self-diagnosis, and recalibration to run science missions autonomously while monitoring system health. The effort is part of NASA's REASIMO work and funded under the COLDTech program to support future missions such as a potential Europa Lander. The prototype used GPT-4o via API and foundational computer vision models; future deployments aim to run optimized local models within strict power and inference-speed tradeoffs.
"Unlike previous approaches that largely rely on deterministic task execution, the AI Space Cortex integrates hierarchical control, active vision-based scene analysis, real-time telemetry assessment, self-diagnosing, and recalibration capabilities," explains the preprint paper that describes the project. AI Space Cortex's designers envision a robotic system that autonomously runs science missions in real-time while checking on its own health. To do so, it relies on computer vision and large language models.
To do so, it relies on computer vision and large language models. For this project, the model consisted of OpenAI's GPT-4o, accessed via API rather than running locally. Future implementations, the paper explains, will explore local model deployment. Doing so will require model size optimizations and balancing inference speed with energy consumption given planetary landers operate under strict power constraints.
Collection
[
|
...
]