Salesforce AI Introduces TACO: A New Family of Multimodal Action Models that Combine Reasoning with Real-World Actions to Solve Complex Visual Tasks

Developing effective multi-modal AI systems for real-world applications requires handling diverse tasks such as fine-grained recognition, visual grounding, reasoning, and multi-step problem-solving. Existing open-source multi-modal language models are found to be wanting in these areas, especially for tasks that involve external tools such as OCR or mathematical calculations. The abovementioned limitations can largely be attributed to single-step-oriented datasets that cannot provide a coherent framework for multiple steps of reasoning and logical chains of actions. Overcoming these will be indispensable for unlocking true potential in using multi-modal AI on complex levels.

https://www.marktechpost.com/2025/01/12/salesforce-ai-introduces-taco-a-new-family-of-multimodal-action-models-that-combine-reasoning-with-real-world-actions-to-solve-complex-visual-tasks/