Compositional Reasoning via Joint Image and Language Decomposition

Multimodal reasoning tasks such as visual ques- tion answering (VQA) require models to pro- cess both language and visual inputs. How- ever, existing approaches typically decompose only language queries, treating images as mono- lithic inputs. We introduce REDI, a framework that jointly decomposes both images and ques- tions into visual sub-domains (segmentation, material, depth, and color) with corresponding sub-questions. REDI uses an MLLM orchestra- tor to select the sub-domains required for each query, generate domain-specific sub-questions with grounded object references (via shared object labels), and fuse worker outputs via consistency-aware aggregation (verify–refine– override) to produce the final answer. This hierarchical multi-agent design mitigates er- ror propagation and improves compositional reasoning across both open- and closed-source MLLMs. On SEEDBench, MMBench, and CLEVR, REDI achieves absolute accuracy im- provements of 8.9%, 8.2%, and 16.0% over base model. Project webpage: https:// madhav-kanda.github.io/redi/