Advancing Multimodal Knowledge-Enhanced Large Language Models: Novel Methods and Innovative Applications.

The research develops advances for multimodal knowledge-enhanced large language models to improve reasoning on complex decision problems from incomplete, ambiguous or inconsistent information. Through dialogue with the user and revision of an external symbolic knowledge representation, the model asks targeted questions and guides the progressive clarification and reformulation of goals, constraints and preferences. Evidence, hypotheses, formulations, tool calls and intermediate results will be linked to provenance, dependencies and validity conditions, used to decide what to explore, verify, reuse or recompute, avoiding redundant processing. The study will test whether this lets small open-weight models on commodity hardware approach commercial models, measuring correctness, constraint satisfaction and solution quality at equal budget, with comparisons and ablation studies. Benchmark: energy-system design within the European valwarp project.