Closing the Neuro-Symbolic Loop: Active Correction with Spatial Logic and Automated Knowledge Refinement
| Typ | Masterarbeit | |
|---|---|---|
| Aushang | Master Thesis Closing the Neuro Symbolic Loop Active Correction with Spatial Logic and Automated Knowledge Refinement.pdf | |
| Betreuer | Wenden Sie sich bei Interesse oder Fragen bitte an: Nicolas Schuler (E-Mail: nicolas.schuler@kit.edu, Telefon: +49-721-608-46537), Vincenzo Scotti (E-Mail: vincenzo.scotti@kit.edu) |
Motivation
Deep learning models often rely on "reasoning shortcuts" and spurious correlations rather than causal mechanisms. While recent research proposes using Multimodal Language Models (MLMs) to translate visual explanations into symbolic logic for verification, the current approach acts only as a "passive verifier" and treats images as unstructured bags of features. This thesis aims to transform this framework into a dynamic, self-correcting system by introducing spatial reasoning to ensure features are geometrically consistent (e.g., ensuring ears are actually above the eyes). Furthermore, you will close the learning loop by implementing a "semantic loss" that uses logical reasoning to actively retrain the neural network, while simultaneously enabling the system to automatically refine its own Knowledge Base when it encounters new data.
Tasks
- Implement Spatial Logic: Extend the intersymbolic translation pipeline to extract spatial predicates (e.g., above, connected to) from heatmaps to prevent errors where features are present but spatially incoherent.
- Develop Active Feedback Loops: Create a training pipeline where the output of the abductive logic solver serves as a supervising signal (semantic loss) to fine-tune the underlying classifier and unlearn spurious correlations.
- Automate Ontology Alignment: Design an agentic workflow where the MLM can propose and implement updates to the Knowledge Base when consistent visual evidence contradicts the current static ontology.
Tools / Technology
Multimodal LLMs, Probabilistic Logic, Neuro-Symbolic AI, Python
Notes
- Thesis offered in German or English.
- Please contact Nicolas Schuler for more information.
- Gain experience with fundamental logical reasoning and ML in SE.
- Research integration: Aligns with ongoing work in AI-based reasoning systems.
- Close supervision: Regular guidance and access to relevant expertise.