Ananjan Nandi

Selected Publications

1. Sneaking Syntax into Transformer Language Models with Tree Regularization

   Ananjan Nandi, Christopher D Manning, and Shikhar Murty

   In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) Apr 2025

   Abs Bib

   While compositional accounts of human language understanding are based on a hierarchical tree-like process, neural models like transformers lack a direct inductive bias for such tree structures. Introducing syntactic inductive biases could unlock more robust and data-efficient learning in transformer language models (LMs), but existing methods for incorporating such structure greatly restrict models, either limiting their expressivity or increasing inference complexity. This work instead aims to softly inject syntactic inductive biases into given transformer circuits, through a structured regularizer. We introduce TreeReg, an auxiliary loss function that converts bracketing decisions from silver parses into a set of differentiable orthogonality constraints on vector hidden states. TreeReg integrates seamlessly with the standard LM objective, requiring no architectural changes. LMs pre-trained with TreeReg on natural language corpora such as WikiText-103 achieve up to 10% lower perplexities on out-of-distribution data and up to 9.5 point improvements in syntactic generalization, requiring less than half the training data to outperform standard LMs. TreeReg still provides gains for pre-trained LLMs: Continued pre-training of Sheared Llama with TreeReg results in improved syntactic generalization, and fine-tuning on MultiNLI with TreeReg mitigates degradation of performance on adversarial NLI benchmarks by 41.2 points. We release all code to guide future research.

   @inproceedings{nandi-etal-2025-sneaking,
     title = {Sneaking Syntax into Transformer Language Models with Tree Regularization},
     author = {Nandi, Ananjan and Manning, Christopher D and Murty, Shikhar},
     editor = {Chiruzzo, Luis and Ritter, Alan and Wang, Lu},
     booktitle = {Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)},
     month = apr,
     year = {2025},
     address = {Albuquerque, New Mexico},
     publisher = {Association for Computational Linguistics},
     url = {https://aclanthology.org/2025.naacl-long.407/},
     doi = {10.18653/v1/2025.naacl-long.407},
     pages = {8006--8024},
     isbn = {979-8-89176-189-6},
   }

2. h4rm3l: A Language for Composable Jailbreak Attack Synthesis

   Moussa Koulako Bala Doumbouya, Ananjan Nandi, Gabriel Poesia, and 5 more authors

   In The Thirteenth International Conference on Learning Representations Apr 2025

   Bib

   @inproceedings{doumbouyah4rm3l,
     title = {h4rm3l: A Language for Composable Jailbreak Attack Synthesis},
     author = {Doumbouya, Moussa Koulako Bala and Nandi, Ananjan and Poesia, Gabriel and Ghilardi, Davide and Goldie, Anna and Bianchi, Federico and Jurafsky, Dan and Manning, Christopher D},
     booktitle = {The Thirteenth International Conference on Learning Representations},
   }

3. Roleplay-doh: Enabling domain-experts to create llm-simulated patients via eliciting and adhering to principles

   Ryan Louie, Ananjan Nandi, William Fang, and 3 more authors

   arXiv preprint arXiv:2407.00870 Apr 2024

   Bib

   @article{louie2024roleplay,
     title = {Roleplay-doh: Enabling domain-experts to create llm-simulated patients via eliciting and adhering to principles},
     author = {Louie, Ryan and Nandi, Ananjan and Fang, William and Chang, Cheng and Brunskill, Emma and Yang, Diyi},
     journal = {arXiv preprint arXiv:2407.00870},
     year = {2024},
   }

4. CTC-DRO: Robust Optimization for Reducing Language Disparities in Speech Recognition

   Martijn Bartelds, Ananjan Nandi, Moussa Koulako Bala Doumbouya, and 3 more authors

   Apr 2025

   Bib

   @misc{bartelds2025ctcdrorobustoptimizationreducing,
     title = {CTC-DRO: Robust Optimization for Reducing Language Disparities in Speech Recognition},
     author = {Bartelds, Martijn and Nandi, Ananjan and Doumbouya, Moussa Koulako Bala and Jurafsky, Dan and Hashimoto, Tatsunori and Livescu, Karen},
     year = {2025},
     eprint = {2502.01777},
     archiveprefix = {arXiv},
     primaryclass = {cs.LG},
   }

5. Simple Augmentations of Logical Rules for Neuro-Symbolic Knowledge Graph Completion

   Ananjan Nandi, Navdeep Kaur, Parag Singla, and 1 more author

   In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) Apr 2023

   Bib

   @inproceedings{Nandi_2023,
     title = {Simple Augmentations of Logical Rules for Neuro-Symbolic Knowledge Graph Completion},
     url = {http://dx.doi.org/10.18653/v1/2023.acl-short.23},
     doi = {10.18653/v1/2023.acl-short.23},
     booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
     publisher = {Association for Computational Linguistics},
     author = {Nandi, Ananjan and Kaur, Navdeep and Singla, Parag and -, Mausam},
     year = {2023},
     pages = {256–269},
   }