Murtaza Nikzad

I train and fine-tune language models, run reinforcement learning on them, and build the harnesses they act through.

Computer science and neuroscience at Davidson College, class of 2027. Bonner Scholar and Alvarez Scholar. This summer I am in San Francisco working on ML/agents research.

Now

The paper

Forward versus Backward: Comparing Reasoning Objectives in Direct Preference Optimization

I fine-tuned Llama 3.1 8B-Instruct with DPO on GSM8K under two objectives. Forward learns to generate the chain of thought. Backward learns to verify: given a problem and a candidate answer, the model judges pass or fail.

Preference optimization makes models more confident and less willing to admit their own errors, even as accuracy improves.

Baseline Forward Backward
Accuracy 83.1 86.6 83.6
Acknowledgement 67.8 44.7 46.3
False positives 13.4 4.3

All numbers are percentages on GSM8K. Acknowledgement is how often the model flags its own wrong answers.

Earlier: Transfer Learning with PointNet for Time Projection Chambers, a poster with R. Doctor, M. P. Kuchera, R. Ramanujan, D. Bazin, and Y. Ayyad at the Davidson Summer Research Symposium.

Selected work

Five hackathon wins overall: Cal Hacks, HackHarvard, VandyHacks, HackNC, and HopHacks.

Elsewhere

Résumé · GitHub · Twitter · LinkedIn · munikzad@davidson.edu