I'm a third-year Information Technology student at DJSCE, Mumbai, and most of my time outside class goes into machine learning research. I like questions where you can run an experiment and find out whether your explanation was right.
So far that has taken me into how transformers are built, how vision models keep track of position, whether video world models actually use the futures they imagine, and how to find cause and effect in time-series data. I work on these with research groups in several countries.
- Research Collaborator, Mechanistic Interpretability Research Group, In Simplicity Technologies: causal experiments on nonlinear query projections in attention, across GPT-2 Small, GPT-2 Medium and GPT-3-Large.
- Open-source collaborator, CausationEntropy at Clarkson University's C3S2: information-theoretic estimators for causal network discovery from multivariate time series.
- Founding research member, Neural Representation Dynamics Lab: positional encodings in Vision Transformers, and world action models.
- Research contributor, StationHouse: ConspiracyBench, and LLM Consciousness Self-Attribution.
- Research lead, DJ InIT.ai, the machine-learning research club at DJSCE.
| Status | Paper | Venue |
|---|---|---|
| Accepted | Beyond Linearity in Attention Projections co-second author |
NeurIPS 2026 Workshop SLMs for Agentic Systems |
| Under review | Anchored but Not Protected: When Refusal Erodes During Fine-Tuning, and Whether Holding One Direction Stops It co-first author |
ICLR 2027 |
| Under review | How Vision Transformers Represent Position: A Mechanistic Comparison of Absolute and Rotary Positional Encodings | ICLR 2027 |
Nine more papers and projects are in progress, targeting ICML, CVPR, ICLR workshops and JMLR. See them on my website →
| Project | What it is | Result |
|---|---|---|
| IOI Circuit Analysis | Replication of the Indirect Object Identification circuit in GPT-2 Small with path patching, plus a stability test under distractor tokens | ρ = 0.781 head-ranking correlation across templates; 98.2% of logit difference retained |
| CBAM-ResNet50 | Channel and spatial attention (CBAM) inside a CIFAR-adapted ResNet-50 | 94.65% validation accuracy vs. 88.49% baseline, with +0.77% parameters |
| GPT from Scratch | A ~30M-parameter decoder-only GPT, trained on ~800M tokens with mixed precision on free-tier T4 GPUs | Validation loss 4.63 (perplexity ~103) |
More on my GitHub
- swin-crowd-counting: Swin Transformer + FPN decoder for crowd density estimation on ShanghaiTech (MAE 41.45 on Part B)
- convnext-crowd-counting: ConvNeXt-based crowd density estimation on ShanghaiTech
- ML-Based_Volatility_Prediction: predicting short-term high-volatility periods in large technology stocks
- AlphaAgent: a multi-agent investment research terminal, with a ReAct agent, a bull/bear debate and an arbiter (LangGraph + Groq)
Coursework
| Course | Provider |
|---|---|
| Machine Learning Specialization | DeepLearning.AI & Stanford (Andrew Ng) |
| Deep Learning Specialization | DeepLearning.AI (Andrew Ng) |
| Natural Language Processing Specialization | DeepLearning.AI |
| Introduction to Large Language Models | NPTEL, IIT Delhi & IIT Bombay |
| Neural Networks for Computer Vision and NLP | NPTEL, IIT Guwahati |
| CS231n: Deep Learning for Computer Vision | Stanford (lectures online) |
Have a research question, a collaboration or a role in mind? I'd be happy to talk: parvamehta26@gmail.com