Projects

Collected teleoperated demonstrations through HuggingFace LeRobot and trained an ACT (Action Chunking with Transformers) policy for screwdriver pick-and-place from a single front-facing camera — no depth sensing, no explicit pose estimation, no scripted motion. Training was run on an AWS EC2 GPU instance and the resulting policy deployed back on the physical SO-101 arm.

Fine-tuned OpenVLA, a 7B-parameter vision-language-action model, on the LIBERO manipulation benchmarks using LoRA adapters, deployed on UMich HPC (NVIDIA A40). Training loss dropped from 19.08 to 1.16 over three epochs. The project explores how far general-purpose VLA foundation models can be adapted to specific manipulation skills with parameter-efficient tuning instead of full retraining.

Evaluated LeWM, a JEPA-based world model, replacing its SIGReg anti-collapse regularizer with free-energy and coding-rate objectives. Joint-embedding predictive architectures learn to predict in latent space rather than pixel space, which makes the choice of anti-collapse term the central design decision. On a two-room navigation dataset, the coding-rate variant lifted hard-protocol accuracy — 100-step planning — from 16% to 75%.

With Prof. Raj Rao Nadakuditi, University of Michigan

Showed that a free-energy (Gaussianizing) loss improves both reconstruction and classification across CNNs, MLPs, and autoencoders, and built a joint Gaussianizing classifier on top of it. Released as an open-source package with a Julia implementation of the matricial free-energy loss alongside the Python version.

Integrated Cartographer SLAM with Grounding DINO in ROS2 and Gazebo. As the robot maps the environment, the open-vocabulary detector populates a persistent semantic map, so a natural-language command (“go to the chair”) resolves to a coordinate and triggers a Nav2 navigation goal. The result is zero-shot object-goal navigation — no environment-specific training and no fixed object vocabulary.

Frontier-based multi-robot exploration in which the team must stay connected while covering an unknown map — coordinating goal assignment against a connectivity constraint rather than letting robots greedily disperse.

Complete autonomous navigation system for a differential-drive mobile robot built on ROS2 and the Nav2 stack — mapping, localization, global and local planning, and recovery behaviors.

State estimation for the PR2 robot using Kalman and particle filters, comparing parametric and sample-based belief representations under noisy motion and measurement models.

Analyzed failure cases of MASt3R, a state-of-the-art multi-view 3D reconstruction method, under challenging capture conditions. Collected and annotated a custom scene dataset with varying lighting, occlusions, and viewpoint distributions, then documented systematic limitations and proposed improvements for robust reconstruction.

Developed multi-head Mask R-CNN models for instance segmentation and object detection in warehouse environments and integrated them into production robot perception pipelines. Implemented perceptual hashing for large-scale dataset deduplication (30% size reduction) and built order-fulfillment optimization using Leiden clustering.

Translating the CyScat electromagnetic scattering simulation package from MATLAB to Julia and Python under Prof. Raj Rao Nadakuditi. CyScat computes scattering matrices for periodic arrays of cylinders using the T-matrix multiple-scattering method with Modified Shanks Transformation for convergence acceleration, supporting dielectric and perfectly-conducting cylinders, Floquet mode analysis, and cascading via the Redheffer star product.

I implemented differentiable physical simulation via forward-mode automatic differentiation straight through the T-matrix solver, so gradients flow from the transmitted field back to physical design variables — enabling gradient-based optimization of refractive index and wavelength for maximum transmission through photonic structures. Derivatives were validated against finite-difference checks to machine precision.

The same machinery gives wavefront optimization via SVD of the scattering matrix: the first right singular vector of S₁₁ is the wavefront that maximizes reflection, and of S₂₁ the open eigenchannel that maximizes transmission. The video below shows an optimized wavefront tunneling through an S-shaped PEC maze at ~99.9% transmission, against only ~8.5% for normal incidence.

Built an end-to-end data generation and labeling pipeline to curate a benchmark of 1M+ paired real/synthetic audio samples using multiple TTS systems. Fine-tuned the CLAP audio-language model to 97% accuracy, outperforming human listeners. Because the paired construction eliminates linguistic cues, the model must rely purely on acoustic artifacts, making it robust to content variation.

Carnegie Mellon University — Dr. Arun Balajee Vasudevan