An end-to-end robotics pipeline that translates natural-language instructions into precise pick-and-place actions using a VLM-based planner.

for demo (full video) please view project_submission.mp4 Thanks!

1. Use ground truth box coordinates

-- Why? : So I can tune the planner

1. no interpolations works poorly (overshoots)
2. therefore i am trying parabolic interpolation of intermediate waypoints after which it should be smooth

1. To avoid appending the second prompt with history of previous appended. => switching to feeding it latest perceieved state!