BRING water and rice to boil in large saucepan; cover. Simmer on low heat 15 min.
STIR in milk; cook; uncovered, 25 min. or until rice is tender, stirring frequently. (Do not overcook. Mixture should resemble thick cooked oatmeal.) Add caramels; cook 5 min. or until melted, stirring frequently. Stir in vanilla. Pour into 2-qt. casserole; top with marshmallows.
HEAT broiler. Broil pudding, 6 inches from heat, 3 to 5 min. or until marshmallows are lightly browned. Serve immediately.