In heavy saucepan, over low heat, melt chocolate. Add sweetened condensed milk and vanilla, cook till smooth.
Making 1 sandwich at a time, spread 1 tablespoon chocolate mixture on each of two graham crackers. Sprinkle 1 with marshmallows and gently press second graham cracker chocolate -side down on top. Repeat with remaining ingredients.
Carefully break each sandwich in half before serving. Wrap with plastic wrap, store at room temperature.