Ramsee is first mentioned as a witness in 1223 under Sifrious de Ramesoue, Ministeriale of the Counts. It was not until 1280 that the village was recorded in the ducal land register. In 1445 Ramsee then appears in the herd place directory . Around 1800 Ramsee had 6 houses and 42 inhabitants. In 1849 three buildings burn down, and later another Sölde . In 1852 the entire village is bought up by the monastery owner Felix Christian Wieninger. Seven years later he sells the village for a profit to the state, which has it demolished except for the church and plants a forest. In 1864 the last building, the church, is demolished. In 1937 the Ramseer Herrsching area was allocated.
Today, apart from a memorial stone at the location of the former church, a cistern and a few hollows (presumably remains of earlier cellars) nothing indicates its existence. There is speculation about the causes, experts believe that the place fell victim to a fire .
Finally, there are stories about strange experiences that are said to have taken place on the bridge between Ramsee and Andechs.