In the Middle Ages, Romencs was located in a circular structure surrounded by moats. A single bridge to the west provided access. Later, the village developed outside the moats, which are still filled with water today. According to local tradition, the current sacristy is the former chapel of the castle, which was named after Saint Jean-de-Romencs. The castle and the Town Hall, which were, together with the church, the centre of the village, no longer exist. Only the church remains.