The castle dates back to the early Bronze Age and contains the Church of Saints Serge and Baku, the Tang Yurji Guru apartment, the Church of All Saints and an old tower.

The castle has been used by many historical figures such as the Emperor Justinian, and has been destroyed and rebuilt several times.

The medieval walls are still standing and the whole site takes an interesting tour of Albanian history.