A descriptive board outside this temple says This is the temple where Kalidas worshipped (Kalidas, who lived in A.D 400, is often considered to be the Shakespeare of India; those who have read his works in the original, often claim that this comparison is an insult to Kalildas!) The plinth is of the first century EC, the Sunga period, while bricks and other pieces recovered from the basement belong to the fourth century, Gupta period; and the tenth and twelfth centuries, Parmar period. Emperor Harshvardhan had the temple renovated in the seventh century. It was rebuilt in modern times by the Gwalior state.
The temple sits on a plinth in the centre of a small courtyard with rooms all around. A golden lion gazes into the sanctum of this single-celled temple. The idol of Kalika is a red head wearing a silver crown. Legend has it that the genius of Kalidas blossomed only after he worshiped in this temple. An unusual feature of this temple is that its tower has red-and-white candy stripes painted horizontally on it. It also has a masonry' Light Tower or deepstambh, presumably built when the Scindias renovated the temple.