There are several legends about the magnificence of Madhur Temple. The Brahmanda Purana tells us that Bhargava Rama established the deity of Lord Vighneswara and offered prayers. The Skanda Purana tells us that Dharmagupta, the king of Trigarta Dravida kingdom drew the sketch of Mahaganapathi on a wall on the banks of river Kashmira as per the suggestion of Vasuki, to ward off the obstacles he came across while performing the Maharudrayaga. He then built a temple around the sketch and established the Linga of  Srimadanantheshwara.

The temple was under the management of the Mayippadi rulers of Kumbla in the 10th century and was renovated to its present form in the 15th century. The temple attracts not only devotees, but also connoisseurs of architecture who find the imposing structure with its peculiar combination of Hindu and Jain architectural styles fascinating.

The temple has a history associated with many rulers, especially the Kumbla ruler, Jai Simha I. There is also a story associated with Tipu Sultan. While on his conquering spree, he reached this temple intending to destroy it. Feeling thirsty, he drank water from the well of the temple. After relaxing a while, he changed his mind and left the temple unharmed. But he made a cut in the roof of the temple with his sword symbolizing the attack. That mark of the sword is preserved even now.