These caves derived its name because of its position close to the Modhi-Math popularly known as Baba Pyara. These caves are excavated in three distinct levels following the general terraced nature of the rock. The first and topmost, northern-most group has four caves. All of these have uniformly an inner hall and outer verandah with a pillared row. However, the largest cave at the west end has three separate inner cells with a common front corridor.
The second group of caves lies to the south of the eastern end of the first group at a much lower level. It is an apsidal chaitya hall. The verandah of this hall has two cells with separate entrances flanking the chaitya hall. The six square pillars of the verandah have lion brackets and chaitya window ornament. The end walls of the verandah carry a winged lion carved in low relief. The northern and eastern flanks of the courtyard have cave groups. The northern group has double cells with common front corridor and pillar screen. The main door lintels of the cells carry the auspicious symbols of the Buddhist, namely, Swastika, Triratna, Ankusa, fish, etc. There are two caves to the south of the courtyard. One of them has two back cells with common front verandah letting into an outer court through two screening pillars. The other group has a front court with seat bench round it and an inner cell. The door jambs are not only decorated with the auspicious symbols but also elaborate decorative designs flanked by curling rows of lions with heads turned back.
The third group has five sets of caves of which the second from the east has the largest squarish inner hall with a single octagonal pillar in the center as a support to the roof.
The Khapra Kodiya Caves are among the earliest and most important three cave groups at Junagadh - the other two being the Baba Pyare and the ‘Buddhistic’ Caves at Uperkot. All these Cave monuments were first reported by Sir James Burgess. Cut into a ridge of trap rock in east- west direction, all the chambers of this group of caves are rather plain. The central part is somewhat narrow, which provides an approach to the caves, facing a kind of broad ‘U’ - shaped quadrangle formed by rock excavation on the southern side. The two prominent wings of the caves comprise of (a) an oblong western wing provided with a grid pattern of water tanks within, having rock-cut steps, for harnessing and storage of rain water, and (b) a roughly wing shaped `L’ shaped wing fashioned to serve as a dwelling chambers for Buddhist monks. There are many scribbling and short cursive letters on the walls of some of the chambers and their corridors. All available evidence taken together, a date of at least circa 3rd - 4th century A.D., even earlier, may be suggested for this group of caves.