Killie had one eye on next weekend's CIS Cup final as they chose not to risk aggravating Alan Combe's calf strain and brought Graeme Smith back in goal.
The comeback keeper's goal was soon breached however, when Caley took the early lead. After patient play on the left, Ian Black swung over a lovely cross which Markus Paatelainen headed high into the net.
Killie responded well in front of a large home crowd, and levelled in the 18th minute with a messy goal. Peter Leven's long throw was missed by the crowd at the near post and Paul di Giacomo stabbed home.
The home fans were delighted nine minutes later when di Giacomo scored his second. He was alert to smartly smash home the rebound after Michael Fraser had done well to keep out Steven Naismith's well controlled volley, but could do nothing with the rebound.
Kilie looked to have the match won before half-time when Naismith brilliantly fired home an overhead kick in the 40th minute, but Caley fought back into the game straight after the break.
Smith did well to tip Gary McSwegan's cracking 30-yarder over the bar, but from the resultant corner, he was found wanting as Darren Dods poked home.
Killie's nerves were almost calmed by Gordon Greer in the 73rd minute, but Paatelainen was on hand on the goal-line to clear the defender's header.
Ross Tokely fired a 25-yard rocket just wide in the 82nd minute, and Graham Bayne saw his snapshot blocked by his own player, but Killie held on to send them to Hampden secure that they will finish the season in the lucrative top-six.