Rangers edged a thriller in the dying moments as they looked set to fall further behind Celtic in the title race.
It took 29 minutes for the Ibrox men to go ahead. Mikel Arteta's corner from the right brushed the head of Peter Lovenkrands and fell kindly for Shota Arveladze and the Georgian calmly tucked the ball home in the far corner.
Most inside the stadium expected a Killie capitulation similar to last week against Celtic, but the home side showed a fighting spirit.
Their first foray upfield was rewarded with the equaliser. Stevie Murray's crossfield pass found Steve Fulton, who appeared to be drifting aimlessly until he produced an exquisite backheeled pass which fooled Paolo Vanoli and Stephen Hughes and allowed Peter Canero to charge to the dead ball line and deliver a neat cross for Gary McDonald to slide the ball home at the far post.
Killie almost replicated the goal two minutes later, but this time McDonald sliced his shot.
Rangers boss Alex McLeish was livid a minute later when the referee waved play on after Lovenkrands appeared to be brought down by Gordon Greer.
His anxiety increased four minutes after the restart when the home side went ahead.
Questions will be asked as to how little Gary McSwegan was able to reach Murray's high cross to nod the ball back to Canero, but the Killie midfielder wasn't bothered as he steered the ball past Stefan Klos from 25 yards.
It looked like one of those days for Rangers as Lovenkrands saw his shot rebound off Frederic Dindeleux, Michael Mols' reflex shot cannoned off the bar and Francois Dubourdeau produced a great save to deny Emerson all in the space of 60 seconds.
Rangers knew they could not afford any slip up and they continued their onslaught on the Kilmarnock goal and the equaliser came in the 62nd minute when Arveladze was set up by Mols and again the Georgian finished precisely.
The winner eventually arrived three minutes from time when the much maligned Nuno Capucho nodded home Arteta's free-kick.