Boyhood fan Dan Burn set Newcastle on the road to their first League Cup semi-final for 47 years with his first goal for the club as stubborn Leicester finally wilted.
Leicester City will be without James Maddison for Tuesday's Carabao Cup quarter-final at Newcastle, but manager Brendan Rodgers hopes the FA Cup win over Gillingham will give his charges a boost.
Manchester City will face Southampton in the Carabao Cup quarter-finals next.
Manchester City edged out rivals Liverpool 3-2 in a five-goal thriller to claim a place in the Carabao Cup quarter-finals.
Marcus Rashford scored a superb solo effort as Manchester United moved into the Carabao Cup quarter-finals with a 2-0 victory over Burnley at Old Trafford.
Newcastle forward Allan Saint-Maximin is convinced it doesn't matter who their opponents are in the Carabao Cup quarter-finals.