Liverpool came from behind yet again this season, with Kasey McAteer scoring for the visitors just three minutes in.
However, the Reds rallied in the second half with a trio of well-worked goals from Cody Gakpo, Szoboszlai and Diogo Jota to secure passage into the fourth round draw and a seventh straight win in all competitions.
It was Szoboszlai's goal in particular which got fans out of their seats. The 22-year-old midfielder, who came on as a substitute, hit a rocket from well outside the box that left Leicester's keeper Jakub Stolarczyk totally helpless.
Speaking after the game, Klopp told reporters that Liverpool's new number eight needs no encouragement to shoot, calling the former Leipzig's man's strikes 'grenades'.
Klopp said, via Liverpoolfc.com: "I don't have to tell him to shoot, he knows that, he knows better than I that he has a pretty good skill. The next one, which goes slightly over, is another grenade.
"You can see that in Premier League games, everyone tries to block him, that's why he didn't come through yet – but if he comes through there's a good chance the ball could be in. A super goal. Wataru passed the ball through and in that moment, there is no midfielder there, there is no defender who can then hold him, that's why it is so important we are winning the ball to try to play the first pass forward or at least have a forward-orientated idea. That helped there, a lot. Good."
Already hailed by many as the signing of the season, Klopp discussed the impact Szoboszlai has had on the team since arriving this summer in a £60m transfer, while praised the new arrival's work ethic and adaptability.
"It's so long ago that he's here I forgot what I thought when he came in," Klopp continued. "Since the first minute of training, it was pretty impressive. He's a top bloke, really top guy. Easy step into the team dressing room-wise and stuff like this. A very naturally confident boy and that helps and a super-hard worker and that's what you see.
"Today he came on and had two very difficult situations; he stepped on the first ball when he defended a counter-attack and lost the ball there. Then the next one I think he lost as well. From that moment on, he was just in the game. That is super-important. He enjoys it a lot being here and the people obviously appreciate it as well. It's a good situation. Long may it continue."
Asked after the game whether the goal was the best he's ever scored, Szoboszlai himself was unsure.
The midfielder said: "It was very special. I don’t know if it was the best goal, I scored one of my best in the national team and also in Salzburg. But [it was] one of the best for sure.
"I’m just doing my things, working hard and always ready to fight for the team, for the fans, for myself. And as I get many chances, I want to give back."