With Tribalfootball in attendance, Balogun played a central role in winning Arsenal a first half penalty, before making it 2-0 with a stunning second half volley.
A James Olayinka finish into the bottom corner settled the match, during which Arsenal were largely the more dangerous side.
Leicester did have their moments and Karl Hein was forced into action on a couple of occasions, but they will be disappointed to have seen Mitch Clarke sent off late on.
The away team actually started the better of the two sides, pressing Arsenal effectively early on and preventing them from progressing the ball forward.
Seven minutes in Batosz Kapustka nearly rewarded the Foxes for their strong opening to the match, shooting just wide from edge of the penalty area after good work from Josh Eppiah.
Leicester went close again ten minutes later when Medley's pass was intercepted by Eppiah in a dangerous area.
It was Arsenal though who opened the scoring from the penalty spot on 24 minutes, after Leicester goalkeeper Tobias Johansson bundled over Nathan Tormey.
Balogun started the move with a twisting run and when Johansson could only parry his shot into the path of Tormey, the keeper brought down the Arsenal winger as he was about to strike.
Robbie Burton converted the subsequent kick, shooting high into the left corner.
The goal seemed to spark the Gunners into the life and they went close through a James Olayinka volley from the edge of the box minutes later.
Steve Bould's team were very much on top by this point and could have had a second before half-time, with Tormey putting in a couple of dangerous balls across the face of goal to no avail.
In response to Arsenal's dominance Leicester switched to a back three at half-time, replacing centre-forward Ali Reghba with wing-back Vantae Daley-Campbell.
The two sides exchanged chances early on, with Tormey's cross glanced just wide by a retreating Leicester defender, before Medley nearly turned Daley-Campbell's searching ball into his own net.
Arsenal remained the better side though and Balogun made it 2-0 on the hour with a powerful volley from outside the area.
Receiving the ball on the turn, the Arsenal striker's touch laid the ball into his path perfectly before he lashed a strike into the top scorer, leaving Johansson no chance.
The Foxes responded positively to going a second goal down and Eppiah forced Hein into a good save from close range on 70 minutes following a well worked move.
However, Arsenal made it three five minutes later when Olayinka was sent clear through on goal before finishing calmly into the bottom corner.
The winger was in acres of space as he advanced on Johansson, but showed no nerves as he rolled the ball into the back of the net to cap off a strong performance.
Late on the away side did draw a few decent saves from Hein, but their night was compounded when Mitch Clarke was sent off in stoppage time for a last man foul on Balogun as he advanced on goal.
Substitutes: Olowu, Hillson, Greenwood, Cottrell, Oyegoke
Leicester City: Johansson; Clark, O'Connor, Ughelumba, L. Thomas; Leshabela, Dewsbury-Hall (c); Eppiah, Wright, Kapustka; Reghba
Substitutes: Substitutes: Daley-Campbell, Storlarczyk, Leathers, Loughlan, McAteer, Shade, Pennant