Southampton rode their luck at times in an error strewn first half, which saw all three central defenders make mistakes that could have been more costly, before Cameron Archer curled a sublime left-footed shot into the far corner to give saints the lead.
Mads Roerslev, Saints recent signing from Brentford, was handed his first-team debut in a much changed Saints lineup from Saturday's defeat at home to Stoke City.
Alex McCarthy, deputising for Gavin Bazunu as he did in the previous round at Northampton, was forced into two full-length saves in the first half which he did well to keep out.
Joshua Quarshie and Ronnie Edwards came in for Jack Stephens and Taylor Harwood-Bellis respectively. Will Still, it seems, is still trying to work out his best centre-back partnerships.
Goal! Norwich City 0-1 Southampton
Saints got the reward that their hard work and possession perhaps deserved after 42 minutes when they pressed Norwich as they tried to play out of defence through a packed central area.
Ross Stewart latched on to a misplaced Norwich pass before laying the ball off to Cameron Archer on the edge of the box, to set his sights.
Archer, from a central position and just inside the penalty area, took a touch and then bent his shot past Grimshaw in the Norwich goal.
Half-Time - Norwich City 0-1 Southampton
Norwich started the second half brightly with Alex McCarthy forced into another big save from a Jeffrey Schlupp volley to keep Saints ahead.
Norwich had a good shout for a penalty turned down by referee Tom Nield. Jovon Makama went down under a challenge by Flynn Downes but there was not enough in the challenge for a penalty.
Goal! Norwich City 0-2 Southampton
Ryan Fraser kept his composure well after Mirko Topic made a hash of clearing a hopeful through ball from Ryan Manning.
The Saints veteran picked his spot from 12 yards out, hitting a low shot into the bottom right hand corner as Grimshaw rushed out.
Fraser's goal after 52 minutes was his last action of the match as Kuryu Matsuki was given another opportunity to get some minutes in a Saints shirt.
Shea Charles was denied by Grimshaw from close range after 67 minutes. Charles' one-two with Cameron Archer put the Southampton midfielder in on goal but his effort cannoned off the keeper to safety.
Still rotated his fron two for the remaining 20 minutes of the tie. Ben Brereton-Diaz and Damion Downs came on for Ross Stewart and Cameron Archer who had both looked lively.
Ante Crnac went close with a fierce 20 yard effort which had McCarthy scrambling to even get near it. Thankfully for the away side, the ball whistled wide of the left-hand post.
Goal! Norwich City 0-3 Southampton
81 mins - Saints substitute Kuryu Matsuki let fly from range after passing up the opportunity to feed Mads Roerslev down the right hand side.
Grimshaw could only wave at the ball as it flew past him into the top right hand corner, sparking a wild celebration from the Japanese attacking midfielder who spent last season on loan at Goztepe in Turkey.