Newport were looking to carry momentum from their mid-week result into the clash but it was Swindon who came out on top after strikes from Kemp and Hepburn-Murphy.
Graham Coughlan awarded Payne, Wood and Maxted their first league starts after all three players impressed during the Exiles recent fixture with Cheltenham Town on Tuesday night.
Newport had the first chance of the game when Bennett's shot was deflected away by the Swindon defender.
While Newport were moving the ball nicely in the first five, it was Swindon who marked the scoresheet first when Dan Kemp rounded an advancing Jonny Maxted and slotted it home to give Swindon the lead after just six minutes on the clock.
It was Newport that looked most likely to score next as the Exiles moved the ball well and created various chances for both Wood and Bogle up top.
Newport had a perfect opportunity to equalise via a Bogle free-kick after he was brought down just outside the area, but the shot sailed over the bar and into the stand behind.
Before the half-time whistle sounded, captain Ryan Delaney had a chance to put his side level with a diving header, but Mahoney made a simple save to keep Swindon ahead at the break.
Half-time
Evans was substituted at half-time for Kiban Rai after he was brought down multiple times in the first 45.
Newport came out for the second-half with a point to prove, the high press was applied forcing mistakes from the Swindon back-line. Rai, Bogle and Wood were pushing up the field and pouncing on any loose passes the press was forcing.
Swindon created multiple attacks down the right-hand side, which each time was dealt with well by McLoughlin and Seberry.
Coughlan went to his bench again in the 57th minute bringing on Olly Thomas and Matty Bondswell for Matt Baker and Nathan Wood.
Bondswell made an instant impact playing in Rai who chested it down and took on the shot, that was deflected away from danger by the Swindon full-back.
Newport eventually broke free from the Swindon defenders when Bennett played the ball over the top to a pacey Olly Thomas but the Swindon keeper reacted well and sprinted up the field to clear the danger.
Newport were relentlessly trying to equalise, headers from Bogle and strikes from Rai were all palmed away by Mahoney who kept his side ahead in the tie.
Against the run of play Swindon launched an attack, the ball found Austin in the box who slotted it home. The goal however was dissalowed due to a foul in the build-up on Bennett.
Coughlan made his final change of the match when he brought on recently returned from injury Declan Drysdale in place of McLoughlin.
It was in the 81st minute that Swindon threatened again, when Maxted made a fantastic initial save from Khan's powerful shot but only into the path of Hepburn-Murphy who doubled the host's lead.
Despite various chances falling for the Exiles, none were converted as the referee blew his full-time whistle.
Full-time
Newport County XI: Maxted (GK), Payne, Delaney, Evans (Rai 45'), Morris, Bogle, Seberry, Bennett, McLoughlin (Drysdale 72'), Wood (Thomas 57'), Baker (Bondswell 57'). Subs: Townsend, Waite, C. Evans.
Swindon Town XI: Mahoney (GK), Hutton, Blake-Tracy, McEachran, Young, Khan, Kemp, Brewitt, Shade, Godwin-Malife, Austin. Subs: Ward, Clayton, Hepburn-Murphy, Uwakwe, Kinsella, Minturn, Obodo.
Attendance: 9628 (715 away)