Ellis Iandolo and Jack Payne goals either side of Dom Telford's strike secured three points for Swindon at Rodney Parade.
Iandolo headed in a corner to put Swindon 1-0 up on 72 minutes, Telford levelled for Newport two minutes later but Payne struck three minutes from time to make it five away wins in a row for the Robins.
The first half started in cagey fashion with both sides' opening chances being shots that were blocked - Alex Gilbert for Swindon on five minutes and Dom Telford for County two minutes later.
Telford provided the game's first shot on target but his shot from a tight angle in the six-yard box was straight at Swindon keeper Jojo Wollacott.
County's top scorer then headed a Cameron Norman cross over the bar before Oliver Cooper had the best chance of the first 20 minutes.
After Swindon failed to properly clear a corner, the ball eventually found its way to Cooper whose powerful half-volley from close range was superbly tipped behind by Wollacott.
From the resulting corner, County came close again when Matt Dolan played the set-piece short to Finn Azaz who cut inside one Swindon player before firing off the top of the near post and behind for a goal-kick.
The half calmed down after that though with the only other shots at goal being Courtney Baker-Richardson having a header grabbed by Wollacott while Gilbert nodded over from the edge of the box for Swindon.
Swindon had the opening opportunity of the second half with that man Gilbert again striking wide of the near post after being played in by Jack Payne.
County keeper Joe Day was needed to make his first big save of the game three minutes into the second half and he answered the call to perfection as he dived to his left to superbly deny Gilbert from the edge of the box.
Newport came close twice in as many minutes when, first, Dolan's crossed free-kick from the right evaded everybody and Wollacott was needed to push it out for a corner and, seconds later, Mickey Demetriou headed a left-sided cross across the box and Telford lept to nod it narrowly over the bar.
With 25 minutes to go, Cooper hit a volley into the floor which bounced awkwardly and Wollacott did well to claim it as County kept pushing for the opener.
It was Swindon who took the lead on 72 minutes though. After Payne had a shot deflected wide of the post, the resulting corner led to Ellis Iandolo heading in from Payne's delivery.
Only two minutes after that though, County were level through who else but Telford who tapped into an empty net to connect with Cameron Norman's low cross from the right of the box.
But, after Baker-Richardson almost won it for the home side with a header that went narrowly wide with seven minutes to go, Payne struck the decisive blow for Swindon, with his effort from the edge of the box too hot to handle for Joe Day and it was 2-1 in the 87th minute.
Unfortunately for Newport, that was the final chance of the game as County tasted league defeat for the first time under James Rowberry.
Newport County: Joe Day, Cameron Norman, James Clarke, Matt Dolan (C), Mickey Demetriou, Ryan Haynes, Ed Upson (Timmy Abraham '90), Finn Azaz, Oliver Cooper (Jake Cain '87), Dom Telford, Courtney Baker-Richardson (Kevin Ellison '87)
Subs not used: Nick Townsend, Alex Fisher, Aaron Lewis, Scot Bennett
Swindon Town: Jojo Wollacott, Kaine Kesler-Hayden, Rob Hunt, Akin Odimayo, Dion Conroy, Ellis Iandolo, Ben Gladwin, Jack Payne (Jordan Lyden '90), Louis Reed, Tyreece Simpson, Alex Gilbert (Jonny Williams '62)
Subs not used: Lewis Ward, Jayden Mitchell-Lawson, Ryan East, Mo Dabre, Harrison Minturn