The Welsh side took the lead in the seventh minute through a smartly driven drop goal from number 10 Sean Connor on the Munster 10 metre line.
Rob Henderson's kicking cameo, following his penalty and conversion at Stradey Park, continued with a 17th-minute penalty reply just inside the Ospreys' right post.
Munster debutant Christian Cullen did not set Musgrave alight but just coming back from injury he adjusted well and did show signs of flair. Cullen's first touch took all of 18 minutes to arrive, and it was in a defensive role under pressure from centre Dave Tiueti.
The record New Zealand try scorer advanced up field as Munster took advantage of back row Steve Tandy's 31st minute yellow card for preventing a certain try.
A first Munster touch down arrived in first half injury time, however, as Shaun Payne was put over under Henson's challenge after an initial incision by skipper Jim Williams.
Henderson missed with the touchline conversion but managed to tag on the extras to Reddan's try on 46 minutes, as the scrum half ghosted inside Henson and right wing Stefan Terblanche for the score.
That cancelled out Connor's only penalty of the night for the Ospreys two minutes earlier.
However having established a 15-6 lead, Munster took their foot off the gas.
A loose Andy Williams pass after 51 minutes, gathered in by Seveali'i, saw the Samoan wing outpace John Kelly on the left touchline and chip past Cullen to score.
Three minutes later, the 25-year-old intercepted a high Lawlor pass seventy metres from the Munster line to out-sprint Cullen.
Connor's conversion was easily slotted over and Munster were behind for only the second time.
Irish centre Henderson had the best chance to level it but his 30-metre penalty fell agonisingly wide on 81 minutes, handing the visitors their first win on their travels since October.
Final Score: Munster 15 Neath Swansea Ospreys 18
Munster (15) Tries: Payne, Reddan. Cons: Henderson. Pens: Henderson.
Ospreys (18) Tries: Seveali'i 2 Cons: Connor. Pens: Connor. Drog goal: Connor.
Munster: Christian Cullen; Shaun Payne, John Kelly, Rob Henderson, Anthony Horgan; Jason Holland, Eoin Reddan; Frank Roche, Frank Sheahan, Gordon McIlwham, Trevor Hogan, David Pusey, Stephen Keogh, Eddie Halvey, Jim Williams (capt).
Replacements (from): Mossie Lawlor, Jason Jones-Hughes, Conrad O'Sullivan, Frank Murphy, Peter Malone, Tom Hayes, James Blaney, Martin Cahill.
Ospreys: Gavin Henson; Stefan Terblanche, Dave Tiueti, Matthew Brayley, Elvis Seveali'i; Shaun Connor, Andy Williams; Andrew Millward / Cai Griffiths, Barry Williams (capt), Adam Jones, Andy Newman, Luke Tait, James Bater, Steve Tandy, Andy Lloyd.
Replacements: Huw Bennett, Andrew Millward / Cai Griffiths, Richard Pugh, Lyndon Bateman, Gareth Morris, Richard Morgan, Adrian Durston.
Referee: Malcolm Changleng (Scotland)