Both sides notched in the opening eight minutes, opening the game right up.
Jarrow scored first, after six minutes, when Jonathan Hurst fired in from 15 yards.
Two minutes later, the Cumbrians’ Ryan Hall equalised from close range, beating keeper Gary Slater.
On the half hour, Peter Kane restored Jarrow’s lead, but Celtic came back again midway through the second half when Luke Close equalised. Jarrow’s winner came on 82 minutes, when Hurst scored again, firing past keeper Johnston from 10 yards.
Man in life-threatening condition after 'serious collision' involving car and pedestrian in South Shields
Appeal to find owner of lost Labrador after dog found near Jarrow Cemetery
Developers bid to add extra floor to former sports bar in bid to 'modernise' vacant building for housing scheme
Campaigners rally against Local Plan after farmland in Cleadon earmarked for 156 homes
South Tyneside weather: Here is the forecast for the borough for the week ahead
Boldon CA beat Coxhoe Athletic by a single goal, with Adam Bell side-footing home in the 35th minute.
South Shields Reserves had a narrow 2-1 home win over Darlington 1883 Reserves.
Shields opened the scoring when Reece Marsden was unmarked in the box and he turned to shoot past keeper Connor Pearce. Seven minutes later, Ryan Bolam curled home a 25-yard free-kick.
The visitors pulled one back on 26 minutes when a Kurt Matthews shot took a deflection and left keeper Curtis Richardson stranded, but they could not force an equaliser.
In the Northern League’s Ernest Armstrong Memorial Cup, Hebburn Town beat visitors Easington Colliery 2-0.
After only 15 minutes, William Jarvis-Smith opened the scoring with a close-range shot. Three minutes later, Chris Thompson doubled the lead with a flicked header.