Alongside older brother Steven, he moved to England from Scotland as a 16-year-old in 1997 but left in 2004 without making a first-team.
However, Gullit's methods allowed Caldwell to come up against Newcastle's big guns in training - and the current Partick Thistle boss remembers fondly, just how good the Magpies' club-record top goalscorer was.
"He (Shearer) was awesome," admitted Caldwell to Open Play.
"He was physical but he used to work extremely hard on his game - everyday he would practice his finishing.
"I remember on Friday he used to get Tommy Craig (reserve team manager) to stand about five yards away from him and literally smash the ball in at him and he was just touch, touch and stop the ball dead. He would set himself and just control it. If he wanted to buy a foul, he could.
"He was a top, top player."
After just over a year in-charge, Gullit was sacked after the Dutchman opted to leave Shearer and Duncan Ferguson on the bench in 2-1 defeat to Sunderland at St James's Park in a bid to earn power over the pair.
Caldwell recalls an "amazing education" under Gullit - but admitted Shearer was always going to win his feud with the 56-year-old.
He added: "Ruud Gullit played a lot of 11v11 games - so the reserves and younger players would come up against them (first-team).
"I remember marking (Alan) Shearer and (Duncan) Ferguson most days in training - which as an education was amazing.
"It was was great for us as an education but it probably wasn't great for Shearer - him and Gullit were not the best of friends.
"There was one day on the pitch when he wanted Alan to do something - and I could tell how he wasn't best pleased that he (Gullit) was showing him how to do things.
"In the next play, the exact same thing happened - Shearer didn't move and just looked over to him - it was a clash of two big personalities - there only going to be one winner and ultimately the manager was sacked."