'King Carlos' has a year left on his contract at Franklin's Gardens, but he has sat out the last seven games on the bench, and the rise of Stephen Myler at fly-half means he is likely to stay there.
The New Zealander is most probably Saints' biggest earner, and at the age of 33, it might suit both parties that the former All Black finds himself regular rugby elswhere.
Spencer and his young family are happily settled in Northampton, but the attraction of earning big money in France and the sunny lifestyle could tempt a move.
Toulon yesterday moved to quash the rumours, with Toulon media manager Benjamin Larrue saying: "We have no links with Mr Spencer. We haven't made contact with him."
But millionaire chairman Mourad Boudjellal is desperate to revamp his club that have slipped to second from bottom in the Top 14 after losing 11 of their 15 matches this season.
And the fact that Boudjellal has announced a press conference for Monday has led to some French newspapers speculating that it is a done deal.
Some cynics might suggest that Toulon are trying to destabalise Saints ahead of their European Challenge Cup clash at Franklin's Gardens on Saturday.
Spencer played for the Wanderers on Monday night in a 71-0 thrashing of Henley's second string and surely won't like the prospect of his illustrious career fading away on the bench at Franklin's Gardens, and moving Spencer on would also free up a sizeable chunk of Saints' £4m budget for next season's squad.
A spokeswoman for Spencer's agent would only say: "There is no news at the moment."
Meanwhile, Saints' Sean Lamont, Euan Murray and Scott Gray have all been included in Scotland's squad for the Six Nations.
Lamont hasn't played for the Scots since damaging his hamstring in the opening Test of their autumn international series against the All Blacks at the beginning of November.
The winger didn't return to action for the Saints until December 14, but has been injury free since and is instantly recalled by coach Frank Hadden.
Scotland squad for the Six Nations championship: Backs: Mike Blair (capt), Ben Cairns (both Edinburgh), Chris Cusiter (Perpignan), Simon Danielli (Ulster), Nick De Luca (Edinburgh), Thom Evans, Max Evans (both Glasgow Warriors), Phil Godman (Edinburgh), Ruaridh Jackson (Glasgow Warriors), Sean Lamont (Northampton Saints), Rory Lawson (Gloucester), Graeme Morrison, Dan Parks (both Glasgow Warriors), Chris Paterson, Hugo Southwell and Simon Webster (all Edinburgh). Forwards: John Barclay, Kelly Brown (both Glasgow Warriors), Geoff Cross (Edinburgh), Alasdair Dickinson (Gloucester), Ross Ford (Edinburgh), Scott Gray (Northampton Saints), Dougie Hall (Glasgow Warriors), Craig Hamilton, Jim Hamilton (both Edinburgh), Nathan Hines (Perpignan), Allister Hogg, Allan Jacobsen (both Edinburgh), Alastair Kellock (Glasgow Warriors), Euan Murray (Northampton Saints), Alasdair Strokosch (Gloucester), Simon Taylor (Stade Francais), Jason White (Sale Sharks).