Soler was in the peloton pile-up that took place 46 kilometres out from the finish line after the fan had posed too close to the road and with their back to the action.
Tony Martin of Team Jumbo-Visma was the first to collide with the placard-wielding spectator, before chaos ensued and forced Soler to leave the stage with fractures in both his arms.
Speaking to La Vanguardia newspaper, the 27-year-old expressed the extent of his frustration, having had to abandon a Grand Tour stage for the second time this year.
“I don’t know what to do, I’m thinking of taking the spectator to court, because that’s an entire Tour canned and I feel very angry,” he said.
“The Giro was just really frustrating because it was a race I had prepared for over a long time and I had a great opportunity.
“But the Tour is even worse, because it wasn’t just a race incident, it was because of a spectator who obviously doesn’t like cycling.
“All that preparation has gone into the bin.
“The fall happened at a point in the race where the road narrowed and we were trying to be well-placed, we were near the front and then I saw all the Jumbo-Visma riders going down and (Mike) Teunissen crashed right in front of me.
“I went flying, somersaulted and landed hard on my hands.
“They both hurt, and so did my face where my glasses had broken and my shoulders too.
“I tried to get up, but I couldn’t, I didn’t have any strength in my arms.
“The mechanic pulled me up by my armpits, and I sat on the side of the road, I was really dizzy.
“There were still 50 kilometres to go. They (the team) told me to try to go on but I don’t know how I did, I couldn’t change gear or brake.
“When I got to the finish, I was worried about the time limit, but I couldn’t even get my clothes off in the buss, they had to cut them off with scissors.
“Then, when we got to the medical truck, they confirmed my injuries.”
With Soler now out of action for four weeks, he has already ruled out any hopes of competing at the Vuelta a Espana, where he won a stage last year.
As for the spectator, it’s believed French police have since apprehended the culprit, who will now face a fine of €1,500 (A$2,373) for involuntarily causing injury.
The Tour de France continues tonight with Stage 6 and you can catch all the action on SBS, the SKODA Tour Tracker app and SBS On Demand from 9:30pm (AEST).
