You have been mislead by the popular myth of fake masculinity, which says that real men are always ready for sex, always have a big, hard erection, and can always stay hard from the beginning of a sexual encounter until they ejaculate. Because your partner failed to conform to this phony ideal, you needed to explain it by thinking that he wasn't attracted to you, something was wrong with your relationship, he was "tricking" you, etc.
The truth is that men are human beings, not machines. Many things can interfere with the process of erection, much of it totally outside of conscious control, such as stress, sleep deprivation, distractions, worry about being able to conform to the phony ideal this time, etc. And yes, it is quite a task to keep the penis fully erect for long periods of time. It requires maintaining elevated blood pressure and heart rate, keeping the involuntary muscles regulating outflow contracted, and so forth. The state of our erections are glaringly obvious during sex, and if momentary weakening of the erection is seized upon as a sign of disinterest, weakness or failure, it makes it more likely that the full erection response will not return. Women can go through changes in their erectile response without anyone noticing, because it's all hidden away out of sight, and they don't have a great mass of erectile tissue on display to be maintained. Just because a man's erection goes down a bit, it doesn't mean he's "lost interest".
Some years back I noticed that it was a common pattern for me to go soft a couple of times before reaching a climax. My partner said it was great that I did this because it gave us more time to play.