Jay Rodriguez scored his first Premier League goal but was left disappointed by the first-half showing that meant his side were beaten by Spurs.
Saints trailed by two at the break after a lacklustre opening 45 minutes, but produced an excellent second-half showing that perhaps deserved at least a point.
Rodriguez's maiden league strike in Southampton colours halved the deficit, but the north Londoners held on for all three points.
"We're disappointed with the result," Jay told Saints Player at full time.
"In the second half I thought we were a lot better, but the first half killed us with the two goals we conceded so we haven't picked up any points.
"We had a belief in us that we were going to show what we can do and we got a goal back, but it obviously wasn't enough.
"It was nice to score a goal, but my aim was to get three points for the team and get us up the table. I'll keep working hard to try and score as many as I can.
"We'll take positives from the second half, but those two goals killed us. It's frustrating, but we all need to keep working hard.
"Everyone is disappointed with the result because the second half showed we deserved something."