Matt Sparrow's sending off and an injury to Alan Navarro could have been used as mitigating factors for the Seagull's Sixfields defeat but Poyet did not hide behind that in his post-match analysis.
"It wasn't a great day really," he told BBC Radio Sussex.
"Forget about the referee. We lost because we were not good enough. I don't look for excuses."
The Cobblers will now hope to draw a top-flight team in the second round of the competition which takes place in the week commencing August 23.