The Cobblers, who have lost their last three matches, now face League Two's bottom side, the managerless Bulls.
"We're playing well but we're not taking our chances," Cardoza told BBC Radio Northampton. "I'm confident that we can be much higher up the table.
"But a bad result against Hereford would upset a number of people, including myself."
Cardoza added: "There would be extreme pressure on me. But the football club is going in the right direction, and Ian Sampson and his coaching team will take the club forward."
Northampton have had an impressive run in the League Cup, beating Brighton and Hove Albion, Reading and Liverpool on their way to the fourth round.
But their league form has suffered, and the Cobblers now lie 22nd after 11 matches, knowing that defeat, and victory for Barnet at Stockport, could deposit them in the bottom two.
"This time last season we were in this position and I was worried," said Cardoza. "But I'm not worried this time. We're so close to getting it right.
"The Carling Cup has taken its toll and the players are tired. Mentally, it's hard on them."
"We're sticking to a policy of bringing through young players, and we want to play football the right way."
"There is inconsistency, but it's the route we want to take. To get out of this league, and the next league, we need to find younger players and keep them for a long period of time."
Meanwhile, the Cobblers have become the latest League Two club to be linked with Crewe Alexandra striker Calvin Zola.
The 25-year-old Congolese forward has already rejected switches to Shrewsbury, Burton Albion, Southend and Bradford.
"He's one player that we're looking at," admitted Cardoza. "Crewe would want some money for him, and he would need to come on a permanent deal, not on loan.
"It might not be until January anyway, but we'd be happy to have him if we could pull it off."
Source: BBC Sport
Source: BBC Sport