The UK voted against the resolution establishing the conference "in light of" the events of 2009 but in June a Foreign Office spokesman said a decision about attendance had not yet been made.
However today Mr Cameron said that after "careful consideration" he had decided Britain should not attend.
"No one should be in any doubt: this Government is 100 per cent committed to tackling racism both at home and abroad. I have made clear that racism has no place in any civilised society, is a menace to social progress and deeply offensive - so it must be confronted everywhere.
"But those aims cannot be met by accepting this invitation," he said. "Ten years ago, the World Conference on Racism saw open displays of unpleasant and deplorable antisemitism.
"It would be wrong to commemorate those displays. Indeed, they should be condemned. And that's why the UK will play no part in this conference."
It is ten years since the UN World Conference Against Racism, held in Durban, South Africa. At the time the event was criticised for descending into a forum to attack Israel. In one session the then Palestinian leader Yasir Arafat denounced Israel, while there was an attempt to reintroduce the "Zionism equals racism" UN resolution of 1975.
Two years ago a review conference was held in Geneva, attracting similar outcry after Iranian President Mahmoud Ahmadinejad was invited to make the opening-day address and accused Israel of being racist. The UK delegation walked out of his speech, although senior UN figures, including the UN Human Rights Commissioner, refused.
Last month former Foreign Office Minister Denis MacShane, who attended the original conference, wrote to Foreign Secretary William Hague urging the British government to pull out of this month's conference.
Vivian Wineman, president of the Board of Deputies, said he was delighted with the decision. "It is farcical that a conference which may again be again be addressed by President Ahmadinajad of Iran can have any pretentions to be taken seriously on the matter of human rights or any other topic," he said.
"I am proud that as a community we have consistently urged Britain's withdrawal and wish to thank the British Government for making this decision."
His comments were echoed by Ros Preston, chair of the Jewish Human Rights Coalition, who expressed hope the UK government would "continue to use their good influence in any future process".
The Israeli embassy said: "This conference long ago evolved into the inverse of its original purpose. We welcome the British government's decision today to place itself on the right side of this conference, on the outside of the door."
On Thursday the change in the universal jurisdiction legislation is due to get Royal Assent and become law.
