"Hitler," he added "perhaps unsurprisingly wanted nothing written down" and the Nazis had also made a point of destroying documents.
Sir Antony said he believed that the Holocaust should be looked at within the wider context of the Second World War.
"I was greatly encouraged when, in 2012, the Holocaust remembrance centre of Yad Vashem in Jerusalem organised a major four-day conference," he recalled.
"It posed the question: '1942: A Turning Point in World War II and in the Final Solution?'
"Now, as we approach the 75th anniversary of the liberation of Auschwitz, I would like to raise the subject again."
Discussing exactly when Adolf Hitler agreed his plan for the Final Solution, Sir Anthony said the "mass murder of Jews" had been made an "integral part" right from the start of the Nazi invasion of the Eastern Front in mid 1941.
But he added that Hitler had a "desire to keep details of mass killings abstract" which was a "psychological paradox."
"Some historians argue that the key decision to use industrial genocide took place in July or August 1941, when a quick victory still seemed to be within the Wehrmacht’s grasp," said Sir Anthony.
"Others think that it did not take place until the autumn, when the German advance in the Soviet Union slowed perceptibly."
Sir Antony said his "estimate" was that the decision took place in the second week of December that year.
The Second World War expert had been introduced to the audience at Portcullis House, Westminster, who included Lord Eric Pickles, former Board of Deputies President Jonathan Arkush and Labour MP Rosie Duffield, by host for the evening Matt Chorley, editor of The Times' Red Box.
Mr Chorley said he had been inspired by the work of the Holocaust Educational Trust as someone who was "not Jewish" and "not an expert" but who had spent the last few years "appalled" at the rise of anti-Jewish racism.
He also criticised the failure of Labour under Jeremy Corbyn to accept of "deal with" its own problem with antisemitism.
Earlier Karen Pollock, chief executive of the Holocaust Educational Trust, had told guests ahead of Holocaust Memorial Day on Monday that "it has never been more important to remember 75 years on as antisemitism remains a blight on our society."
Ms Pollock praised Mr Chorley for being "source of wisdom and intergrity" who articulated better than anyone the shock over antisemitic discourse in society and politics with his phrase "This is not normal."
In a speech that was warmly applauded by all in attendance, HET ambassador Mumtahina Sultan, a sixth form student at Plumstead Manor School, described how a visit to Auschwitz left her determined to fight the rise of racism and antisemitism in society.
