"What happened to Luciana Berger and to a lesser extent others was the most dangerous and vitriolic and pretty widespread onslaught that there's been," he said.
"What was outrageous and most perturbing was that if what had happened to Luciana had happened to someone else and was homophobic or sexist abuse, the response would have been hugely bigger. The fact that she did not get the support, instantly, from senior political colleagues is outrageous."
Asked whether Mr Miliband had let his shadow cabinet member down, Mr Mann said: "Not just him, but including him. It was a moral failure, a moral and ethical failure of profound implications.
"What underwrites it is a fear of tackling antisemitism. This was violent language, physical threats of violence, it involved police forces across the country. She was in the most danger.
"The lack of support was profoundly disturbing, and I think the same thing would have been a problem if she'd been in another political party. There would have been the same malaise in any other party.
"Where were people when it comes to a traditional, dangerous, violent antisemitic threat? A lot of people were not there, and it's outrageous."
As chairman of the All-Party Parliamentary Group Against Antisemitism for the past decade, Mr Mann has overseen the increased prominence of the issue across media, culture and society. But he believes there is still much to do. "At its heart is a failure to understand how damaging antisemitism is to individuals."
Making antisemitic remarks should be as much a "no-go" area - and "mundane" as challenging other forms of racism, Mr Mann explained.
"This has not been as routine as it ought to be, for politicians to be speaking out. That's part of the problem. It has become the norm to speak out against racism only in the past 30 years, but it's become less common to speak about antisemitism.
"The role for people like me in the future is purely to say what we think and not to hide away. More people need to do their tiny bit. That's all I've ever asked. Challenge on occasion, speak out on occasion."
Mr Mann's new book, Antisemitism: The Oldest Hatred, which is published next week, is a collection of high-profile speeches and writings on Jew-hatred. It includes the work of philosophers such as Jean-Paul Sartre, political leaders including Barack Obama and Nelson Mandela, and figures from the arts - Charlie Chaplin and Michael Douglas among them.
The Bassetlaw MP said he hoped the book would be treated as a "user's guide" to combating antisemitism and speaking about the topic in public.
"If you're a student in a debate this is the kind of resource you could use," he said. "I'm sure academics in this field would also find it useful but that's not the aim - not for it to gather dust, but to be thumbed through and used."
In February, Mr Mann's parliamentary group produced one of the most detailed recent accounts of how Jews are abused in Britain. After taking evidence from hundreds of witnesses it made recommendations to the government with the intention of improving the way authorities tackle antisemitism.
Since the group's previous report in 2005, much has changed, he said: "Among the establishment - in which I include me and all of Parliament - there's a deeper recognition. You wouldn't get away with things now that you would 10 years ago.
"There are arguments over how things are challenged, how often, who by, but nothing goes unnoticed or uncommented on. It used to, routinely. And that's a very big change. The fight against antisemitism is now in the political mainstream."
The response to February's report - including David Cameron's pledge of millions of pounds of additional funding for Jewish organisations and synagogues for extra security - was proof, the MP said, of how "relatively easy" it had been to get the government to act.
"We are knocking at an open door. The Prime Minister is listening. There are no political arguments in other parties about it, there's a consensus."
His key concerns for the future come with the challenge of social media, which is seen as the breeding ground of abuse. "The issue for us is that it's easier to intimidate, and for a young Jewish person the possibility of being abused is higher because there are more mediums for doing it," Mr Mann said. "It's a problem we haven't got to grips with."
Future plans for Mr Mann's group include a closer look at the situation at university in the United States, where the anti-Israel boycott movement is flourishing. There will also be a visit to Edinburgh to press the ruling Scottish National Party on the rise in Jew-hatred north of the border, and also further work to tackle problems in football.
He remains as bullish as ever. "Antisemitism is not the sexiest subject. Does it worry me? No. We have got a stronger group in Parliament now. No question about it."
