One in five adults think they are allergic to food with wheat being the most common.
However when they are actually tested just two per cent have a genuine allergy or intolerance, a report from the University of Portsmouth said.
Women are more likely to report a food allergy than men and it is thought the increase in the use of home testing kits and fad diets and celebrity 'fashion' may be fuelling the misconceptions.
It means huge numbers of people are avoiding foods that can safely eat and may be missing out on vital nutrients, the researchers said. Conversely symptoms that people think are linked to food may be due to another underlying condition that is going undiagnosed and therefore untreated, they warned.
Half of the population believe wheat allergy or intolerance is common but in reality it is as rarer than peanut allergy which has recently been found to hit only one in 50.
Lead author Dr Carina Venter, from the University of Portsmouth, said: "There is a clear discrepancy between the number of people who report that they have food allergy or intolerance and the numbers whose food allergy or intolerance can be confirmed by a medical diagnosis.
"Self-diagnosis and other diagnostic tests not conducted by qualified medical professionals are not reliable.
"Food allergy is usually investigated via a skin prick test by a medical professional with access to the patient's clinical history, sometimes in conjunction with a period of eliminating the suspect food.
"Parents who believe their child has a food allergy may feel anxious about their health and go to great lengths to ensure their child avoids certain foods.
"Children are more prone to nutritional problems when foods are excluded from the diet so it's even more critical that they receive a correct diagnosis."
Surveys have found that around 20 per cent of people said they had a food intolerance ranging from wheat and eggs to milk and peanuts.
The researchers said that 4.5 per cent of people thought they were allergic to wheat where as studies in children had shown that in reality less than one in 200 actually had a problem.
For those with genuine food allergies or intolerance it is vital they receive advice about replacing the nutrients they are missing out on which is why it is important people do not self diagnose, the researcher said.
Dr Venter added: "If you exclude bread from your healthy, balanced diet, you'll won't be able to enjoy a great tasting food which is naturally low sugar, low in salt, is calcium enriched and is a good source of protein and fibre."
The Wheat Hypersensitivity Report was commissioned by the Flour Advisory Bureau.
The figures did not include coeliac disease where the immune system does not function correctly causing weight loss, diarrhoea, and stomach cramps.
Symptoms of wheat allergy usually start within two hours of consumption including hives, itching, gastrointestinal symptoms and wheezing.
Wheat intolerance is similar but the symptoms are usually less severe and occur over a longer time frame and after eating more.