The researchers say a person loses two months for every kilogram overweight they are - and seven years for smoking a packet of cigarettes a day.
Unusually, the Edinburgh university team found their answers by analysing differences in people's genetic code or DNA.
Ultimately they think it will reveal new ways of helping us to live longer.
The group used the genetic code of more than 600,000 people who are taking part in a natural, yet massive, experiment.
If someone smokes, drinks, dropped out of school and is overweight, it can be difficult to identify the impact of one specific unhealthy behaviour.
Instead, the researchers turned to the natural experiment.
Some people carry mutations in their DNA that increase appetite or make them more likely to put on weight, so researchers were able to compare those programmed to eat more with those who were not - irrespective of their wider lifestyles.
Dr Peter Joshi, from the university's Usher Institute, said: "It doesn't mess up the analysis. You can look directly at the effect of weight, in isolation, on lifespan."
Similar sets of mutations have been linked to how long people spend in education and the enjoyment they get from smoking or drinking.
The research team also found specific mutations in human DNA that alter lifespan, reported in the journal Nature Communications.
Mutations in a gene (a set of instructions in DNA) that is involved in running the immune system could add seven months of life on average
People with a mutation that increased levels of bad cholesterol knocked eight months off life expectancy
A rare mutation in a gene - APOE - linked to dementia reduced lifespans by 11 months
And one that made smoking more appealing cut lives by five months
Dr Joshi says these genetic variants are the "tip of the iceberg". He says around 20% of the variation in lifespans may be inherited, but only 1% of such mutations have yet been found.
However, he said that while genetics does influence lifespan, "you've got even more influence" through the choices you make.
Dr Joshi told the BBC: "We hope to discover novel genes affecting lifespan to give us new information about ageing and construct therapeutic interventions for ageing."
There are also some disease mutations that clearly affect life expectancy, and to devastating effect, such as the Huntington's gene. People with Huntington's often die in their 20s.
However, in order to follow people until the end of their lives, many of the people studied were born before 1940.
Prof David Melzer, from the University of Exeter Medical School, said: "An extra year of education then may have been much more important than it is now."