The problem I see with LSD, as stated by others, is the changes it causes to one's mind and brain chemistry.
It is quite apparent that LSD is physically harmless. However, this doesn't mean the psychological effects should be overlooked. People report of LSD "changing and opening up their mind." It totally changes people and turns them into different people. They become more as stereotypical "hippies," conpletely falling out of line with traditional society.
Those who do psychedelics are entirely different people once they do. I don't see the risk as worth it.