The police have set up cordons and are on-site to carry out an inspection. The fire service has evacuated parts of the premises.
The suspicious letter arrived at the social services on Thursday.
There are four people who have symptoms, and one of them has gone for a check-up, says police spokesperson Leif Fransson.
The people exhibited allergy-like symptoms, according to Växjö Municipality.
There is no information on the condition of the person who was taken to hospital. The others were initially isolated but were then allowed to leave the premises after being checked and sanitised by medical staff.
In the afternoon, the police announced that the national bomb squad was on its way to investigate the letter.
