A security source tells the news agency AFP on Sunday that "150 people were on board the ship that sank" while another source says that "searches are ongoing".
Abdusattor Esoev, head of the UN's migration organization (IOM) in Yemen, tells the news agency AP that 68 people have so far been found dead and that 74 are missing. A total of 154 Ethiopians were on board and twelve have survived.
The bodies of the dead have been taken to different morgues in the area.
The province's security directorate writes in a statement that the security forces "are currently conducting a large operation to recover the bodies of a significant number of Ethiopian migrants who drowned off the coast of Abyan when they tried to enter Yemeni territory illegally".
"Many bodies"
"Many bodies have been found on different beaches, indicating that a number of victims are still missing at sea", the statement continues.
Abyan stretches northeast from the port city of Aden.
It is unclear what caused the incident. But shipping in the area is plagued by several security challenges. Often, boats and ships lack basic safety equipment.
Furthermore, the networks that smuggle people in the conflict-ridden region generally take great risks in their operations.
Exposed to abuse
Moreover, both pirates and the Iran-backed Houthi rebels often attack ships moving in the waters off Yemen's coast.
Despite the war that has ravaged Yemen since 2014, irregular migration through the poor country has continued, especially from Ethiopia, which has also been ravaged by ethnic conflicts.
In countries such as Saudi Arabia and the United Arab Emirates, there are many foreign workers from Africa and southern Asia.
According to the UN, tens of thousands of migrants who have been exposed to abuse during their journeys have been left in Yemen.