A spokesman for the French coastal authority Premar said that after the boat encountered difficulties in the sea near Calais in the early hours of Saturday, two passengers may still be missing.
About 58 people were rescued by British and French coastguards, officials said.
A number of people were seen being brought off a lifeboat, some on stretchers, in Dover.
The extent of injuries remain unclear and the exact numbers of those rescued have changed since the morning as more information was released.
The six people who died were Afghan men thought to be in their 30s, the AFP news agency reported Philippe Sabatier, deputy public prosecutor for the French coastal city of Boulogne, as saying.
He said those rescued included some children and were mostly from Afghanistan, although there were some Sudanese.
French authorities said a passing ship first raised the alarm that an overloaded boat was in difficulty off the coast of Sangatte.
When the French lifeboat arrived, they found numerous people in the sea, with some screaming for help.
The Dover lifeboat, which was already in the Channel dealing with another boat carrying migrants, joined the rescue operation.
A volunteer, who was on one of the rescue boats, told the Reuters news agency migrants were using shoes to bail water out of the sinking boat.
Anne Thorel said there had been “too many” people on board.
Rescue crews say this is the seventh time this week that they have had to pull people from the water.
In its latest update, Premar said interviews with survivors suggest 65 or 66 people were on the boat. Often boats are so overloaded it is difficult to tell how many people are on them.
Premar said 22 or 23 people were taken to Dover by UK rescue crews and a French boat took 36 to Calais.
Two French boats were still searching for the two people who could still be missing, it added.
A French Navy aircraft and a helicopter had been deployed to help the search.
The MP for Calais, Pierre-Henri Dumont, said authorities are interviewing the migrants who are able to speak and not too unwell, to establish what happened and where they are from.
Although the incident happened in French territory, with these types of operations, British and French rescue teams work together to rescue as many people as possible.
Enver Solomon, chief executive of the Refugee Council, thanked the rescuers for their efforts but urged the UK government to work on creating an “orderly and humane asylum system”.
The English Channel is one of the most dangerous and busiest shipping lanes in the world, with 600 tankers and 200 ferries passing through it every day.
Home Secretary Suella Braverman called what happened a “tragic loss of life”.
A UK government spokesperson said the deaths were “devastating and our thoughts are with the victims’ families and friends at this time”.
They added: “This incident is sadly another reminder of the extreme dangers of crossing the Channel in small boats and how vital it is that we break the people smugglers’ business model and stop the boats.”
Dover MP, Natalie Elphicke, said the incident highlighted the need for joint patrols on the French coast.
“These overcrowded and unseaworthy death traps should obviously be stopped by the French authorities from leaving the French coast in the first place,” she said.
On X, formerly known as Twitter, shadow immigration minister Stephen Kinnock said it was an “appalling, deeply shocking tragedy”.
Another small boat also got into difficulty but all on board have been rescued, the UK Coastguard said.
In the last two days more than 1,000 people made the journey across the Channel to the UK, government figures show. More than 100,000 migrants have crossed in small boats since 2018.
At least 27 migrants died after a dinghy sank while heading to the UK from France in November 2021, the highest recorded number of deaths from a single incident.
Four people died at sea while trying to cross in December 2022.
The incident comes after the UK government faces pressure over fears of a Legionella outbreak on its new migrant barge, Bibby Stockholm, moored in Portland Port, Dorset. The first migrants to board the vessel had to be removed after bacteria was found in the water system.