The move comes amid a normalisation of ties between the US and Cuba.
The Caribbean country's presence on the list alongside Syria, Iran and Sudan was a sticking point for Cuba during talks to reopen embassies.
A US trade embargo against Cuba remains and can only be ended by Congress.
Mr Obama met Cuban President Raul Castro at the Summit of the Americas in Panama last week, four months after he announced a historic thaw in ties with the communist island nation.
The US president said on Tuesday that the government of Cuba had "not provided any support for international terrorism" over the past six months.
He added that it had "provided assurances that it will not support acts of international terrorism in the future".
A statement from Josefina Vidal, Cuba's top diplomat responsible for dealing with the US, said: "The Cuban government recognised the fair decision made by the president of the United States to eliminate Cuba from a list that it never should have been included on, especially considering our country has been the victim of hundreds of acts of terrorism that have cost 3,478 lives and maimed 2,099 citizens."
In Havana, ordinary Cubans welcomed the move. Erlinda Geroncelle told Associated Press: "We are not terrorists, just the opposite. We are supporters of peace and tranquillity, and good things for all."
Sara Pino told Reuters: "It's time that Obama realised that Cuba is not what they say the world over. It's time to re-establish ties between Cuba and the United States."
Cuba was first placed on the state department list in 1982, for what the US called efforts "to promote armed revolution by organisations that used terrorism".
The US believes Cuba had long provided a safe haven for members of the Basque separatist group ETA and Colombia's Farc guerrilla group, according to its 2013 Report on Terrorism.
White House press secretary Josh Earnest said the US still had differences with Cuban policies and actions, but they were not "relevant" to the terror list.
Republican Senator Marco Rubio, a candidate for the US presidency, condemned the White House decision, saying Cuba remained a state sponsor of terrorism.
Key moments in US-Cuban relations since Cuba's 1959 revolution:
1960: US breaks off diplomatic relations and imposes trade embargo
1961: US backs failed Bay of Pigs invasion; Fidel Castro proclaims Cuba a communist state and begins to ally with USSR
1962: Fearing US invasion, Castro allows USSR to deploy nuclear missiles in Cuba. Crisis takes the two superpowers to brink of nuclear war
1980: Around 125,000 Cubans, many of them released convicts, flee to the US when Castro temporarily lifts restrictions
1993: US tightens embargo, which introduces some market reforms amid economic woe following collapse of the Soviet Union
1999: Cuban child Elian Gonzalez picked up in refugee boat off Florida coast and placed with relatives in Florida, against wishes of father in Cuba. After numerous court decisions, US federal agents seize him to return to father
2002: US Under-Secretary of State John Bolton adds Cuba to "axis of evil"
2014: President Obama announces moves to normalise diplomatic and economic ties in a "new chapter" of US-Cuba relations