The rapper had been doing time for gun possession.
He had been given two years for that offence and had completed most of the sentence before being released.
US authorities escorted the 36-year-old out of jail, although it is not yet clear whether he will have to serve more time in the tax case.
Ja Rule still has to serve some of a 28-month sentence he was carrying out at the same time for not paying his taxes.
He had been in protective custody because of his celebrity status.
According to his lawyer, Stacey Richman, the musician wants to get back to work as soon as possible to start making the £1.1m he has been ordered to pay back in taxes.
"Many people are looking forward to experiencing his talent again," added Richman.
She described him as a "devoted father" who was looking forward to his daughter's graduation.
'Choked up'
Ja Rule, whose real name is Jeffrey Atkins, had a Grammy nomination for his 2002 album 'Pain is Love' and has also starred in films including 'The Fast and the Furious' and 'Scary Movie 3'.
He went to prison in June 2011 and was given the earliest release date possible, said state correction spokeswoman Linda Foglia.
In the gun case, New York City police said they had found a loaded .40-calibre semiautomatic gun in a rear door of Ja Rule's $250,000 (£163,000) luxury car after it was stopped for speeding.
He pleaded guilty to that offence in 2010.
In March 2011 Ja Rule admitted that he had failed to pay taxes on more than $3m (£1.9m) he earned between 2004 and 2006.
"I in no way attempted to deceive the government or do anything illegal," he told the judge at the time.
"I was a young man who made a lot of money. I didn't know how to deal with these finances and I didn't have people to guide me, so I made mistakes."
GUYS, ARE YOU HIS FAN?
