Irene Peters, who works for the Los Angeles child welfare department, says Gavin Arvizo had even denied he had ever slept in the same bed as the star.
Mr Jackson denies abusing the child and holding his family against their will.
The first members of the singer’s family have begun to give evidence at his trial in Santa Maria in California.
His 16-year-old cousin, Simone Jackson, described her visits to Neverland Ranch and her contact with the Arvizo family, who are accusing Mr Jackson of abuse.
‘Like a father’
Ms Peters said she met Gavin’s mother Janet and her children on 20 February 2003, a few weeks after the US airing of a documentary in which Mr Jackson had spoken of his relationship with the boy.
He said, ‘Everybody thinks Michael Jackson sexually abused me. He never touched me’
Los Angeles social worker
In it, Mr Jackson and the then 13-year-old were seen holding hands, and the singer said he would sometimes sleep in the same bed as his child visitors.
Mr Jackson now stands accused of molesting the boy between 20 February and 12 March of that year, as well as of plying him with alcohol and conspiring to hold his family captive to keep them away from reporters.
“I asked him (Gavin) if he had ever been sexually abused by Michael Jackson and he became upset,” Ms Peters said.
“He said, ‘Everybody thinks Michael Jackson sexually abused me. He never touched me.”
Ms Peters said the boy’s mother had told her and other social workers that Mr Jackson was like “a father to her children”.
However, Ms Arvizo had mentioned the fact that Mr Jackson wanted to send her to Brazil and that she did not want to go there, Ms Peters said.
On Monday, several witnesses told the jury that Ms Arvizo did not try to call for help while she and her family were allegedly being held captive at the star’s ranch.
Also on Tuesday, the star’s cousin told jurors that she had once watched as Gavin and his 11-year-old brother took two bottles of wine from the kitchen at Neverland.
Ms Jackson said she told them not to take the alcohol, but they just ignored her.
Earlier in the trial, the prosecution had alleged that Mr Jackson often gave wine to children visiting him at his home.
But the defence will now argue that the evidence of this witness shows that the Arvizo children often took alcohol without permission, and were wilful and badly behaved.