Why do people (or "some" people) natively from the Virgin Islands have an accent?

well it's not part of the u.s it's a territory, so probably before the us got there there was at least some people and they could say the english words correctly and it spread on, or there born like that like your born speaking english i asume.