Taxis congregate around the airport and port in Gustavia.
The best option is a rental or scooter, located at the airport, or delivered straight to your hotel. Be warned that there are only two gas stations on the island, and both are closed on Sunday.
If St. Barts had a public bus system, it would require tinted windows and payment by credit card. This just isn’t the kind of island where residents ‘do public transport.’