I put a little natural dish soap in a spry bottle with h2o and spray. You need to reapply quite a bit but it helps keep the pests away.

Dish soap, neem oil and Dr. Bronners mixed with water are all excellent suggestions. It sounds like you might have whitefly, which when I worked at a nursery, was the most common insect to invade the herb table. The only thing I would add would be to make sure to get the undersides of the leaves sprayed well as whitefly lay their eggs there and not on the top surface. Maybe even look under the leaves and remove any with heavy infestation. You should be able to tell if it is whitefly if when you move or touch the plant, it almost looks like a little snowstorm bursts out of it.

Another possibility is that you have fungus gnats, which don't really harm the plant much but are very annoying as they tend to fly towards your mouth when you go near them. Those live in the top couple of inches of soil and are usually from the plant being a bit too wet. If the flies are dark instead if white, maybe try watering the plant from a saucer under the pot instead of from above to allow the top two inches of soil to dry out. The gnats need a lot of moisture to survive so this often takes care of them without having to actually spray anything.

Good luck!

if you can find out what bugs they specifically are, you can then find out what natural predators they have (if any)....if they do have regional predators, find out what you can do to attract those predators, i.e. other food items such as plants you can keep around, liquid foods, etc.

and don't forget to find out that the predator won't do any damage to any other part of your garden (most likely they won't since they are predators)