depends, if there is no danger of frost, and you are having warm temps, and don't care about the size of the fruits, let em go. If there are still cold nights, or temps aren't normally in the 80's (F) this time of year, I'd pinch off all blooms. If you want the largest plant pinch off the blooms. If you want a few really large fruits from each plant pinch off all but two on each "arm" or branch.
That's how I handle it, others will likely have other ideas.