So the tooltip says that Prismatic Shot has a maximum of three targets, and normally that's what happens regardless of how many mobs are within range of the first target.

However.... if you get them into the right size bottleneck so that they're all mooshed together and stuck, that limit doesn't apply properly--I fired off a Prismatic Shot on a group of about fifteen or so mooks (with a couple Marauders for good measure) and it damaged all of them.