The puzzle pieces are pretty much just additional collector's items
and don't have much function beyond that. The pieces can be found
while playing the game by doing many different activities all
around the site. If you wish to spoil yourself with where you can
get the pieces for each puzzle, you can find that info in this wiki article.
There's no rewards for completing any individual puzzle aside from
your own sense of accomplishment. However, there is one reward
related to the puzzle pieces at large I'm including in a spoiler
below: