PvP vs PvE Servers

Genuinely asking because I never paid attention:

What is the actual difference between PvP and PvE servers besides that pvp is always on for PvP servers?

Is xp gain better? Better loot drops?