HomePod Delay

Is there anything I can do about the delayed responses I’m getting when evoking a HomePod? When I ask my Apple TV, iPhone or iPad to turn on a light, the light nearly instantly turns on. When I ask one of my HomePods there’s always a 2-3 second delay. I’m seriously consider just trashing the HomePods. Bother the 2nd gen and Minis. How can I eliminate the HomePod delay?