The ones in the ground are ground wasps there are also a hornet that looks like a yellow jacket that makes a paper nest like white faced hornets. There are also yellow jacket wasps that make the grey comb like nest as wellThey are not yellow jackets. Yellow jackets rarely nest above ground.
Hornets' nests are spherical and covered with a paper like substance.
You are dealing with hornets.
Yellow jackets normally have a hole in the ground leading to their nest.
Hornets have spherical nests attached to a structure.

