This claim goes to the ontological heart of the Jewish imagination. What Judaism policed so vigilantly, and passed on to Christianity, is the Uncreated/Created divide. God creates ex nihilo, out of nothing.
This belief is commonly asserted, and seems rudimentary, but it stands before us as an apophatic mystery. We have no idea what it means that God "exists," as the only existence we are familiar with is on our side of the Uncreated/Created divide. The only existence we know of is created-existence, being on this side of the ex nihilo divide. Paganism, in the Jewish mind, is forgetting this ontological contrast and coming to worship created things instead of the Creator.
Perplexingly, this seemingly rudimentary claim is often misunderstood. By Christians, by pagans, and by atheists. Perhaps the idea is not so rudimentary. Some capacity for abstract thought is required. Because it's so common to hear people say "I don't believe God exists" or "I have doubts that God exists" only to inquire into what they mean by "existence" to find them muddled and confused on the issue. They are imagining created-existence (reasonably enough, since this is the only sort of existence we can grasp) and failing to recognize or police the Uncreated/Created divide. Far too many people assume God is a Cosmic Loch Ness Monster, a fanciful creature rumored in legend that doesn't actually exist in the big lake of the cosmos. Such a view of God's existence fails to attend to the ontological asymmetry we need to police when speaking of God.
The person who really got me to wrap my head around this concept was Pseudo-Dionysius and his treatise on the Divine Names. Lightly editing Pseudo-Dionysius here: God is the cause of everything, our origin, being, and life. God is transcendently beyond what is, the Source of every source. God is the Life of the living, the Being of beings, and the Goodness that commands all things to be and keeps them going.
