It's not that Harry Potter is a religious text, it's that both Harry Potter and, for example, The Bible, are both fictional fantasy stories dealing with magic.
The argument is that if they believe Harry Potter should be banned, so should the Bible, Quran, Torah and other religious texts for also...