Your grandfather's birth certificate will show a date and a place of birth. Check the marriage records there for his parents, for up to say 10 years earlier. Church records can be very useful, so find out which churches were in the vicinity then. If you can find a baptism entry, then you may find other details too. Parents' names and addresses, maybe just the town or parish, perhaps even occupation of the father. Godparents were often siblings of the parents.
Many old newspapers are on microfilm at libraries, so try them too, for possible birth announcements.
Have you tried census records for the census years before and after your grandfather's birth? If after, you will have details of other children alive at the time of the census, plus ages of those children and of the parents. That will guide you back to a year of birth. And could you check the school records in the town where your grandad was born, if they are still available? Could be useful info there too.
You don't say where you are searching from. If USA, I have looked at http://www.ellisisland.org/
and there were 586 x James Moore but no James Alfred Moore listed. There was just one for Maud Hodder, from Chard, Somerset, England who arrived in US in 1922 aged 20. Assuming your great-grandad was a couple of years older than your great-grandmom narrowed down the James Moore entries to a possible 14. You need to register with the site to see the detailed entries, but it is free (or so it says).
Hope this helps.