First, get as many additional details about your grandparents and great-grandparents as you can -- where they lived, when, etc. This information may come in handy later on. Then you need to figure out why it's hard to go back.
If your ancestors were in the U.S., you might be able to use census records, tax records, property records, old newspapers, death certificates, military records, and cemetery records to learn more about them and their ancestors. Some of these resources are only available online if you pay, but most of them can also be found in libraries or by writing to the appropriate government agency of cemetery. (You may still need to pay for copies of the records.)
Some local and state census records are being digitized and are available for free.
If your ancestors were not in the U.S., you can search immigration and naturalization records. Ellis Island records are searchable for free, and they often contain information about place of origin and names of family members. You may have to pay for a nice printed copy of a ship manifest image, but you can view it online for free. Death certificates and birth certificates (for children) may also contain information about the person's place of origin. Once you find out where your ancestors came from, you can look into resources in those countries.
Helpful free information...