You should keep all records from time of birth. If purchased through a breeder, obtain all registrations, including microchip registration. All medical files will need to be transfered with permission.
If the dog is purebred, you will want to keep the AKC papers in case you want to make more puppies in the future. For any dog, keep records of all vaccinations, which you'll need if the dog ever goes to boarding.
For a new born puppy you would need the Puppy shot record, Vaccination record, Puppy health record. The printable version of these forms can be downloaded from the net.
Adoption papers/bill of sale, breeding contract if you have one, microchip information, tag, license and registration information, all medical records, shots & rabies records, insurance if you have it.
