Start walking short distances with him/her on a leash. Carry small dog treats that you can give frequently. Hold the treat on your side down towards you knee. This will intice the dog to follow you without pulling on the leash. Give him treats frequently to let him know he is doing it correctly.