Here's a quickstart guide for Album Cover Art Downloader. Go ahead and run albumart-qt. It looks like this:
Click File | Open and navigate to the directory containing the albums you want to find cover images for. The program will load all the directories under the path you chose and try to guess an artist and album for each entry. Now you should be looking at something like this:
If you are feeling lazy, just select Edit | Download missing cover images and sit back while the program downloads covers for all the directories lacking them. The automatic downloader only accepts exact matches when searching covers, so it may miss some directories.
To set a cover manually, pick a directory from the list and check that the artist and album fields at the top are sensible. Now click Download Covers. After a few seconds you should see a list of cover images on the right. If no images are found, try to simplify the artist and album fields (e.g. clear the album field completely). You can also drop an image from your file manager on the list to use that one. A successful search looks first like this:
And then like this:
Go ahead and choose a suitable cover from the list. Finally, click Set as Cover to confirm your choice.
Lather, rinse, repeat and you're done.
You can download the program from the main page.