As a general rule, if it's a plant, part of a plant or was a plant, it can be composted. In addition, here are some things that are often overlooked: egg shells, coffee, tea bags, herbivore feces, chicken/duck/goose feces, plant based bedding material for the above, and shredded paper. Unused (unpainted) drywall scraps and wood ash may be used in limited proportions. Too much will through off the pH balance of the compost.
Avoid weeds that have gone to seed and keep in mind some partially developed seed-heads can pull enough nutrient from the cut plant to finish development. If questionable, lay the weeds in the sun until dry and before adding. Also avoid vegetables cooked with grease or meat should be avoided for the home composter. Finally, dog and cat feces should not be used to keep from spreading pathogens and disease to your garden/food supply. (There are sites that discuss composting dog feces for use on non-edible plants, but that is outside the focus of this blog.)
[This next paragraph involves composting meat and is not for the squeamish. Highlight the text to view or scroll down to skip.]
As a note, animals and offal may be composted, but require a large pile more suitable to a farm than a backyard. The process takes longer, but the material will break down, bones and all. Again, stick with herbivores and poultry unless spreading on pasture. If you have enough animals to be composting death loss and offal, I recommend keeping this a separate pile that is spread only on pastures to further limit a limited concern. There should still be plenty of manure, bedding, and vegetative compost left for the garden.
The largest concern of composting meat is attracting scavengers which can be a nuisance. A side benefit is hauling away the beef offal that won't be eaten can reduce processor fees as the butcher doesn't have to pay for disposal.
Does it matter what I put in?
There should be a mix of material put in the pile in order to speed the breakdown process, but also keep it slow enough to avoid odors. There are books written about this and mixing the right ratio of Carbon and Nitrogen. If you want to get into the topic that deeply, please check my booklist. However, it boils down to mixing greens and brown, wets and dries, the ying and the yang. As a general rule, keeping these at a 50/50 mix is optimum. More browns will slow the process whereas greens will speed it up (to a point), but it will still work. So don't worry about being meticulous with the mixing, just use it as a target.
So what is a brown? Think dried leaves, plant stems, wood mulch, sawdust, and straw. This is the Carbon that adds the dry matter to keep the pile from becoming a stinking slimy cesspool. The stink and slime is due to decomposition so rapid that it causes fermentation. Essentially, browns are the brakes to slow the process.
Greens are easier to think of, but are not limited by just color. A green would be any moist vegetation (cauliflower, orange peel, tomato waste, potato peelings) and also includes manure. Greens are the accelerator and speed the breakdown to compost.
As an awareness, I do want to mention that different material will have more Carbon or Nitrogen than others. The ideal ratio of C to N is between 25:1 and 30:1. For example, wood chips are 400:1, dried leaves 60:1, and grass clippings 12:1. Again, I'm staying out of this for now and would reference the book list for more information.
What about turning the pile and these expensive tumblers?
Composting is an aerobic (requires oxygen) breakdown of organic material. Over time, the oxygen in the center of the pile is depleted causing the process to slow. Adding oxygen back in to the center allows the pile increases the rate of material breakdown.
There are three typical ways to add oxygen back to the pile. First, insert a drainage pipe into the center of the pile. The downside is it can allow heat to escape, so this works best with a higher amount of greens. Second, use a compost tumbler. The limitation is the batches are small which may not be a downside. Third, is to mix/stir the compost. This is the lowest cost but highest labor.
My recommendation is to use a pile system with periodic mixing for a beginner. One could always purchase a tumbler later if mixing is too much work, but I'm sticking with, "free is good". Piles do benefit from some containment. Wire fencing (especially the 1x2" grid type), pallets tied together, scrap wood constructs, hay/straw bales, and junk tires all work well for a compost pile. Personally I like to use tires with the sidewalls cut out as they can be moved one ring at a time when turning the pile. This is much less work due to better ergonomics. In addition, the tire rings can double as potato planters to increase size and yield, but that's a whole 'nother topic.
I have experimented using a washer and dryer drum for composting. I think a pair of dryer drums could make a decent tumbler if welded together, but the washer drum was just too small to be effective. The washer would make a better planter than anything else.