Small Plastic Anchor Holes
The most common anchor holes are from those little plastic inserts that split when you drive a screw into them. They leave holes around 1/4 inch diameter when removed.
First, get the anchor out. Sometimes they pull out easily with pliers. Sometimes you need to thread a screw in partway and pull. Sometimes, honestly, I just hammer them deeper into the wall and fill over them. If it's sitting flush or below the surface, it's not causing problems.
Fill with lightweight spackle. Press it in firmly, wipe off the excess. The spackle will shrink slightly, so slight overfilling is fine. Let it dry, add a touch more if needed, sand smooth.
Prime before painting if you're doing a full repaint. For touch-ups, you can often get away without primer if you feather the paint carefully.
Toggle Bolt and Large Anchor Holes
Toggle bolts leave big holes. The wings that provide the grip are larger than the hole they go through, so when you remove the bolt, a 1/2 to 3/4 inch hole remains. Some heavy-duty plastic anchors leave similar sized holes.
These are too big for spackle alone. The spackle will sag into the void and crack when it dries.
Method 1: Mesh Patch
Self-adhesive mesh patches work great for toggle bolt holes. Get a patch larger than the hole by at least an inch on all sides. Stick it over the hole and cover with joint compound, not spackle.
The mesh bridges the void and gives the compound something to grab. Two or three thin coats, feathered out a few inches, and you can't tell there was ever a hole there.
Method 2: Plug and Fill
For holes that are round and clean, I sometimes plug them before filling. Take a small piece of crumpled newspaper or paper towel and push it into the hole to create a backing about 1/4 inch below the surface. Then fill with setting compound.
The paper gives the compound something to sit on so it doesn't fall into the wall cavity. Once it sets, the second coat covers the plug completely.
Enlarged Holes from Anchor Failure
This is worse than just removing an anchor. When an overloaded anchor pulls out, it doesn't leave a clean hole. It tears out a chunk of the drywall core, leaving an irregular crater with crumbled edges.
First, clean out the loose material. Pick and scrape until you're down to solid drywall. The hole will get bigger during this step. That's okay. Solid edges make better repairs than crumbly ones.
Depending on final size, either use a mesh patch or the California patch method. For anything over about 2 inches, consider cutting out the damaged area and installing a proper drywall patch.
My garage had a shelf that pulled out of the drywall after years of holding way too much weight. The damage was about 4 inches across when I finished cleaning it up. Cut it out to a square, installed a patch with backing, taped and mudded like new work. Couldn't have just filled that one.
Preparing for New Anchors in Same Location
Sometimes you need to put a new anchor exactly where an old one failed. The drywall there is compromised, but you need that specific spot.
Fill the old hole with setting compound, not spackle. Setting compound is stronger and won't compress like spackle can. Let it cure completely, usually 24-48 hours for full strength even though it's dry to touch much sooner.
Then drill your new hole through the hardened compound. It'll grab an anchor almost as well as fresh drywall. Not quite as strong, but adequate for most applications.
For heavy loads, add a larger backing plate to distribute force. Or better yet, find a stud to anchor into instead of relying on drywall anchors at all. Lesson I've learned repeatedly: drywall anchors have limits. Studs don't.
Special Cases
Anchors Still in the Wall
For plastic anchors that won't budge, I often just leave them and fill over. As long as the anchor is flush or below the surface, it's not causing problems. Hammer it in a bit if needed, then fill the remaining dimple with spackle.
Metal anchors are trickier because they might rust. If you can't remove a metal anchor, at least seal it with a bit of compound to prevent rust bleeding through your paint.
Holes Near Electrical
Be aware of what's behind the wall. If an anchor hole is near a switch or outlet, there might be wires in the cavity. Don't shove mesh or backing material in blindly. Use a flashlight to check before pushing anything into the wall.
Textured Walls
Matching texture over repairs is its own skill. For anchor holes, the patch area is small, so texture mismatch is very visible. Either texture to match, which might require practicing on scrap, or consider repainting the whole wall to make the slight texture difference less obvious.
Preventing Future Anchor Problems
Most anchor failures happen because the wrong anchor was used for the load. Those little plastic anchors that come free with shelving are good for maybe 15 pounds. People load 50 pounds on them and wonder why they pull out.
Match the anchor to the load with some safety margin. For anything heavy, toggle bolts or snap toggles are much stronger than expanding plastic anchors. For really heavy stuff, find the studs.
Pre-drill to the right size. Too small and the anchor won't seat properly. Too big and it won't grip. The anchor package tells you what drill bit to use. Actually use that size.
I keep a variety pack of anchors in the garage now. Different sizes and types for different applications. Costs about $20 and saves the frustration of using whatever's on hand when the right anchor makes the job work properly.