Looks like Asian Bush Honeysuckle. Roundup will kill it. That stuff spreads like wildfire and kills of all the undergrowth eventually by shading it to death.
It's Nightshade. Some varieties are edible, some are not. The poisonous varieties can be used to make Belladonna. There is lots of confusion between the varieties so I wouldn't mess with it. Especially if it has red berries. Kill it with Roundup, Spectricide or Finale.
It's Bitter Nightshade, aka blue bindweed, aka bittersweet. I'd hit it with whatever kind of herbicide you have and let it die before pulling it up. If you collect the berries and throw them away, it will prevent them from falling to the ground and germinating next spring (wash your hands after, no big deal).
The reason it's growing below your fence is that a bird ate a berry and was sitting on your fence when he pooped out the seed. Same way that poison ivy, raspberry, mulberry, etc are often found in fence-rows.