Part of credit score is demonstrating a history of paying off debt. I never heard of that lowering one's FICO score; in fact, a good way to lower your score is to lower your credit usage/credit availability ratio. Applying for and using a new credit card, for example, temporarily lowers your credit score, because average account age is lowered. But as long as you don't go crazy and carry balances, your low credit usage really bumps up your score. We applied for an Amazon Visa, which we only use on Amazon for the 5% cash back, and pay off monthly. We were approved with a ridiculously high line of credit. Within 2 months, our FICO score jumped 20 points. I found out that having an excellent credit rating not only impacts on available loan rates, but cars insurance as well.A credit score is essentially your "I love debt score" and shows how much you like playing kissy face with the bank. Paying down debt lowers your score because the calculation rewards you for borrowing lots of money.
Stop playing the game and stop borrowing money. Then you won't ever worry about your score again.