Here's the thing, you're the only person who can choose to give up and let the air out of your own balloon. Allowing the opinions or the discouraging chatter of others should never dictate that decision. Most of us have projects, dreams and life visions that we have launched headlong into. It takes passion along with dedication to keep our visions and dreams in motion. It also takes equal parts encouragement to press on. Especially when things become a bit scary and we feel all alone in it.
You're not the only one battling with personal discouragement. There's an entire bleacher full of people like you pushing forward in spite of feeling a bit deflated. It's part of the process. I'm not saying don't let go if you feel confirmation in your heart that it's time. What I am saying is don't let go for the wrong reasons. Feelings change, they will pass.