You are correct to receive the highest degree of Celestial Kingdom which is to receive exaltation, requires Baptism, Confirmation, Endowment, and Eternal Marriage.  
 
The Scriptures tell use that this is the case and doesn't offer much explanation. The Living Prophets have told us that Marriage between a Woman is ordained of God. The Lord commands man and woman to marry and replenish the Earth.  
 
In my opinion what it ultimately comes down to is that without a marriage and having to spend a lifetime cooperating and serving your spouse you are deficient spiritually. It is easy to serve someone on your own terms, but marriage is a 24/7 365 day a year service project and it is never on your own terms. It bring humility and charity into your life. Whatever the lords reason may be is unknown really, but that is my opinion on the subject.

Here is what the scripture says: 
 
"In the celestial glory there are three heavens or degrees; 
And in order to obtain the highest, a man must enter into this order of the priesthood [meaning the new and everlasting covenant of marriage]; 
And if he does not, he cannot obtain it. 
He may enter into the other, but that is the end of his kingdom; he cannot have an increase." (D&C 131:1-4) 
 
Without celestial marriage, a person may enter the Celestial Kingdom and live with Heavenly Father, but they will not be able to enjoy all that exhaltation has to offer. True exhaltation involves having an eternal posterity. That is, I believe, what the "increase" is that is mentioned above. You cannot have children in heaven without a spouse. Just as Adam needed Eve, each of us, in order to reach our full potential, must receive this covenant of marriage. Those who do not have a fair opportunity to receive this for themselves in this lifetime will probably have the opportunity during the Millenium.

I understand what you are saying but it doesn't seem fair. If I am a loyal Mormon, Baptized in the church, and put others before my self I still can't return to Heavenly Father just because I wasn't married.

I think you've misunderstood what was said. You can be with your Heavenly Father,you just can't be like him. Heavenly Father is married. If he weren't, he could not have spirit children such as us. To be like him, and to be able to have spirit children like him, you must also be married. It's not a punishment. 
 
Also, fairness implies unequal opportunity. We all have or will have the same opportunity. Some may choose not to marry. In that case, their state is just because it was their choice.

As long as I spend the rest of eternity with our Heavenly Father then I think i'll have all I need to be happy. I don't have any doubts that I will be married sometime in my life, but then again I never know.

All will have the oppertunity to have an eternal marriage if they desire so. Whether that marriage happens in this world or the spirit world is up to the Lord, but everyone will have the opportunity to receive all the ordinances of salvation including eternal marriage.

It almost feels like the marriage part was introduced to create growth in the church? Do you really think God is married? I guess he could be, but I never have read that in scripture.