Do Muslims Worship Ka'aba?

Islam came to preach the worship of Allah the Only One, and the abandon of worshipping whatever else "And they have taken to themselves gods, apart from Him, that create nothing and themselves are created and possess for themselves, neither harm nor profit; neither do they possess death, nor life, nor uprising. (i.e. resurrection)." (AL-Furqan: 3 )
Allah Says : "And in no way did We send (any) Messenger even before you except that We revealed to him, (saying), "There is no god except I; so worship Me." (Al Anbiya: 25 )
Ibrahim (Peace be upon him) admonished his father about worshipping idols saying :" And as Ibrahîm (Abraham) said to his father Éazar, "Do you take to yourself idols for gods? Surely I see you and your people in evident error." (Al-Anaam : 74 ) ; " As he said to his father, "O my father, why do you worship that which neither hears nor beholds, nor avails you anything? " (Maryam : 42 )
About those idols and stones that were worshipped Allah says: "Do they have legs with which they walk, or (even) do they have hands with which they assault, or (even) do they have eyes with which they behold, or (even) do they have ears with which they hear? Say, "Invoke your associates, thereafter plot against me, (and) then do not respite me." (AL-Araf : 195 )
The Prophet (Peace be upon him) was asked : "What is the greatest sin ? " , he answered " to make peers to Allah although He created you ".
The first thing The Prophet (Peace be upon him) made after his victorious return to Makka was breaking all the idols in and around Kaaba while uttering the verses "And say, "The Truth has come and untruth has expired; surely the untruth has been certain to expire." (AL-Isra : 81 ).

After all this; how could any one assume that Muslims worship Kaaba??
Kaaba is just a Qiblah (Direction) for Muslims to their faces towards it during prayers as a form of target unification. They visit it, circumambulate around it submitting to the orders of Allah. Definitely they know its just stones that has no good and no harm in itself; but as Muslims they have to submit to Allah's orders even if they can't reason the wisdom behind that.
Quoted from here with some modifications