The X boss said users will still be able to block people from directly messaging them, however.
But many people on social media said it will make it hard for people to remove abusive posts from their timeline.
It is the latest in a series of changes Mr Musk has made since taking over the site in a $44bn deal last year.
Currently, when users "block" an account, it stops that account's posts from appearing in the blocker's timeline, and vice versa.
An account that is blocked can no longer send messages to the blocker, nor can it view their posts.
Former Twitter founder, Jack Dorsey, seemed to agree with Mr Musk's decision, posting: "100%. Mute only".
But there are concerns that muting an account would not be sufficient protection from cases of harassment, abuse or stalking.
The mute function currently only stops notifications about an account's posts. An account that is muted can still view the muter's posts and reply to them.
One user called Mr Musk's decision a "huge mistake", saying there are "toxic people" on the platform whom users simply did not want to interact with in any way.
Removing a blocking feature could also potentially violate the terms and conditions of stores like Apple's App Store and Google Play.
Both stores have conditions stating that social media apps should allow users facilities to filter harassment or bullying.
It could mean X is no longer downloadable from those stores.
If the policy goes ahead, it is not clear if all those accounts which are blocked will automatically become unblocked.
Users do however have the option to make their account private, hiding their tweets from the public and only allowing accepted followers to view their posts.
Elon Musk, the richest man in the world, made a series of changes when he took over the social media site, including sacking the company's top executive team and introducing a charge for the site's "blue tick" - or verification - feature.
Elon Musk is a prolific poster on X, and he's well known for not always being serious or following through on the many ideas he throws out to his 153 million followers.
X itself rarely responds to journalist queries so it's difficult to verify anything he states on behalf of the firm. But, as its owner, he's by default a significant, if unreliable, source.
The block button is an established tool for those who feel attacked, bullied or simply want to shut out an account with whom they have a strong disagreement (and X is full of those).
Report an account and one of the first bits of advice you get is to either block or mute it while it is investigated. That's not unique to X.
Muting an account means you don't see it - but it still sees you. And being forced to remain visible to someone you are trying to avoid or feel afraid of seems like an unusual move.
Musk has been clear that he wants his "digital town square" to be a platform where all voices are heard, but he's running the risk of bumping up against both app store terms and conditions and social media regulations around protecting users from online harms.