I generally shy away from downvoting even things I dislike. I don’t want to fall into an echo chamber where all I see are only the things I agree with.

This is not how Lemmy works. If you don’t ever up or down vote anything, and if you have your client hide the up/down votes on all comments and posts, you will still see the same content as someone who prodigiously votes. Votes here are, in the truest possible way, optional.

But for people who do utilize up/down votes, they are a signal. Sometimes they’re a weak signal, sometimes a bad signal, and yet other times might be a strong signal from the community. How to evaluate the signal is a matter of continued debate.

Now, as an aside, a major difference in the Fediverse compared to commercial social media networks is that the block functionality here is used for more than just dodging abusers. Precisely because there is no algorithm, blocks and mutes are how users curate what they don’t want to see. This alone does not create a so-called online echo chamber, the same way that most people don’t want to see a steaming pile of poo on their morning walk. People reasonably deserve to have a pleasant experience online. And depending on their life circumstances, this might be a lot better than their IRL situation.