so what else is a factor for reputation? Or is it like if you dont pay to get your mail-domain whitelisted we lower your reputation score?

ip-reputation is also important. Mailgun, an email service for mass mailing, is doing an „ip-warmup“ if you choose a dedicated ip. So, if you are self-hosting with dynamic-ip, i think you would have a very very low ip-reputation.